• January 28, 2022

Leveraging Open Source & Hybrid Cloud for z/OS: A Rube Goldberg Evolution to COTS Value

This content was originally presented at the inaugural Open Mainframe Project Summit in September 2020.

I’ve been working on the mainframe for over 50 years, lately focusing on the open source capabilities of IBM’s proprietary operating systems. Throughout the years, I’ve noticed that businesses have three main goals for their mainframe environment:

  1. Get off the mainframe: Some businesses want to get off of the mainframe and do it as expeditiously as possible. In this case, they want to transfer their data to a new system and keep it secure.
  2. Protect the mainframe: Other businesses want to keep using the mainframe in the same way as always, but they aren’t looking to invest in it. They want to ensure their data remains secure.
  3. Leverage the mainframe: Many businesses believe that the mainframe is a critical component of their IT infrastructure. They are looking to modernize, enhance and leverage it for the best business value.

Regardless of the businesses’ perception of the mainframe, open source and cloud technologies can be leveraged in each of these environments to help satisfy business goals. Yet, most businesses still don’t seem to understand how cloud and the mainframe can work together to get results. When I talk to mainframe customers, there are generally four capabilities that they want in their cloud experience: end user experience, production application, data lake and open source dev ops tooling. Unfortunately, all of these components are individually provisioned. There’s no such thing as all of these components in a single operating environment, like z/OS. The customer might leverage any one of the popular public cloud mechanisms, but they’ll get a cloud instance that does a single task. For scale and disaster recovery, they have multiple cloud instances working together to meet their service level goals. In this post, I will introduce applications and capabilities that work as both open source or cloud-ready solutions for these common problems.  

Data Lakes

It’s important to remember that the mainframe is not an island. It requires a great system of engagement for the end user interface. An enlightened company will realize this and find a way for the mainframe, cloud technology and open systems to collaborate for overall success.  Yet not all IBM Z systems are legacy systems and not all legacy systems run on IBM Z, which can inhibit the ability to modernize. 

The crown jewel of the mainframe is its ability to host data as the system of record, and provide the transaction processing to update and manage that data. So regardless of the businesses’ intentions, access to that data is a critical success factor. There are two primary ways to ensure data access. The first way is data replication or copying it onto the cloud environment. The other way is using data virtualization or sharing the data to the cloud environment.  

Think about this new data access as a data lake. There are multiple types of data that can be consumed, and these might come from a variety of sources. With replication, data is copied from a source and may be merged with other data to build this lake. By using data virtualization, it looks to share data by creating a close-to-real-time model.

There are many technology options available through IBM to help build data lakes, including Cloud Pak for Data, Data Virtualization Manager for z/OS and Datagate. Regardless of which technology is leveraged, a wide range of system z and non-system z data can be leveraged.

Accessing Data 

Analytics applications can be leveraged to access the data stored in the lake and to garner insights from the data. Using the 80/20 rule, if the majority of the data is resident on z/OS, then you should host the analytics application there as well. Programs like Spark, or programming languages like Python are regularly used to build these analytic applications. These applications can also run on z/OS and provide faster time to value by operating locally against the resident data. 

Back-Up, Manage and Secure Data 

Although disk drives have been used to back up data for years, there are two IBM products that have been introduced to help: IBM Cloud Tape Connector for z/OS and IBM Advanced Archive for DFSMS/hsm. These enable the cloud to be a target for data backup, archive and migration. They are also built with security encryption in mind, so that the data is protected on every step of the journey.

Legacy passwords and user IDs are no longer up to industry standards or regulations. These now mandate multi-factor authentication (MFA) for security. MFA combines something you know, something you have and something you are, biometrics, to securely access accounts and data. With the IBM Z MFA offering, you can incorporate MFA into a variety of third-party vendor solutions, including those that are cloud or open source hosted. Although you will never achieve single sign on for all platforms, you can have a consistent sign on to the mainframe and other systems through the businesses’ IT infrastructure. 

User Experience

There’s a lot being done to improve the user experience on a mainframe. Among them is the use of Zowe technology by middleware products and the operating system. The overall goal is to reduce the effort it takes to complete tasks, reduce training time and simplify legacy tasks and experiences. There are many other open source tools available to accomplish similar goals in a way that’s consistent across the mainframe and other platforms. These tools can help with performance analysis and security alert auditing, diagnostics, system image issues, and more. 

Open Source and Cloud for the Mainframe

Regardless of the businesses’ goal—get off the mainframe, stabilize the mainframe or leverage the mainframe—open source and cloud technologies can help them speed up their desired transitions. Businesses can grow their overall IT skills in a synergistic fashion across platforms. 


Jim Porell 9 Posts

I am a Solutions Architect at Rocket Software, focusing on pre and post sales technical assistance for Rocket developed products from IBM. Prior to joining Rocket, I was an independent consultant and retired IBM Distinguished Engineer. I held various roles as Chief Architect of IBM’s mainframe software and led Business Development and marketing of Security and Application Development for the mainframe. My last IBM role was Chief Business Architect for Federal Sales. I held a TS/SCI clearance for the US Government, was a member of the US Secret Service Electronic Crimes Taskforce in Chicago and co-authored several security books. I've done cybersecurity forensic work at a number of Retail, Financial and Government agencies and created a methodology for interviewing customers to avoid security breaches for large enterprises. I have over 40 years working with Information Technology.


  • Abe Kornelis Reply

    May 23, 2021 at 6:13 am


    thanks for sharing your thoughts.

    Yet there is one small detail on which I’d like to comment: you’re using the word ‘legacy’ without further clarification. I think this imposes a risk of being misunderstood: many have negative associations with it. For me however, legacy associates with vinted & valuable, tried & trusted, proven & dependable. Your colleague Lilac Schoenbeck comments likewise on the word legacy in https://blog.rocketsoftware.com/2020/10/why-legacy-shouldnt-be-a-bad-word/

    Vinted & valuable software is the software that (usually) has been running a company’s core business for many years. Read: it is part of the company’s administrative foundation, helping it earn its bread & butter. You know that; I’m aware I’m not telling you anything new. Such software also has seen significant investments over the years and deserves a much better connotation than it usually gets from the word legacy.

    I do not object to the word legacy itself – but I do think we are all well advised to make explicit what positive connotations we associate with it. Not doing so leaves the readers free to apply their own preconceived associations to the term. You may end up being misread and misunderstood. Which would be truly sorry, for you have a valuable and insightful message to share.

    Kind regards,
    Abe Kornelis

  • Jim Porell Reply

    August 19, 2021 at 10:39 am

    Hi Abe, my apologies for the delay in a response to you. I agree with you and Rocket’s tag line “Legacy loves Legendary”. But I did use the term legacy in the post a couple of times. The important use was this one:
    “Yet not all IBM Z systems are legacy systems and not all legacy systems run on IBM Z, which can inhibit the ability to modernize. ” Translation -some customers treat it as old and want to get off. But so many use the mainframe as the core of their business and are re-vitalizing it, while moving away from racks of poorly constructed alternative servers.

    In the other instances, I was referring to outdated actions, like authentication with only “legacy” userids and passwords. Businesses need to move to multi-factor authentication asap as it is far too easy to hack passwords off of desktops. And there was another “old” reference to tasks that can be modernized.

    I’ll go back to a comment from the 1990’s from the IBM General Manager of the mainframes then. His comment was something on the order of: “Others are calling the mainframe a dinosaur. That’s something we should embrace. The dinosaur lasted 50 million years. We should last that long too!” 🙂

Leave a Comment

Your email address will not be published. Required fields are marked *