• July 24, 2021

Leveraging Open Source & Hybrid Cloud for z/OS: A Rube Goldberg Evolution to COTS Value

This content was originally presented at the inaugural Open Mainframe Project Summit in September 2020.

I’ve been working on the mainframe for over 50 years, lately focusing on the open source capabilities of IBM’s proprietary operating systems. Throughout the years, I’ve noticed that businesses have three main goals for their mainframe environment:

  1. Get off the mainframe: Some businesses want to get off of the mainframe and do it as expeditiously as possible. In this case, they want to transfer their data to a new system and keep it secure.
  2. Protect the mainframe: Other businesses want to keep using the mainframe in the same way as always, but they aren’t looking to invest in it. They want to ensure their data remains secure.
  3. Leverage the mainframe: Many businesses believe that the mainframe is a critical component of their IT infrastructure. They are looking to modernize, enhance and leverage it for the best business value.

Regardless of the businesses’ perception of the mainframe, open source and cloud technologies can be leveraged in each of these environments to help satisfy business goals. Yet, most businesses still don’t seem to understand how cloud and the mainframe can work together to get results. When I talk to mainframe customers, there are generally four capabilities that they want in their cloud experience: end user experience, production application, data lake and open source dev ops tooling. Unfortunately, all of these components are individually provisioned. There’s no such thing as all of these components in a single operating environment, like z/OS. The customer might leverage any one of the popular public cloud mechanisms, but they’ll get a cloud instance that does a single task. For scale and disaster recovery, they have multiple cloud instances working together to meet their service level goals. In this post, I will introduce applications and capabilities that work as both open source or cloud-ready solutions for these common problems.  

Data Lakes

It’s important to remember that the mainframe is not an island. It requires a great system of engagement for the end user interface. An enlightened company will realize this and find a way for the mainframe, cloud technology and open systems to collaborate for overall success.  Yet not all IBM Z systems are legacy systems and not all legacy systems run on IBM Z, which can inhibit the ability to modernize. 

The crown jewel of the mainframe is its ability to host data as the system of record, and provide the transaction processing to update and manage that data. So regardless of the businesses’ intentions, access to that data is a critical success factor. There are two primary ways to ensure data access. The first way is data replication or copying it onto the cloud environment. The other way is using data virtualization or sharing the data to the cloud environment.  

Think about this new data access as a data lake. There are multiple types of data that can be consumed, and these might come from a variety of sources. With replication, data is copied from a source and may be merged with other data to build this lake. By using data virtualization, it looks to share data by creating a close-to-real-time model.

There are many technology options available through IBM to help build data lakes, including Cloud Pak for Data, Data Virtualization Manager for z/OS and Datagate. Regardless of which technology is leveraged, a wide range of system z and non-system z data can be leveraged.

Accessing Data 

Analytics applications can be leveraged to access the data stored in the lake and to garner insights from the data. Using the 80/20 rule, if the majority of the data is resident on z/OS, then you should host the analytics application there as well. Programs like Spark, or programming languages like Python are regularly used to build these analytic applications. These applications can also run on z/OS and provide faster time to value by operating locally against the resident data. 

Back-Up, Manage and Secure Data 

Although disk drives have been used to back up data for years, there are two IBM products that have been introduced to help: IBM Cloud Tape Connector for z/OS and IBM Advanced Archive for DFSMS/hsm. These enable the cloud to be a target for data backup, archive and migration. They are also built with security encryption in mind, so that the data is protected on every step of the journey.

Legacy passwords and user IDs are no longer up to industry standards or regulations. These now mandate multi-factor authentication (MFA) for security. MFA combines something you know, something you have and something you are, biometrics, to securely access accounts and data. With the IBM Z MFA offering, you can incorporate MFA into a variety of third-party vendor solutions, including those that are cloud or open source hosted. Although you will never achieve single sign on for all platforms, you can have a consistent sign on to the mainframe and other systems through the businesses’ IT infrastructure. 

User Experience

There’s a lot being done to improve the user experience on a mainframe. Among them is the use of Zowe technology by middleware products and the operating system. The overall goal is to reduce the effort it takes to complete tasks, reduce training time and simplify legacy tasks and experiences. There are many other open source tools available to accomplish similar goals in a way that’s consistent across the mainframe and other platforms. These tools can help with performance analysis and security alert auditing, diagnostics, system image issues, and more. 

Open Source and Cloud for the Mainframe

Regardless of the businesses’ goal—get off the mainframe, stabilize the mainframe or leverage the mainframe—open source and cloud technologies can help them speed up their desired transitions. Businesses can grow their overall IT skills in a synergistic fashion across platforms. 


Jim Porell 9 Posts

I am a Solutions Architect at Rocket Software, focusing on pre and post sales technical assistance for Rocket developed products from IBM. Prior to joining Rocket, I was an independent consultant and retired IBM Distinguished Engineer. I held various roles as Chief Architect of IBM’s mainframe software and led Business Development and marketing of Security and Application Development for the mainframe. My last IBM role was Chief Business Architect for Federal Sales. I held a TS/SCI clearance for the US Government, was a member of the US Secret Service Electronic Crimes Taskforce in Chicago and co-authored several security books. I've done cybersecurity forensic work at a number of Retail, Financial and Government agencies and created a methodology for interviewing customers to avoid security breaches for large enterprises. I have over 40 years working with Information Technology.


Leave a Comment

Your email address will not be published. Required fields are marked *