Exit [ DataStage ] - and retain its business processes

 

Exit  [ DataStage ] - and retain its business processes When we talk about leaving DataStage (or another ETL tool), we often think of "technical migration." While this is certainly a thorny issue, it's an oversimplification. In reality, the real challenge lies (primarily) elsewhere.   In simplified terms, DataStage (or other ETL tools) contains two essential elements at the heart of a company's collective intelligence: Data that we will call  " information assets" , Transformation flows that carry business rules and transform this data, which we will call  "functional assets" .   The risk of a poorly prepared migration is simple: We will change tools… but we will lose or degrade all the intelligence built up over a long period in business processes. Mapping the data and transformations The first step is therefore to offer everyone the opportunity to understand these two assets (informational and functional) before any migration project. 1- Understanding the data: the "informational heritage" We propose to do this a mapping of flows and dependencies, from a data point, to visualize upstream sources, intermediate tables and downstream uses - all technologies combined (DataStage, other ETL /ELT, procedural languages, etc.). This overview makes it possible to reconstruct the real informational heritage, beyond a particular tool.   2- Understanding the transformations: the “functional heritage”  We propose for this purpose a mapping of transformations and business rules which allows you to navigate in the processes and zoom in on a transformation node (here in a DataStage job) in order to visualize the parameters, the rules applied, the dependencies, etc. The rules are thus inventoried, documented and linked to the data flows that carry them.  Leaving DataStage, but to go where? Once these processes have been identified, it seems relevant to translate them into an open language, readable by the teams: the choice of SQL is obvious. For what ? It's a universal language - understood by most data teams, It is a timeless language – present in all modern platforms with only minor adaptations. It's a portable language – processes are therefore no longer confined to a single tool. In these times of geostrategic tension, this makes sense ("interoperability").           With SQL, we can faithfully reproduce DataStage logic:  filters, transformations, joins, CDC, before/after management, controls… without loss of business rules, 1 for 1. Demo film (5'): Mapping your flows, exiting DataStage  A quick and tangible ROI Highlighting DataStage flows and processes is not just for preparing a migration. The benefits are immediate: Removal of treatments that have become unnecessary. Simplification of the data architecture, Reduction in operating costs and… licensing costs  .   If the goal is to move away from DataStage, then the ROI becomes considerable:  partial or total elimination of a technological dependency, more readable processes and an open foundation for today's and tomorrow's modern data architectures.   Migrating effectively: 28-week Project Plan (indicative)  Before discussing migration, we must first make visible what DataStage actually contains: The source data requested, the target data supplied (information assets),  the business flows and processes that transform them. Once revealed, these processes can be preserved (if they have value) and migrated to more open architectures to unlock the intelligence they contain. We believe SQL is a particularly suitable target. PS: same for Genio, Talend, BODS, Informatica, etc.