How can analytics pipelines be made portable?
How can analytics pipelines be made portable? Today, major cloud platforms offer extremely powerful data environments that are generally the target in data stack modernization projects: Microsoft Fabric, Databricks, Glue, Snowflake, BigQuery, etc. These platforms are very well suited to many analytics and Big Data uses. But in many projects, another question is beginning to emerge: how to ensure that a technological choice made today does not create a perpetual dependency? Behind the processes often lie specific runtimes, proprietary APIs, native authentication mechanisms and, more generally, strong dependencies on the chosen Cloud provider. It also concerns his own informational sovereignty. In short... A portable and interoperable approach For several major clients, we are working on approaches aimed at preserving the portability of data pipelines and limiting their dependence on a specific environment. The idea is to keep SQL processes open, more ea...