The new Cloud data pipelines: The principles of ELT and lots of SQL
The new Cloud data pipelines : The principles of ELT and lots of SQL ! For years, ETL tools like BODS, DataStage, Informatica, Ab Initio, and Talend dominated the data pipeline building landscape. This presented a window of opportunity: The hardware constraints and limited performance of "on-premises" Data Warehouses suggested that transformations be moved to third-party servers, hence the lesser success of ELTs (Oracle Data Integrator, ODI are examples). Companies wanted to democratize the creation of data pipelines with business-oriented tools. But cloud architectures have changed the game, particularly due to the power of cloud data warehouses. Are we witnessing a massive, but SQL-boosted, revival of the principles that were at work in ELT? If so, how can we be part of this dynamic? Why are companies abandoning E & L logic in the Cloud? Historical ETL technologies revolu...