Accelerated implementation of a hybrid cloud-based data lake with LeapLogic
88% auto-transformation of Teradata BTEQ queries in 50% less time compared to manual migration
A Fortune 500 multinational bank wanted to migrate their enterprise data warehouse from Teradata to a modern data platform with the following objectives:
- Enhance processing capacity and manage increasing volumes of data
- Reduce maintenance cost of existing data warehousing and analytics platforms
- Reduce validation time for baseline data issues, data quality, and other Teradata query gaps
- Replicate the existing data ingestion mechanism on the target environment
LeapLogic – an Impetus product for automated workload transformation identified and offloaded data and workloads to a hybrid big data-based cloud solution. LeapLogic automatically converted Teradata SQL queries into equivalent Spark QL/Hive QL and executed them on the Hadoop/Hive environment. It also converted the business code into equivalent Python code and allowed users to run a set of data validation checks.
The transformed queries were validated both syntactically and semantically to ensure 100% precision. Finally, the post-processed analytical data was loaded back to the source enterprise data warehouse for reporting, access, analytics, and providing recommendations.
- Auto-transformed 1203 out of 1374 BTEQ queries with 87.55% success
- Converted Hive schema with 104 tables and 300 GB of compressed data
- Generated campaign data and reports on Hive to match the Teradata output
- Packaged and executed the core process for CDH production deployment
- Recommended performance tuning and cluster sizing for the production environment
- Performed semantic validation for all the transformed queries based on an indigenous query validation engine. LeapLogic generated diverse sample datasets and compared data on the source and target for different use cases.
LeapLogic helped the client move away from Teradata and realize the following benefits:
- Saved 50% time compared to manual migration
- 88% auto-transformation of Teradata queries
- Lowered annual Teradata licensing cost
You may also be interested in…