Case Study

20% SLA improvement by modernizing Teradata workloads

Fortune 500 global enterprise technology provider achieves 90% automated transformation


Business needs

A global enterprise technology provider for banking, retail, and hospitality industries wanted to modernize their Teradata workloads to Spark Scala for reducing maintenance costs and enhancing scalability and agility. In addition, they were looking to resolve the mismatch in the metadata definition of their enterprise data warehouse and enterprise data lake. To address this, they wanted to leverage a modern data processing engine that would deliver analytics faster.

 

Auto-migrated 1860 BTEQ scripts along with ~5000 SQLs and 64 TB data

 

Solution

The Impetus team partnered with the enterprise technology provider to help them migrate one of the most complex finance applications to a modern data platform. The team leveraged LeapLogic, an Impetus product for automated workload transformation to auto-migrate 1860 BTEQ scripts along with ~5000 SQLs and 64 TB data from Teradata to Spark Scala to run on Hadoop in four steps – assessment and prescription, transformation, validation, and execution.

 

Achieved ~90% auto-transformation of BTEQ scripts

 

One of the requirements was to execute the transformed workloads within 6 hours, for which the Impetus team:

  • Optimized the workloads by breaking bigger sequential scripts/jobs into parallel executable scripts to meet business SLAs
  • NCE_casestudy_screenshot1
    Example of scripts that can be orchestrated in parallel
  • Used a single Spark session to execute all the scripts of a specific job to reduce overall execution time
  • Recommended target cluster sizing for the production environment
  • NCE_casestudy_screenshot1
    Sample assessment recommendations

LeapLogic was also used to:

  • Migrate code to Spark Scala programs and execute the code on the modern data platform
  • NCE_casestudy_screenshot2
    Screenshot of code execution on the modern data platform
  • Create a mapping sheet to map columns between enterprise data lake and data warehouse for resolving the metadata mismatch
  • Automate unit testing for data and logic migration
  • Perform cell-level validation to ensure matching data post migration and processing
  • NCE_casestudy_screenshot3
    Cell-level data validation post migration

Impact

  • Achieved ~90% auto-transformation of Teradata BTEQ scripts
  • Executed the migrated code in 5 hours, beating the targeted 6-hour SLA
  • Saved 50% cost compared to manual transformation
  • Ensured zero business disruption

LeapLogic helped the enterprise technology provider rapidly transform workloads from Teradata to Hadoop. The solution simplified and accelerated the transformation process with speed, reliability, and minimal coding, while avoiding business disruption.


You may also be interested in…