Teradata to Snowflake Migration at Major Retailer

About Customer

Intricity worked with a private department store based on the East Coast of the US with just under 300 stores located in 16 states and a growing digital presence. The retailer has a wide assortment of national brands and private label fashion apparel, shoes and accessories for all age groups along with top name cosmetics, a wedding registry, and a large selection of quality merchandise for the home.

About Intricity

Intricity is a team of specialized Data Management, Data Warehousing, and Business Intelligence experts. The team members at Intricity have been handpicked over the course of 20 years, and represent the top talent globally in Data oriented disciplines.

Challenge and Wins

Challenge

The retailer had been a Teradata customer for 8 years. Over time the query speed of the retailer’s Teradata environment degraded, resulting in unacceptable turn around times for data science queries. The situation weighed down on the relationship between business and IT. Additionally, the retailer was faced with the constraints of an ever encroaching Amazon footprint in the retail space. The retailer needed better inventory and customer analytic performance to ensure that they could compete with Amazon’s market presence. They decided to engage in a digital transformation which clearly Teradata was not capable of supporting. An evaluation was set up to test Snowflake against the same Monday morning stresses that Teradata had to endure. The results were a “no brainer” for them. Later in the sales cycle Google BigQuery got evaluated, but they failed to produce the same level of simplicity and simplicity of conversion.

The retailer’s Teradata instance had 8 years of established integration code. To get their arms around the effort they produced complexity counts around the content currently connected to Teradata like utilities, initial load, ingestion, transfer & scheduling, tables, views, and data volumes. Intricity leveraged these statistics to produce a fixed priced “lift and shift” migration proposal.

Navigating Constraints

The retailer’s original Data Warehouse technology stack was migrated to Teradata in 2010. This original migration brought a lot of legacy code which was written for that original deployment and was at least 20 years old. However, this code was being leveraged in Teradata in 2018. The code was written in a combination of SQL, Job Information Language, and Shell Scripts. The complexity of this integration was very high as it was deployed with extensive logging, self-healing routines, and code segments some more than 20 years old.

Over the 20 year period many code segments, and their relationship to the larger integration picture, had been forgotten. Thus the migration of this customer’s deployment required an ability to drill into the hidden complexities within the code to ensure the migration completed effectively. This was especially the case with the code that was migrated into Teradata from their previous legacy environment, much of which was not even being used. This code was mixed in with all the current code but no determination was available on whether these legacy code segments were being used.

Win 1:

Intricity leveraged BladeBridge to automate the migration of 80% of the SQL, JIL, Shell Scripts, and Teradata code to Snowflake by using its automated tooling. The tooling provided a cycle where code patterns could be batched, converted, tested, and further reviewed for erroring code patterns. This cycle repeated itself until all the patterns had been migrated.

By leveraging BladeBridge's automation, Intricity was able to zero into the code segments that were strictly unique or required deeper individual validation.

Win 2:

In addition to migrating the code to work in Snowflake, Intricity also was able to isolate and retire code segments that were not being presently used. This enabled the retailer to emerge with a much cleaner code base than they originally had.

Call to Action

Intricity has created a repeatable framework for conducting these migrations which has a proven track record. This framework is used not just for Teradata migrations but for any source system to Snowflake. For a short description of how this framework is specifically used with Teradata see Intricity’s Teradata to Snowflake Migration slick.