Case Study: Retiring On-Prem Systems & Migrating to Snowflake

About Client

The Client is one of the top most recognized American insurance companies and has serviced countless home and car owners since the early 1900s. Today they are one of the most technically advanced property and casualty insurance companies in the United States. 

The Challenge

The Client needed to retire its on-prem systems, namely Hadoop and Microsoft's Parallel Data Warehouse (PDW), in order to migrate to Snowflake and the cloud. Neither Hadoop nor PDW was performant enough to get the Client what it needed. Not only would this Snowflake migration address the performance issues but it also would create massive cost savings in retiring those systems in the long run. 

Recognizing the limitations of the current systems, the migration to Snowflake was championed and backed by business and IT management members. Intricity was tasked with the challenge of navigating those constraints. 

Navigating Constraints

The volumes of data were incredibly high and deeply integrated into downstream processes. Migrating this data required a thoughtful understanding of process streams and a relationship with the business constituencies. Additionally, while there was a vast amount of data, there also was a vast number of data products, tables, and transformations within the Hadoop and PDW environments. These downstream applications had many interaction points with Hadoop and PDW but the two main sources of logic were HiveQL and SSIS. 

The business rules needed in the future state were absolutely critical, but the depth of the application logic represented a massive corpus of knowledge to be uncovered. 

With the project finish line set to the end of 2023, the Client is already experiencing wins and will continue to do so. What is critical about this first project – the migration to the cloud and Snowflake – is that it will set up the overall success for projects in the future, such as modeling and identity resolution. Here are the wins achieved so far: 

Win 1: All Analytical Data On One Platform

The Client is enabled with a plaform to retire the Hadoop and PDW systems. All analytical legacy systems had a single platform target plan to Snowflake. As applications converted, it was easier for the Client's teams to share different parts of information across the organization which enabled better, more accurate data-based decision making. 

Win 2: Centralized Management

With the data centralized, the distribution and useage had a singular location for managing the data store that did not have limitations on size of data or size of audience. This made a huge difference in the overall performance of the new applications being rolled out. The team members internally had the ability to share data using Snowflake's native data sharing capabilities. 

Win 3: Cost Efficiency

Each application carried with it a massive savings impact to the organization in cost to maintain, speed of decision making, and scalability to their audience. The mobility of the teams leveraging Snowflake made it possible to quickly iterate and surface information throughout the organization. One of the business sponsors pointed out, "Our team can see the dollar signs in savings with every modernization project."

Win 4: Snowflake's Elastic Compute & Storage

The Client is now able to take advantage of Snowflake's elastic compute and storage capabilities. For example, there are certain queries and data applications that the Client was never able to successfully run in Hadoop. Now the Client is able to run those very queries effectively in Snowflake and do analysis that was never done previously. 

Learn more about: 

• Snowflake and what it is
• How to improve your Snowflake environment
• Our thoughts on Hadoop