Here's the whitepaper referenced: Data as Plastic
Have you ever asked yourself whether other companies are experiencing the same problems you are?
This video shares the common problems that Organizations face in managing their data as they grow in revenue. https://www.intricity.com/basic/intricity101-landing-page
Text from the Video:
I’ve noticed that there seems to be a maturity continuum which nearly every organization moves through in managing data. In this video I’d like to outline this continuum which we see different organizations follow. Now I want to point out up front that this doesn’t represent what ideally should occur within organizations, but rather what naturally occurs while an organization grows in revenue and the habits that are formed throughout this lifecycle. I’ll start with organizations that are roughly 50 Million dollars in revenue.
I like to call this the reflexive stage. In this stage, organizations are basically reacting to each request for information on a one off basis. There is little to no reuse in the data gathering process. So every time a manager or director needs updated data or information, it requires a manual effort to deliver it. Often organizations in this stage will have multiple tools being used between individuals, and a lot of man hours spent on what managers might think are simple requests. Organizations in this stage are seeking to create some repeatability in the work they do. So they’ll often deploy Operational Reporting tools that allow standardized reports to be delivered to end users. Also these organization often purchase desktop data analysis tools that allow these one off requests to be more easily delivered and analyzed. The use of these tools often bridge the organizations growth into the Inclusive Stage.
In the Inclusive Stage organizations begin to realize they have a lot of operational reports and a lot of desktop analytics with nested assumptions. When a change needs to be made to those assumptions, each individual report needs to be opened and edited. Also, the business users begin to want control of their own analysis and are no longer willing to allow an IT person to be their liaison to their data. In addition they want more direct access to shared logic so they can work as teams. This is where organizations begin adopting Business Intelligence Platforms, and Standardized Business Intelligence Logic that can be shared across users. These platforms allow for business users to log into secured enterprise applications that are built for collaborative analytics and shared logic, which the IT team can maintain. And this is where a critical juncture occurs. If organizations do not quickly transition to the Truthful Stage, they will find themselves believing that their BI platform is faulty or end up incredibly frustrated with the data consumption process.
In the Truthful Stage the organization begins to see that the different sources of data need to come together in order to provide a cohesive story. This usually reveals itself in executive meetings where there are multiple versions of the truth and they can’t make decisions. For organizations that have lingered in the Inclusive Stage, they often believe their BI tool is faulty and are seeking its immediate replacement. Replacing the BI tool is usually a mistake, as the problem is really only addressed through a Data Mart or a Data Warehouse implementation, using an ETL tool.
Then we enter the Governed Stage. Once a Data Warehouse and BI Platform are in place, organizations begin to increasingly realize that their data is not clean. This becomes apparent when they open a customer, say GE for example, and they find multiple company names that represent GE including ones that are simply spelled incorrectly. Additionally, they will find that many of the records that are being stored will have incorrect mailing addresses and other data quality problems. These wide array of data issues become more obvious as the organizations centralize their data usage. To address this, organizations will adopt data quality tools to make changes at a bulk level and Master Data Management solutions to consolidate their critical attributes into surviving records and hierarchies that the business can agree on and control. The business will commonly organize a data governance team to manage the data.
This leads us to the Expansive Stage. As organizations become more and more mature in building data warehouses and governing their data, they quickly realize that the size of the data footprint is slowing down the accessibility of the information. This moves organizations into one of 2 strategies. The first strategy is archiving. This is where inactive data is offloaded and maintained in a separate archived tier, relieving the production system from antiquated records which are rarely accessed, which speeding up the production systems. The second strategy is a performance expansion of the data warehouse called a Data Warehousing Appliance. These appliances are specialized configurations of hardware and software which widen the pipe for larger database queries, allowing billions of rows to be accessed on the fly. These data warehousing appliances are often lumped into a larger category which the market calls Big Data. In the Expansive Stage we also see organizations adopting other Big Data solutions like Hadoop. Hadoop provides a cost effective way of processing large quantities of unstructured data. The architecture behind Hadoop originated from Google’s efforts to deal with their massive indexes while reducing the cost of hardware.
Like I said in the beginning, these stages don’t represent an ideal path organizations should follow, but rather a natural path based on the problems an organization faces as they grow. Perhaps in an ideal world you could build a Master Data Management backbone before you rolled out all your applications and you could have your data warehouse in place before you built your business intelligence content. But in the real world, it’s difficult to budget a fix to a problem that isn’t readily noticeable.
Now that you can see the whole picture, you probably will identify your organization still dealing with reflexive stage issues, even though your organization is in latter stages. Or perhaps you’ll be dealing with Governed Stage issues and you are still in the Reflexive stage. There really is no cookie cutter model for how organizations should behave but it is important to understand what solutions exist for the problems you are facing today. Intricity is highly aware of these solutions and a very capable partner in helping you plan the future. I recommend you reach out to Intricity and talk with one of our specialists. We take an active role in our client’s strategic roadmap, and we simplify the complexity of its realization.
