Businesses want real time updates to their data, but they don't want to tie up the application (more…) systems that create that data, because it slows down the organizations core processes. This video discusses how Data Replication mediates this paradox.
Text from the Video:
In Business Intelligence solutions we run into a common paradox. Businesses want real time updates to their data, but they don’t want to tie up the application systems that create that data, because it slows down the organizations core processes.
So let’s break up this paradox into its two opposing components, then we will share a balanced way of solving it. And let’s start with real time side of the paradox.
What does real time mean to your business? To most, it means updates measured in minutes to hours. However, in some industries, like financial services, this means data lands in reports quite literally at the same time it does in the transaction system database. Not surprisingly, as updates become that instantaneous, so do the costs and complexities of its implementation. And this is why most organizations opt for some level of latency in their real time reporting. Process wise, this means that first the application writes directly to its own database then we interrogate that database for whatever data we need. This process creates an inherent latency which we must work with the business to agree on. Once we have an agreed latency we can then work on the other end of the paradox, which is the effect that large database queries have on enterprise application performance.
Enterprise applications need highly available database systems. This is because business processes are directly linked to these applications, and any down time can pause these revenue generating processes. So in nearly every business, the enterprise application databases become highly critical, and practically sacrosanct. Depending on the timing question, we often can avoid these adverse impacts simply by timing a single batch extract of data at night time, while these processes are not in use. However, most organizations have operational reporting requirements that need to deliver information about a process while the transaction is in flight, so the nightly loads just won’t cut it.
This is the point where data replication comes into the picture and mediates the paradox. The concept of data replication is quite simple, but it plays an important role. Data replication allows a database to be copied to a secondary Database as record updates occur. This is done by capturing small changes and updates to the application database and passing those updates to the secondary clone database. Now this does have a small impact on the production database, but because it is extracting as the data is updating, the impact is very small and manageable. This means that in the middle of the day I can hit the clone database with many queries without worrying about the application being slowed down in any way. Additionally, the leading tools for replication allow organizations to replicate to heterogeneous database technologies.
Even though we can design reports up against the data replication clone, this doesn’t replace the need for other BI and data integration solutions such as data warehousing. Rather the replication system is part of a larger picture for delivering your data, and it specifically shields your production system from the burden which these solutions introduce.
Additionally, data replication is a useful component in backup and disaster recovery strategy. Often when a disaster occurs, organizations are left with day old backups to rely on. However, by using replication, organizations can enable their backup systems to contain the latest data up to the point of the disaster.
Because of Intricity’s exposure to literally hundreds of industries with a wide array of data requirements, we are highly qualified to help you through your information management challenges. I recommend you reach out to Intricity and talk with one of our specialists. We can put our experience to work in your organization to expand the value of your data.
