Where Data and Process Meet

Big Data was a scam. Those are strong words, perhaps too strong, but Big Data lured a lot of business stakeholders into a "pixie dust" belief system about the data-to-information process. The message went something like this... "If you have all your data in one place you'll have all the information you need." This was sold as if the refinement of data into information was a nonissue. But the refinement of data into Information is THE issue. The part that is often ignored is the Informational part of the equation.

Data engineers not knowing what is informational is like a chef not knowing what dish the patron wants to eat. They may have all the raw ingredients but the person eating needs to choose their dish first. But even in this line of thinking, there is a scalability trap.

The Trap

Business users needing information is an endless pursuit. First, new data is constantly something needing to be considered. Second, new perspectives on what is informational is a moving target. So what often happens with organizations is that they hyper-focus on the canvas for delivering information. Analytics vendors love these mindsets as the organization will open its purse strings to pay for better information. After all, the demo was amazing. "Beware the BI Demo", highlights the danger of falling into this trap.

While we compared the chef not knowing what the patron wants to the data engineer not knowing what business users want, the belief that data engineers are the "cooks of information" is a great way of creating a mutiny between Business and IT. The "cooks of information" approach is usually one of the symptoms after an analytics platform purchase. Data engineers feel compelled to tactically address issues with the analytics canvas rather than focusing their time on scaling the information backbone. So they become short-order cooks of each data visualization, and since they don't run the business they're doomed to fail in this task. The complaint we hear from business stakeholders is, "IT can't keep up with our information requests." The complaint we hear from IT stakeholders is, "Business users are just asking for all the data in a report and the analytics tool can't handle requests like that."

The long-term problem is that the logic for scaling the information backbone gets embedded into the proprietary analytics layer. YIKES!

Whether that reason for data engineers becoming short order cooks came from the introduction of a new analytics tool or whether the culture of the organization organically devised it, changes here are essential. Getting out of this hamster wheel requires data to be turned into building blocks.

Information as Components

Information to business users ties to business processes which tie to the elements of those business processes. Business process owners often ask for the same elements to manipulate and constrain their data sets. For example, almost every business process owner wants to be able to slice their data by date and time. We call these data elements "dimensions" or "attributes". Obviously, not every dimension is important across every business owner. For example, the Supplier Name might be supercritical for the Procurement team, but Sales might not care about that attribute at all.

The Bus Matrix

Knowing which dimensions of data are important to which process owners turns out to be an important step in determining what is informational to business stakeholders. The shared interest in data attributes allows the organization to conform the data in ways that are informational to many business users. This gets expressed in a document which the data management industry calls a Bus Matrix.

If you look closely at the Bus Matrix you'll see the Y-axis is to categorize a Subject Area, Process, or Event. Essentially these represent the acting entities of an organization. Then across the X-axis, we have the data elements requested (dimensions). The process of extracting these elements occurs within a set of strategic interviews that Intricity conducts with the organization. If you would like to learn more about the interviews we do, you'll find a whitepaper about it here.

Once the Matrix is filled out you can see what dimensions of data the organization needs and which acting parties within the organization need it. Now we're looking at information in the form of its building blocks instead of the canvas they'll be dropped into. This is where the data engineers should be focusing, as it provides the base foundation of scalably for delivering a wide range of content. The work isn't complete of course. This is just the beginning of the road. The real investments from here are in creating a database target that houses these dimensions and creating a loading process that conforms raw data into these dimensions.

Once this database (or data warehouse) is in place, business users have a curated version of the truth that is not tightly coupled to a single canvas. Instead, the business is supplied with an open set of building blocks that can supply data to N number of canvases. Additionally, the agreement on what "truthful information" means is defined by the parties that have the authority to do so. This not only means that those parties use the solution, but also that it can be agreed from the top-down as "sanctioned truth." The investment in all this effort is made possible by the fact that the end result is a database that is openly accessible and queryable by any analytics tool or user with the appropriate credentials. Rather than some proprietary singular analytics canvas that is too tightly coupled to a single vendor.

As you can see, there is no pixie dust that turns data into information. There's also no pixie dust for enabling an organization to define what truth is. The gulf between data to information can only be obtained if the definition of what information means to the business can be uncovered. The Bus Matrix is one of the many tools Intricity uses to assemble those fundamental building blocks as part of its data strategy engagements.