In the past few months, I’ve spent a lot of time talking to companies about data., We’ve talked about applications that contain data, reporting, systems, integration, schemas, governance, and much more. Underlying all of those discussions is one fundamental challenge comes through:
Most organizations have data spread across
multiple systems that don’t talk to each other.
When we press deeper, and explore why that is, there are a few key reasons that emerge from the conversation:
- Applications are the main way that people interact with data (thankfully – most people don’t like working directly in tables or columns).
- But applications are owned by different owners (groups, executives, people, departments, etc)
- Not everyone has access to every application. This can be for governance, isolation/separation-of-duties, or just reasons of not wanting to pay for licenses for everyone.
- Every application stores data differently.
And data integration doesn’t require that applications be integrated. In fact, maintaining application integrations is tricky. Applications change. Data fields get added, removed, and modified.
But may data all looks different!
Of course it does. It should. What’s important in your customer relationship management (CRM) system may be irrelevant in your intranet. What’s most crucial in your intranet may be only marginally relevant in your billing system. The important data defines what the data looks like in those applications. And since the data looks different and changes, many companies don’t know how to make all the data play nicely together.
This is why we say that schemas are preventing data insight – many companies don’t realize that there’s a better way.
That way is to take an external system, and add data there. This sometimes called a data integration layer, data hub, data lake, logical data warehouse (different from a traditional data warehouse) or data workbench.
Q: Doesn’t that create a second copy of the data?
Q: Isn’t that ineffecient?
A: In theory, yes, but storage is cheap.
Q: Does that make my data fungible in another location?
A: No, the data is read-only in the external data hub.
Q: Does the data integration layer provide any intelligence?
A: Some do, some don’t. FlockData does.
Get in touch if you want to find out more!