The path to data centricity 5: why a data mesh
Your modern-day multinational turns at full regime with loads of data entering. Your technical team has put in place a solution able to ingest data and distribute it to the relevant consumers. But you are struggling to bring business managers across the company on board and break down your monolith solution into a collaborative distributed ecosystem.
One big remaining step must be accomplished: the organisational changes.
In short, the governance and organization part is decentralized throughout the organization. A little like a microservice architecture, the data mesh proposes to distribute the data management capabilities and responsibilities to the different data domains. Each business domain is responsible for making the data it generates available to the other domains. It is responsible for treating its data as a data product.
The main pillars of a data mesh are:
- A federated governance model that is "defined and managed centrally" but executed in a federated way.
- Data products rather than one single canonical model that does not change over time: for the creator of the data mesh, this concept of a single canonical model cannot work in a data-driven enterprise.
- Abstract complexity of the underlying data infrastructure. A data engineer should focus only on the logical transformation, not the tech.
- Self-service data infrastructure: discover data, explore, consume in your data domain.
A data mesh approach has three main recommendations for organisations:
- Product thinking: data owners should behave as product owners, advocate for their data as an asset, and focus on customer experience.
- Domain-oriented ownership (role & responsibilities): it makes the business managers responsible for making available to data scientists data that will bring value to their department through analytics and AI.
- Abstract the tech into a layer of self-service.
However, senior managers should keep in mind that this cannot be provided by any tool and requires a consequent change in mindset. The data mesh concept is mainly a data governance approach applied with the right underlying technology platform.
Our Recommendation:
The data mesh does not explicitly provide a data architecture pattern. It offers a data governance framework. This is why we can’t compare it to the data architecture patterns described earlier.
With that being said, we cannot deny that a data mesh is a very powerful way of managing data. It is highly recommended, but it requires a mature data platform (we will detail this in a later article), a strong data culture in all business lines, and a powerful governance board. This is a journey.
Our expertise in designing and building a self-service data platform can provide a great start to a data mesh architecture. Our expertise in data management and governance can help you measure your maturity in the domain, define where you want to be and set the roadmap to get there.
Our White Paper
To go further, read our white paper that walks you through the different solutions available and help you make a choice based on your business needs.