Data Mesh is identified as one of the hottest topics to disrupt and dominate the market in 2021. The Data Mesh Architecture 1.0 was first researched and published as HP Technical Report in December 1992 by Chia Chao, Robert English, David Jacobson, Bart Sears, Alexander Stepanov, and John Wilkes. After three decades, we are now witnessing the rise of Data Mesh Architecture 3. (blueiceland.is) 0, which is now gaining all the attention. Data Mesh in its current avatar and definition is mainly credited to Zhamak Dehghani. The question one needs to ask is why Data Mesh is relevant today?
As enterprises transform to data first and real-time decision-making, huge investments are made into data platforms and the supportive ecosystem. However, many such transformations fail as the monolithic tech architectures of the past are cumbersome, expensive, and inflexible to meet the growing appetite of the business for insights. One of the primary reasons most data transformation initiatives fail is the sunken costs around integrations eating up the overall digital transformation budgets.
Data mesh is the paradigm shift in the data architecture world, making it efficient in managing big analytical data. The concept is proposed and built on four principles:
- Domain-oriented decentralization of data ownership and architecture.
- Domain-oriented data served as a product.
- Self-serve data infrastructure as a platform.
- Federated governance to enable ecosystems and interoperability.
These four intuitive principles attempt to address several known challenges posed by centralized analytical data management. They transcend the available analytical data technologies with quick time to value.
Data Mesh is no antidote to the challenges posed by data-driven business initiatives. However, the underlying principles, practices, and core technologies are intended to converge on solving some of the most pressing, unaddressed modernization objectives.
From a different viewpoint, Data Mesh introduces new ideas around ‘data product thinking’ and how it can help drive a more cross-functional approach to business domain modeling and creating high-value data products.
For any technology to become the industry norm and gain technologists’ acceptance, it must effectively demonstrate its applicability and business use cases.
Below use cases exemplify the magnitude of capabilities that a Data Mesh brings to enterprise data.
- Application Modernization – In the race towards application modernization, legacy app monoliths typically depend on extensive monolithic databases, raising the question, “how to implement a phase-wise migration plan to minimize disruption, risks, efforts, and costs?” The perfect answer is served by the Data Mesh architecture abilities, which extend strong support in achieving operational excellence for a phased transition.
- Data Availability & Continuity – KPI’s and SLA’s around resiliency and continuity are important for applications. It makes mission-critical systems a distributed eventual consistency data model, which is usually not acceptable. On the flip side, a Data Mesh forms the foundation for decentralized yet 100% accurate data across sites.
- Event Sourcing – Microservices need to exchange data payloads with each other. A Data Mesh can fuel the foundation tech for microservices-centric data interchange by providing the tooling and frameworks to make patterns repeatable and reliable at scale.
- Streaming Analytics – Streaming data ingest capability is backed by the foundation tech of Data Mesh. There is a predicted reduction of impact on the source applications, data fidelity enhancement, and empower a real-time analytics use case valuable to the data product owners, post Ingesting events by a stream.
- Streaming Data pipelines – A Data Mesh can provide a data pipeline layer that is autonomously regulated and works with analytic data repositories. It brings self-service data discovery, data preparation, and data transformation into required data product formats.
A Data Mesh is not just a new buzzword on top of an old tech architecture. It brings unique value, distinctly different from commonplace solutions that have already been around for decades. A proper Data Mesh is a mindset, an organizational model, and an enterprise data architecture approach. It weaves together data product thinking, decentralized data architecture, event-driven actions, and a streaming-centric ‘service mesh’ style of microservices design.
Are you planning to implement data mesh in your organizations? Our team of experts is here to help.