Escaping The Silos: Emerging Tech Might Help Improve Data Integration
A modern data integration strategy is critical to support the data and analytics requirements Data is too important to be kept in a silo, out on the edges of a business. It’s a recipe for business anarchy, leading to uncoordinated efforts, uninformed decisions, inconsistent messaging as operational workers are unable to access relevant data about […]
Topics
A modern data integration strategy is critical to support the data and analytics requirements
Data is too important to be kept in a silo, out on the edges of a business. It’s a recipe for business anarchy, leading to uncoordinated efforts, uninformed decisions, inconsistent messaging as operational workers are unable to access relevant data about customers, products, and supply chains. When data is siloed, it’s also hard for leaders to get a holistic view of company data.
Businesses increasingly agree with that prognosis.
The exponential volume of data and its increased complexities, compliance pressure, and need for real-time information are putting pressure on businesses to address data integration challenges. Quick access to reliable, real-time information to make better business decisions is vital for every business, primarily to support current customer experience initiatives.
According to The 2021 State Of Martech Survey, close to 57 per cent of marketing executives believe that “data stored in silos” is the biggest challenge.
Marketers can highly benefit from a modern data integration strategy that supports the data and analytics requirements, including real-time customer 360, data intelligence, and modern edge applications.
Although use of AI capabilities in data integration is still emerging, the technology is helping enterprises to automate data integration, especially data ingestion, classification, identifying duplicates, and orchestrating silos, processing, security, and transformation.
Modern data integration technologies, according to Forester, focus on advanced automation, connected data intelligence, and persona-based interactive tooling, helping organisations accelerate various use cases and other data integration requirements.
Forrester identified data as a service, data mesh, knowledge graph, and query accelerator as four technologies that fall into the “experiment” category, but will continue growth in the coming years as demand for real-time data grows across applications.
Data as a service
Data as a service (DaaS) delivers a standard data access layer through application programming interfaces (APIs), SQL, ODBC/JDBC, and other protocols, leveraging data platforms such as data virtualisation, data mesh, integration platform as a service (iPaaS) and others.
DaaS delivers a data access layer to support querying, reporting, data access, and integrated and custom-built applications that helps leaders to take a common view of business and customer data using industry-standard protocols.
Data mesh
Still, in its infancy, a data mesh offers the ability to optimise mixed workloads by matching processing engines and data flows with the proper use cases. It interfaces to the event-driven architecture, enabling support for edge use cases.
A data mesh enables a communications plane between applications, machines, and people, while matching the data, queries, and models to the solution to keep human and machine speaking the same language.
It enables developers, data engineers, and architects to become more productive and accelerate various business use cases.
Knowledge graph
Still evolving with support for automation, built-in AI/machine learning, and self-service capabilities, a knowledge graph makes use of graph engines to support complex data connections and integration. It helps build recommendation engines, cleanse data, perform predictive analytics and connect data quickly.
Developers, data engineers, and data architects can rapidly work through unrelated data to accelerate application development and new business insights.
It leverages a graph data model to store, process and integrate connected data, building a knowledge base to answer complex questions and modern insights.
Query accelerator
The query accelerator market has gained some traction to help developers and data engineers optimise queries quickly and move to compute closer to data, thus minimising data movement.
Query accelerators speed up queries through an improved query optimiser, moving to compute closer to data and fetching only selected data from data sources such as distributed databases, data warehouses, data lakes, object stores, and files.
A query accelerator helps businesses to accelerate analytics and data searches through a simplified query that can be run by business analysts, business users, and IT organisations.
Meanwhile, Forrester recommends enterprises must invest in self-service data integration, customer data platforms, data connectors; data fabric platforms, data virtualisation, integration platform as a service (iPaaS), and real-time data pipeline/streaming. Although they still haven’t matured, these solutions have proven their business value.
If you liked reading this, you might like our other stories
Key Digital Marketing Trends to Watch for in 2021
How Chatbots Can Enhance Customer Experience Using NLP
