Cloud Giants Down, AI Up — Campus Technology

Data Integration Market: Cloud Giants Down, AI Up

While data integration is seemingly a natural fit for cloud giants’ computing platforms, the cloud lags in the data integration market, which is being hugely transformed by AI.

“By 2027, AI assistants and AI-enhanced workflows incorporated into data integration tools will reduce manual intervention by 60 percent and enable self-service data management,” said research firm Gartner in a new Magic Quadrant report on data integration tools.

That market is thriving as organizations seek better ways to handle data for various needs, including operations, analytics, and AI. It involves software that combines data from different sources to support tasks like data access, transformation, enrichment, and delivery.

And it’s a crowded market at the top, in the “Leaders” quadrant, where the cloud giants are eclipsed by lesser players, including Informatica, Oracle, and IBM. Among those cloud giants, Microsoft does rank highly on the “ability to execute” axis and trails only those three companies above on the “completeness of vision” axis.

Amazon Web Services (AWS) and Google, meanwhile, trail on the vision axis behind eight other leaders.

Gartner weighed in on why those cloud giants might not be dominating this space, which has seen the market share of the top five vendors shrinking from 71% in 2017 to 53% in 2023.

“One of the main reasons for this is that, as organizations migrate data integration to the cloud, they’re evaluating modern cloud-native/SaaS vendors and public cloud hyperscalers (CSPs), including native cloud data ecosystems,” the report said. “Another reason is that smaller vendors with more focused offerings, or those that target business users through innovative offerings, pricing, packaging and go-to-market strategies, continue to disrupt larger vendors. Vendors gaining market share have a common theme: They focus on leadership in specific data integration styles such as data virtualization, data replication or streaming; and/or they focus on data integration delivered as a native and managed cloud service. Vendors need to find the right balance between all-encompassing platform solutions and easily accessible point solutions to keep pace.”

Gartner notes the products work on-premises, in a public or private cloud or in hybrid cloud implementations, often being consumed as Software-as-a-Service (SaaS).

Common use case scenarios or business problems addressed by tools in the space include:

• Data engineering: Data integration by technical user personas to develop, manage and optimize data pipelines in analytical use cases
• Delivering modern data management architectures: Data integration to enable modern data management design patterns like lakehouse, data fabric, data mesh and deliverables like data products
• Enabling less-technical data integration: Data integration activities by less-technical user personas for various analytical demands of data, such as analytics and business intelligence (ABI), and data science use cases
• Operational data integration: Data integration to implement various operational data integration use cases, such as consolidation of master data, delivering and using data hubs, interenterprise and partner data sharing, and application integration

While AI figures to be a transformative technology here, like everywhere else, it also figures into the common features of data integration tools, one of which Gartner lists as: “Augmentation features that leverage generative AI (GenAI) and prepackaged ML algorithms to auto-generate data pipeline code and documentation, optimize data integration operations (e.g., anomaly detection, autorecovery), and use natural language to query as well as transform data.”

The firm also noted “the use of GenAI assistant is now standard in most data integration tools, allowing creation, observability and management of data pipelines through NLQ type interfaces.”