Matt Aslett's Analyst Perspectives

The degree to which data platforms are critical to efficient business operations cannot be overstated. Without data platforms, enterprises would be reliant on a combination of paper records, time-consuming manual processes and huge libraries of physical files to record, process and store business information. The extent to which that is unthinkable highlights the level at which today’s enterprises and society as a whole rely on data platforms. The core persistence, management, processing and query requirements of data platforms have been well established since the emergence of the database market in the early 1970s. The data platforms market continues to evolve to keep pace with emerging capabilities and requirements. Two of the most significant trends driving the data platforms sector in recent years are cloud computing and artificial intelligence.

ISG Research defines data platforms as software products that provide an environment for organizing and managing the storage, processing, analysis and presentation of data across an enterprise. Data platforms support and enable operational applications used to run the business, as well as analytic applications used to evaluate the business, including AI, machine learning and generative AI. The data platforms software market encompasses relational and non-relational operational databases, as well as data warehouses, data marts, data lakes, data lakehouses and other analytic data platform products. Many other categories of analytics and data products rely on the processing engine capabilities of data platforms, including data operations platforms and tools and data intelligence platforms and tools. Although the primary users of data platforms are workers in technical roles, including database administrators, application developers, data engineers and data architects, data platforms must support a range of workloads aimed at users with differentiated responsibilities and functional requirements, including business users and managers as well as data analysts and data scientists.

At the heart of any data platform is the storage and management of a collection of data. This is typically provided by a database management system—more commonly referred to simply as a database—that provides the data persistence, data management, data processing and data query functionality that enables access to and interaction with stored data. The data platforms market has traditionally been dominated by the relational data model and relational database management systems. However, one approach does not suit all use cases, and enterprises use a variety of data platforms to fulfill the spectrum of requirements for myriad applications. Non-relational data models that pre-date relational, such as the hierarchical model, remain in use today. However, the use of non-relational data models has become widespread in recent years through the adoption of NoSQL databases and the integration of key-value, document and graph model capabilities into relational databases.

Data platforms were traditionally deployed on-premises, but in recent years, many enterprises have adopted cloud-based data platforms with a view to improving operational efficiencies by reducing the need for upfront investment in physical infrastructure as well as the ability to scale cloud services up and down to match fluctuating requirements. ISG’s 2024 Market Lens Cloud Study illustrates the extent to which the database market is now dominated by the cloud, with 58% of participants using cloud environments for more than half of database and data platform workloads.

Adoption of cloud computing environments has also led to the widespread use of object stores as a data persistence layer, particularly for data lakehouse environments, with query engines such as Apache Spark, Apache Presto and Trino adding the data management, data processing and data query functionality required of a data platform. Additionally, support for open table formats is now a critical feature for providers of analytic data platforms to enable the persistence and analysis of structured and unstructured data in object storage.

There have always been general-purpose databases used for both analytic and operational workloads, but data processing architectures have traditionally involved the extraction, transformation and loading of data from the operational data platform into a separate external analytic data platform. This enables operational and analytic workloads to run concurrently without adversely impacting each other, protecting the performance of both. Over time, dedicated analytic data platforms have also evolved differentiated architectural approaches designed to improve query performance, while specialist analytic data platform providers have accelerated the development of in-database AI/ML functionality to address the growing demand for AI and avoid the cost and complexity of moving data to an external environment.

The increased focus on AI-driven intelligent applications is significantly impacting how software providers approach the data platforms market. Analytic data platform workloads are typically targeted at data and business analysts and include decision support, business intelligence, data science and AI and ML. Operational data platform workloads typically target business users and decision-makers. Examples include finance, operations and supply chain, sales, human capital management, customer experience and marketing applications.

The need for real-time applications driven by online predictions and recommendations has increased the requirements for operational data platforms to support real-time AI/ML inferencing. Consumers increasingly engage with data-driven services differentiated by personalization and contextually relevant recommendations. Additionally, worker-facing applications are following suit, targeting users based on their roles and responsibilities. These intelligent applications, while operational in nature, rely on real-time analytic processing to deliver functionality, including contextually relevant recommendations, predictions and forecasting driven by ML and GenAI. I assert that through 2027, the development of intelligent applications providing personalized experiences driven by GenAI will increase demand for data platforms capable of supporting hybrid operational and analytic processing.

The popularization of GenAI has also had a significant impact on the requirements for data platforms to support storing and processing vector embeddings. These are multi-dimensional mathematical representations of features or attributes of raw data that are used to support GenAI-based natural language processing and recommendation systems. Vector search can also improve accuracy and trust with GenAI via retrieval-augmented generation, which is the process of retrieving vector embeddings representing factually accurate and up-to-date information from a database and combining it with text automatically generated by a large language model.

Cloud and AI are two of the most significant factors driving the evolution of data platform capabilities, but I recommend that all enterprises considering options for new data platform deployments also evaluate other product experience functionality addressing adaptability, manageability, reliability and usability as well as the customer experience provided by the software provider. Those were the criteria ISG used to evaluate data platforms software providers in the 2024 Data Platforms Buyers Guides and will once again provide the framework for our assessment of data platform providers in 2025.

Regards,

Matt Aslett