The adoption of cloud environments for analytic workloads has been a key feature of the data platforms sector in recent years. For two-thirds (66%) of participants in ISG’s Data Lake Dynamic Insights Research, the primary data platform used for analytics is cloud based. Many enterprises adopted cloud-based analytic data platforms with a view to improving operational efficiencies by reducing the need for upfront investment in physical infrastructure as well as the ability to scale cloud services...
Read More
Topics:
data operations,
Data Platforms,
Analytics and Data
I previously wrote about the importance of open table formats to the evolution of data lakes into data lakehouses. The concept of the data lake was initially proposed as a single environment where data could be combined from multiple sources to be stored and processed to enable analysis by multiple users for multiple purposes.
Read More
Topics:
Data Platforms,
Analytics & Data,
Streaming Data & Events
Although the terms data fabric and data mesh are often used interchangeably, I previously explained that they are distinct but complementary. Data fabric refers to technology products that can be used to integrate, manage and govern data across distributed environments, supporting the cultural and organizational data ownership and access goals of data mesh. Data fabric and data mesh are also both related to logical data management, which is the approach of providing virtualized access to data...
Read More
Topics:
Data Intelligence,
Analytics and Data
As I noted in the 2024 Buyers Guide for Operational Data Platforms, intelligent applications powered by artificial intelligence have impacted the requirements for operational data platforms. These applications, infused with contextually relevant recommendations, predictions and forecasting, are driven by machine learning and generative AI.
Read More
Topics:
Data Platforms,
Analytics & Data
As I recently noted, the term “data intelligence” has been used by multiple providers across analytics and data for several years and is becoming more widespread as software providers respond to the need to provide enterprises with a holistic view of data production and consumption. I assert that through 2027, three-quarters of enterprises will be engaged in data intelligence initiatives to increase trust in their data by leveraging metadata to understand how, when and where data is used in...
Read More
Topics:
Data Intelligence,
Analytics and Data
I previously wrote about data mesh as a cultural and organizational approach to distributed data processing. Data mesh has four key principles—domain-oriented ownership, data as a product, self-serve data infrastructure and federated governance—each of which is being widely adopted. I assert that by 2027, more than 6 in 10 enterprises will adopt technologies to facilitate the delivery of data as a product as they adapt their cultural and organizational approaches to data ownership in the...
Read More
Topics:
data operations,
Analytics and Data
As I explained in our recent Buyers Guide for Data Platforms, the popularization of generative artificial intelligence (GenAI) has had a significant impact on the requirements for data platforms in the last 18 months. While there is an ongoing need for data platforms to support data warehousing workloads involving analytic reports and dashboards, there is increasing demand for analytic data platform providers to add dedicated functionality for data engineering, including the development,...
Read More
Topics:
Analytics,
natural language processing,
Data Platforms,
AI and Machine Learning,
Generative AI,
Model Building and Large Language Models,
Machine Learning Operations
The final of the men’s 100 meters at the Paris Olympics this summer was a reminder that being successful requires not just being fast but performing at the right time. Being fast is obviously a prerequisite for participating in an Olympic 100-meter final, and all the competitors finished the race in under 10 seconds, with just 0.12 seconds separating the first man from the last. While all the athletes were fast, what separated the winner of the gold medal—USA’s Noah Lyles—was execution. He was ...
Read More
Topics:
Analytics & Data,
Streaming Data & Events
Enterprises face a bewildering level of choice in relation to data platforms, as evidenced by the number of software providers and products assessed in our recent Data Platforms Buyers Guide. There are numerous data platform providers and products to choose from, but also a diverse array of functional and architectural options. Is the workload primarily operational or analytic? Will it be deployed on-premises or in the cloud? Should it be distributed or centralized? Data warehouse or data...
Read More
Topics:
Data Platforms,
AI and Machine Learning,
Data Intelligence,
Analytics and Data
I have written on multiple occasions about the increasing proportion of enterprises embracing the processing of streaming data and events alongside traditional batch-based data processing. I assert that, by 2026, more than three-quarters of enterprises’ standard information architectures will include streaming data and event processing, allowing enterprises to be more responsive and provide better customer experiences.
Read More
Topics:
Data Platforms,
Analytics & Data,
Streaming Data & Events,
AI and Machine Learning
The artificial intelligence and machine learning landscape was profoundly altered by the emergence of generative AI into the mainstream consciousness during 2023. The widespread availability of GenAI models and cloud services has lowered the barriers to individuals and enterprises engaging with AI for various use cases, including generating content, querying data, writing code, preparing data for analysis, documenting data pipelines and using software products more effectively. The impact that...
Read More
Topics:
Analytics & Data,
AI and Machine Learning
I am happy to share insights gleaned from our latest Buyers Guide, an assessment of how well software providers’ offerings meet buyers’ requirements. The Data Platforms Ventana Research Buyers Guide is the distillation of a year of market and product research by ISG and Ventana Research.
Read More
Topics:
Data Platforms,
Analytics & Data,
analytic data platforms,
Operational Data Platforms
I previously wrote about the ongoing importance of event brokers and event management in enabling enterprises to adopt event-driven architecture and event stream processing. Many enterprises adopt EDA as the design pattern for maximizing events to deliver real-time business processes. There are many advantages to using EDA, including a cultural shift away from batch processing towards real-time analysis and decision-making.
Read More
Topics:
Analytics & Data,
Streaming Data & Events,
Data Intelligence
I previously wrote about the potential for generative artificial intelligence technology to enhance the integration sector by facilitating outcome-driven approaches for automatically generating integration pipelines in response to declared business requirements. The use of GenAI in data and application integration remains nascent, but multiple software providers are embracing the potential for GenAI to improve the productivity of integration experts and facilitate self-service integration by...
Read More
Topics:
Analytics & Data,
Data Intelligence
I recently wrote about the role data observability plays in generating value from data by providing an environment for monitoring its quality and reliability. Data observability is a critical functional aspect of Data Operations, alongside the development, testing and deployment of data pipelines and data orchestration, as I explained in our Data Observability Buyers Guide. Maintaining data quality and trust is a perennial data management challenge, often preventing organizations from operating...
Read More
Topics:
data operations,
Analytics & Data,
Data Intelligence
Enterprises are embracing the potential for artificial intelligence (AI) to deliver improvements in productivity and efficiency. As they move from initial pilots and trial projects to deployment into production at scale, many are realizing the importance of agile and responsive data processes, as well as tools and platforms that facilitate data management, with the goal of improving trust in the data used to fuel analytics and AI. This has led to increased attention on the role of data...
Read More
Topics:
data operations,
Analytics & Data,
AI and Machine Learning
The emergence of generative artificial intelligence (GenAI) has significant implications at all levels of the technology stack, not least analytics and data products, which serve to support the development, training and deployment of GenAI models, and also stand to benefit from the advances in automation enabled by GenAI. The intersection of analytics and data and GenAI was a significant focus of the recent Google Cloud Next ’24 event. My colleague David Menninger has already outlined the key...
Read More
Topics:
Analytics,
natural language processing,
Data Platforms,
Analytics & Data,
AI and Machine Learning,
Generative AI
I recently wrote about the development, testing and deployment of data pipelines as a fundamental accelerator of data-driven strategies as well as the importance of data orchestration to accelerate analytics and artificial intelligence. As I explained in the recent Data Observability Buyers Guide, data observability software is also a critical aspect of data-driven decision-making. Data observability addresses one of the most significant impediments to generating value from data by providing an...
Read More
Topics:
Analytics,
Data Ops,
data operations,
Analytics & Data,
AI and Machine Learning,
Generative AI,
Machine Learning Operations
I previously wrote about the potential for rapid adoption of the data lakehouse concept as enterprises combined the benefits of data lakes based on low-cost cloud object storage with the structured data processing functionality normally associated with data warehousing. By layering support for table formats, metadata management and transactional updates and deletes as well as query engine and data orchestration functionality on top of low-cost storage of both structured and unstructured data,...
Read More
Topics:
Analytics,
Data Platforms,
Analytics & Data
Many organizations have adopted DataOps to apply agile development, DevOps and lean manufacturing processes to the development, testing, deployment and orchestration of data integration and processing pipelines. The most likely ultimate outcome of these pipelines is the analytics reports and dashboards enterprises rely on to make business decisions.
Read More
Topics:
Analytics,
Analytics & Data,
Data Intelligence
Analytics software is used by business analysts and decision-makers to facilitate the generation of insights from data. It encompasses business intelligence and decision intelligence software, including reports and dashboards as well as embedded analytics and the development of intelligent applications infused with the results of analytic processes. Analytics software enables enterprises to improve business outcomes by operating more efficiently, accelerating product development and enhancing...
Read More
Topics:
Analytics,
AI,
Analytics & Data,
Generative AI
I recently wrote about the development, testing and deployment of data pipelines as a fundamental accelerator of data-driven strategies. As I explained in the 2023 Data Orchestration Buyers Guide, today’s analytics environments require agile data pipelines that can traverse multiple data-processing locations and evolve with business needs.
Read More
Topics:
Analytics,
data operations,
Data Platforms,
Analytics & Data,
AI and Machine Learning,
Generative AI,
Data Intelligence
I previously explained how master data management helps provide trust in data, making it one of the most significant aspects of an enterprise’s strategic approach to data management. More recently, I discussed how it has a role to play in accelerating data democratization as part of data intelligence initiatives. Along with data quality, MDM enables organizations to ensure data is accurate, complete and consistent to fulfill operational business objectives. While it is an established and mature...
Read More
Topics:
Product Information Management,
Operations & Supply Chain,
Analytics & Data,
Sustainability Management,
Data Intelligence
I wrote recently about the role that data intelligence has in enabling enterprises to facilitate data democratization and the delivery of data as a product. Data intelligence provides a holistic view of how, when, and why data is produced and consumed across an enterprise, and by whom. This information can be used by data teams toensure business users and data analysts are provided with self-service access to data that is pertinent to their roles and requirements. Delivering data as a product...
Read More
Topics:
Analytics,
Data Ops,
data operations,
Data Platforms,
Analytics & Data,
AI and Machine Learning,
GenAI,
Data Intelligence
The development, testing and deployment of data pipelines is a fundamental accelerator of data-driven strategies, enabling enterprises to extract data from the operational applications and data platforms designed to run the business and load, integrate and transform it into the analytic data platforms and tools used to analyze the business. As I explained in our recent Data Pipelines Buyers Guide, data pipelines are essential to generating intelligence from data. Healthy data pipelines are...
Read More
Topics:
Analytics,
AI,
data operations,
Data Platforms,
Analytics & Data,
AI and Machine Learning,
Data Intelligence
As enterprises seek to increase data-driven decision-making, many are investing in strategic data democratization initiatives to provide business users and data analysts with self-service access to data across the enterprise. Such access has long been a goal of many enterprises, but few have achieved it. Only 15% of participants in Ventana Research’s Analytics and Data Benchmark Research say their organization is very comfortable allowing business users to work with data that has not been...
Read More
Topics:
Analytics,
data operations,
Analytics & Data,
AI and Machine Learning,
Data Intelligence,
Data Products,
Data Democratization
Cloud computing has had an enormous impact on the analytics and data industry in recent decades, with the on-demand provisioning of computational resources providing new opportunities for enterprises to lower costs and increase efficiency. Two-thirds of participants in Ventana Research’s Data Lakes Dynamic Insightsresearch are using a cloud-based environment as the primary data platform for analytics.
Read More
Topics:
Analytics,
AI,
Data Platforms,
Analytics & Data,
AI and Machine Learning,
Generative AI,
Data Intelligence
It is well known that data integration, transformation and preparation represent a significant proportion of the time and effort required in any analytics project. Traditionally, operational data platforms are designed to store, manage, and process data to support worker-, customer- and partner-facing operational applications, and data is then extracted, transformed, and loaded (or “ETLed”) into a separate analytic data platform, which is designed to store, manage, process, and analyze data....
Read More
Topics:
Analytics & Data,
Data Intelligence
I have previously written about the impact of intelligent operational applications on the requirements for data platforms. Intelligent applications are used to run the business but also deliver personalization, recommendations and other features generated by machine learning and artificial intelligence. As such, they require a combination of operational and analytic processing functionality. The emergence of these intelligent applications does not eradicate the need for separate analysis of...
Read More
Topics:
Analytics,
Artificial intelligence,
Data Platforms,
Analytics & Data,
AI and Machine Learning,
Generative AI
The increasing importance of intelligent operational applications driven by artificial intelligence (AI) is blurring the lines that have traditionally divided the requirements between operational and analytic data platforms. Operational data platforms have traditionally been deployed to support applications targeted at business users and decision-makers to run the business, with analytic data platforms typically supporting applications used by data and business analysts to analyze the business.
Read More
Topics:
embedded analytics,
Cloud Computing,
Analytics & Data,
analytic data platforms,
Operational Data Platforms,
AI and Machine Learning
I have previously written about the functional evolution and emerging use cases for NoSQL databases, a category of non-relational databases that first emerged 15 or so years ago and are now well established as potential alternatives to relational databases. NoSQL is a term used to describe a variety of databases that fall into four primary functional categories: key-value stores, wide-column stores, document-oriented databases and graph databases. Each is worthy of further exploration, which is...
Read More
Topics:
Data,
data operations
In recent years, many enterprises have migrated data platform workloads from on-premises infrastructure to cloud environments, attracted by the promised benefits of greater agility and lower costs. The scale of cloud data platform adoption is illustrated by Ventana Research’s Data Lakes Dynamic Insights research: For two-thirds (66%) of participants, the primary data platform used for analytics is cloud based. As the quantity and importance of the data platform workloads deployed in the cloud...
Read More
Topics:
business intelligence,
Cloud Computing,
data operations,
robotic automation,
Analytics & Data,
analytic data platforms,
AI and Machine Learning
Ventana Research recently announced its 2024 Market Agenda for Analytics and Data, continuing the guidance we have offered for two decades to help enterprises derive optimal value and improve business outcomes.
Read More
Topics:
embedded analytics,
Business Intelligence,
Data Governance,
Data Management,
natural language processing,
data operations,
Process Mining,
Streaming Analytics,
Analytics & Data,
Streaming Data & Events,
analytic data platforms,
Operational Data Platforms,
AI and Machine Learning
Discussion about potential deployment locations for analytics and data workloads is often based on the assumption that, for enterprise workloads, there is a binary choice between on-premises data centers and public cloud. However, the low-latency performance or sovereignty characteristics of a significant and growing proportion of workloads make them better suited to data and analytics processing where data is generated rather than a centralized on-premises or public cloud environment. ...
Read More
Topics:
Cloud Computing,
Internet of Things,
Data,
Digital Technology,
Analytics & Data,
analytic data platforms,
Operational Data Platforms,
AI and Machine Learning
As articulated in Ventana Research’s Data Platforms Buyer’s Guide and DataOps Buyer’s Guide research, the combination of cloud computing and advanced analytics has lowered the cost of storing and processing large volumes of data, accelerating the emergence of new data platform and data operations products that enable organizations to gain operational efficiency and competitive advantage. The right combination of data platform and data management products is essential to ensure that the right...
Read More
Topics:
Data Management,
Data,
Digital Technology,
data operations,
Analytics & Data,
analytic data platforms,
Operational Data Platforms
The phrase ‘big data’ may have largely gone out of fashion, but the concept of storing and processing all relevant data continues to be important for enterprises seeking to be more data-driven. Doing so requires analytic data platforms capable of storing and processing data in multiple formats and data models. This will be an important focus for the forthcoming Data Platforms Buyer’s Guide 2024.
Read More
Topics:
Analytics,
Business Intelligence,
Data Management,
Data,
Digital Technology,
data operations,
Analytics & Data,
AI and Machine Learning
I recently discussed how fashion has a surprisingly significant role to play in the data market as various architectural approaches to data storage and processing take turns enjoying a phase in the limelight. Pendulum swing is a theory of fashion that describes the periodic movement of trends between two extremes, such as short and long hemlines or skinny and baggy/flared trousers. Pendulum swing theory is similarly a factor in data technology trends, with an example being the oscillation...
Read More
Topics:
Analytics,
Cloud Computing,
Data Management,
Data,
Digital Technology,
data operations,
Analytics & Data,
AI and Machine Learning
I previously described how Oracle had positioned its database portfolio to address any and all data platform requirements. The caveat to that statement at the time was that any organization wanting to take advantage of the company’s flagship Oracle Autonomous Database could only do so using Oracle Cloud Infrastructure (OCI) cloud computing service, their own datacenter or a hybrid cloud environment. The widespread popularity of Oracle Database and the advanced automation capabilities delivered...
Read More
Topics:
Analytics,
Business Intelligence,
Cloud Computing,
Data Management,
Data,
Digital Technology,
Analytics & Data,
analytic data platforms,
Operational Data Platforms,
AI and Machine Learning
I recently articulated some of the reasons why IT teams can fail to deliver on the business requirements for data and analytics projects. This is an age-old and multifaceted problem that is not easily solved. Organizations have a role to play in alleviating the issue by ensuring that their business processes and project planning support a collaborative approach in which business and IT professionals work together. Data and analytics product vendors can also help by delivering products that are...
Read More
Topics:
Cloud Computing,
Data Governance,
Data Management,
Data,
Digital Technology,
Analytics & Data,
AI and Machine Learning
I previously wrote about how document-database providers have added support for ACID transactions and the SQL query language, making their products increasingly suitable for use as replacements for applications that previously depended on relational databases. Adoption of non-relational NoSQL databases is by no means reliant on displacing incumbent relational databases, and initial adoption is often driven by differentiating capabilities, such as developer agility and application flexibility....
Read More
Topics:
Data,
Operational Data Platforms
I have previously written about the functional evolution and emerging use cases for NoSQL databases, a category of non-relational databases that first emerged 15 or so years ago and are now well established as potential alternatives to relational databases. NoSQL is a term used to describe a variety of databases that fall into four primary functional categories: key-value stores, wide-column stores, document-oriented databases and graph databases. Each is worthy of further exploration, which is...
Read More
Topics:
Cloud Computing,
Data,
Digital Technology,
Analytics & Data,
Operational Data Platforms
I previously wrote about the challenge facing distributed SQL database providers to avoid becoming pigeonholed as only being suitable for a niche set of requirements. Factors including performance, reliability, security and scalability provide a focal point for new vendors to differentiate from established providers and get a foot in the door with customer accounts. Expanding and retaining those accounts is not necessarily easy, however, especially as general-purpose data platform providers...
Read More
Topics:
Analytics,
Cloud Computing,
Data,
Digital Technology,
Analytics & Data,
Streaming Data & Events,
analytic data platforms,
AI and Machine Learning
I previously discussed the trust and accuracy limitations of large language models, suggesting that data and analytics vendors provide guidance about potentially inaccurate results and the risks of creating a misplaced level of trust. In the months that have followed, we are seeing some clarity from these vendors about the approaches organizations can take to increase trust and accuracy when developing applications that incorporate generative AI, including fine-tuning and prompt engineering. It...
Read More
Topics:
Analytics,
Business Intelligence,
Data,
Digital Technology,
natural language processing,
Analytics & Data,
analytic data platforms,
Operational Data Platforms,
AI and Machine Learning
I am happy to share insights gleaned from our latest Buyers Guide, an assessment of how well vendors’ offerings meet buyers’ requirements. TheVentana Research 2023 Data Observability Buyers Guide is the distillation of a year of market and product research by Ventana Research. Drawing on our Benchmark Research, we apply a structured methodology built on evaluation categories that reflect the real-world criteria incorporated in a request for proposal to data vendors supporting the spectrum of...
Read More
Topics:
Data,
data operations,
Data Observability
The 2023 Ventana Research Buyers Guide for Data Observability research enables me to provide observations about how the market has advanced.
Read More
Topics:
Data,
data operations,
Data Observability
I am happy to share insights gleaned from our latest Buyers Guide, an assessment of how well vendors’ offerings meet buyers’ requirements. The Ventana Research 2023 Data Orchestration Buyers Guide is the distillation of a year of market and product research by Ventana Research. Drawing on our Benchmark Research, we apply a structured methodology built on evaluation categories that reflect the real-world criteria incorporated in a request for proposal to data vendors supporting the spectrum of...
Read More
Topics:
Data,
data operations,
Data Orchestration
The 2023 Ventana Research Buyers Guide for Data Orchestration research enables me to provide observations about how the market has advanced.
Read More
Topics:
Data,
data operations,
Data Orchestration
I am happy to share insights gleaned from our latest Buyers Guide, an assessment of how well vendors’ offerings meet buyers’ requirements. The Ventana Research 2023 Data Pipelines Buyers Guide is the distillation of a year of market and product research by Ventana Research. Drawing on our Benchmark Research, we apply a structured methodology built on evaluation categories that reflect the real-world criteria incorporated in a request for proposal to Data vendors supporting the spectrum of Data...
Read More
Topics:
Data,
data operations,
Data Pipelines
The 2023 Ventana Research Buyers Guide for Data Pipelines research enables me to provide observations about how the market has advanced.
Read More
Topics:
Data,
data operations,
Data Pipelines
I am happy to share insights gleaned from our latest Buyers Guide, an assessment of how well vendors’ offerings meet buyers’ requirements. The Ventana Research 2023 DataOps Buyers Guide is the distillation of a year of market and product research by Ventana Research. Drawing on our Benchmark Research, we apply a structured methodology built on evaluation categories that reflect the real-world criteria incorporated in a request for proposal to Data vendors supporting the spectrum of DataOps....
Read More
Topics:
Data,
Data Ops,
data operations
I previously described how Databricks had positioned its Lakehouse Platform as the basis for data engineering, data science and data warehousing. The lakehouse design pattern provides a flexible environment for storing and processing data from multiple enterprise applications and workloads for multiple use cases. I assert that by 2025, 8 in 10 current data lake adopters will invest in data lakehouse architecture to improve the business value generated from the accumulated data.
Read More
Topics:
Analytics,
Business Intelligence,
Data Governance,
Data Management,
Data,
Digital Technology,
Analytics & Data,
analytic data platforms,
AI and Machine Learning
The 2023 Ventana Research Buyers Guide for DataOps research enables me to provide observations about how the market has advanced.
Read More
Topics:
Data,
Data Ops,
data operations
The data platforms market may appear to have little or nothing to do with haute couture, but it is one of the data sectors most strongly influenced by the fickle finger of fashion. In recent years, various architectural approaches to data storage and processing have enjoyed a phase in the limelight, including data warehouse, data mart, data hub, data lake, cloud data warehouse, object storage, data lakehouse, data fabric and data mesh. These approaches are often heralded as the next big thing,...
Read More
Topics:
Cloud Computing,
Data Governance,
Data Management,
Data,
Digital Technology,
data operations,
Analytics & Data,
Streaming Data & Events,
analytic data platforms,
Operational Data Platforms,
AI and Machine Learning
Despite a focus on being data-driven, many organizations find that data and analytics projects fail to deliver on expectations. These initiatives can underwhelm for many reasons, because success requires a delicate balance of people, processes, information and technology. Small deviations from perfection in any of those factors can send projects off the rails.
Read More
Topics:
Analytics,
Business Intelligence,
Data Management,
Data,
Digital Technology,
data operations,
AI and Machine Learning
I have written before about the rising popularity of the data fabric approach for managing and governing data spread across distributed environments comprised of multiple data centers, systems and applications. I assert that by 2025, more than 6 in 10 organizations will adopt data fabric technologies to facilitate the management and processing of data across multiple data platforms and cloud environments. The data fabric approach is also proving attractive to vendors, including Microsoft, as a...
Read More
Topics:
business intelligence,
Analytics,
Cloud Computing,
Data Governance,
Data Management,
Data,
Digital Technology,
Analytics & Data,
analytic data platforms,
AI and Machine Learning
Organizations today have an ever-increasing appetite for platforms that improve the speed and efficiency of data analytics and business intelligence (BI). The ability to quickly process data enables organizations to make well-informed decisions in real time. This agile approach to data processing is crucial for staying ahead in today's competitive landscape. With the rising need for data-driven insights, organizations face the difficulty of dealing with massive volumes of distributed business...
Read More
Topics:
Data Management,
Data,
data operations,
analytic data platforms,
Direct Data Mapping
I recently wrote about the various technologies used by organizations to process and analyze data in real time. I explained that while the terms streaming data and events and streaming analytics are often used interchangeably, they are separate disciplines that make use of common underlying concepts and technologies such as events, event brokers and event-driven architecture. Confluent’s acquisition of Immerok earlier this year provided a reminder of this fact. Confluent is one of the most...
Read More
Topics:
Analytics,
Cloud Computing,
Data Governance,
Data,
Digital Technology,
Streaming Analytics,
Streaming Data & Events
Real-time business is a modern phenomenon, and business transformation has accelerated many business events in recent years. However, the execution of business events has always occurred in real time. Rather, it is the processing of the data related to business events that has accelerated instead of the event itself.
Read More
Topics:
Analytics,
Data,
Streaming Analytics,
Streaming Data & Events
It is a mark of the rapid, current pace of development in artificial intelligence (AI) that machine learning (ML) models, until recently considered state of the art, are now routinely being referred to by developers and vendors as “traditional.” Generative AI, and large language models (LLMs) in particular, have taken the AI world by storm in the past year, automating and accelerating the development of content, including text, digital images, audio and video, as well as computer programs and...
Read More
Topics:
business intelligence,
Analytics,
Cloud Computing,
Data Governance,
Data,
Digital Technology,
natural language processing,
Analytics & Data,
analytic data platforms,
AI and Machine Learning
As I have previously explained, we expect an increased demand for intelligent operational applications infused with the results of analytic processes, such as personalization and artificial intelligence-driven recommendations. These systems rely on the analysis of data in the operational data platform to accelerate worker decision-making or improve customer experience.
Read More
Topics:
Analytics,
Data,
Digital Technology,
Streaming Analytics,
Analytics & Data,
Streaming Data & Events,
Operational Data Platforms,
AI and Machine Learning
I have previously written about the functional evolution and emerging use cases for NoSQL databases, a category of non-relational databases that first emerged 15 or so years ago and are now well established as potential alternatives to relational databases. NoSQL is a term used to describe a variety of databases that fall into four primary functional categories: key-value stores, wide-column stores, document-oriented databases and graph databases. Each is worthy of further exploration, which is...
Read More
Topics:
Data,
Operational Data Platforms
The publication of Ventana Research’s 2023 Operational Data Platforms Value Index earlier this year highlighted the importance of incorporating analytic processing into operational applications to deliver personalization and recommendations for workers, partners and customers. This importance is being accelerated by interest in generative AI, especially large language models. The emergence of intelligent applications has impacted the requirements for operational data platforms with the need to...
Read More
Topics:
Analytics,
Cloud Computing,
Data,
Digital Technology,
Analytics & Data,
analytic data platforms,
Operational Data Platforms,
AI and Machine Learning
Despite best intentions, many organizations still struggle with some fundamental aspects of data processing and analytics. Taking data from operational applications and making it available for analysis is a first step, but data management remains a perennial challenge. Data movement and transformation difficulties can lead to delays and data quality problems that prevent organizations from generating value from data. The inability to govern and integrate data from multiple data sources prevents...
Read More
Topics:
Cloud Computing,
Data Management,
Data,
Digital Technology,
data operations,
Analytics & Data
Maintaining data quality and trust is a perennial data management challenge, often preventing organizations from operating at the speed of business. Recent years have seen the emergence of data observability as a category of DataOps focused on monitoring the quality and reliability of data used for analytics and governance projects and associated data pipelines. There is clear overlap with data quality, which is more established as both a discipline and product category for improving trust in...
Read More
Topics:
Data Management,
Data,
data operations
Organizations increasingly rely on real-time analytics to make informed decisions and stay competitive in today’s data-driven business landscape. As the complexity of data grows with the continuous addition of diverse sources, customers and workers alike expect real-time responsiveness. Accelerated query performance is crucial to process and extract valuable insights from data in a timely manner. Traditional analytics applications are often insufficient for managing the scale, velocity and...
Read More
Topics:
Data Management,
Data,
data operations,
Streaming Data & Events,
analytic data platforms
I previously explained the arguments in favor of adoption of distributed SQL databases, the new generation of operational data platforms designed to combine the benefits of the relational database model and native support for distributed cloud architecture. It is critical for distributed SQL vendors to engage with developers to ensure they are considering the importance of resilience that spans multiple data centers and/or cloud regions as they choose the databases that will underpin...
Read More
Topics:
Cloud Computing,
Data,
Digital Technology,
Analytics & Data,
Operational Data Platforms
Data fabric has grown in popularity as organizations struggle to manage data spread across multiple data centers, systems and applications. By providing a technology-driven approach to automating data management and governance across distributed environments, data fabric is attractive to organizations seeking to simplify and standardize data management. I assert that by 2025, more than 6 in 10 organizations will adopt data fabric technologies to facilitate the management and processing of data...
Read More
Topics:
Cloud Computing,
Data Management,
Data,
Digital Technology,
data operations,
Analytics & Data,
analytic data platforms
The data and analytics sector rightly places great importance on data quality: Almost two-thirds (64%) of participants in Ventana Research’s Analytics and Data Benchmark Research cite reviewing data for quality and consistency issues as the most time-consuming task in analyzing data. Data and analytics vendors would not recommend that customers use tools known to have data quality problems. It is somewhat surprising, therefore, that data and analytics vendors are rushing to encourage customers...
Read More
Topics:
Analytics,
Data Governance,
Data Management,
Data,
Digital Technology,
natural language processing,
Analytics & Data,
AI and Machine Learning
Master data management may not attract the same level of excitement as fashionable topics such as DataOps or Data Platforms, but it remains one of the most significant aspects of an organization’s strategic approach to data management. Having trust in data is critical to the ability of an organization to make data-driven business decisions. Along with data quality, MDM enables organizations to ensure data is accurate, complete and consistent to fulfill operational business objectives.
Read More
Topics:
Data Governance,
Data Management,
Data,
data operations
The data platforms market has traditionally been divided between products specifically designed to support operational or analytic workloads, with other market segments having emerged in recent years for data platforms targeted specifically at data science and machine learning (ML), as well as real-time analytics. More recently, we have seen vendor strategies evolving to provide a more consolidated approach, with data platforms designed to address a combination of analytics and data science, as...
Read More
Topics:
Analytics,
Business Intelligence,
Cloud Computing,
Data,
Digital Technology,
Analytics & Data,
analytic data platforms,
Operational Data Platforms,
AI and Machine Learning
The recent publication of our Value Index research highlights the impact of intelligent applications on the operational data platforms sector. While we continue to believe that, for most use cases, there is a clear, functional requirement for either analytic or operational data platforms, recent growth in the development of intelligent applications infused with the results of analytic processes, such as personalization and artificial intelligence (AI)-driven recommendations, has increasing...
Read More
Topics:
Analytics,
Business Intelligence,
Cloud Computing,
Data,
Digital Technology,
Analytics & Data,
analytic data platforms,
Operational Data Platforms
As engagement with customers, suppliers and partners is increasingly conducted through digital channels, ensuring that infrastructure and applications are performing as expected is not just important but mission critical. My colleague, David Menninger, recently explained the increasing importance of observability to enable organizations to ensure that their systems and applications are operating efficiently. Observability has previously been the domain of the IT department but is increasingly...
Read More
Topics:
Data Management,
Data,
Digital Technology,
Analytics & Data
I have written recently about the increasing importance of managing data in motion and at rest as the use of streaming data by enterprise organizations becomes more mainstream. While batch-based processing of application data has been a core component of enterprise IT architecture for decades, streaming data and event processing have often been niche disciplines typically reserved for organizations with the highest-level performance requirements. That has changed in recent years, driven by an...
Read More
Topics:
Data,
Streaming Data & Events
To execute more data-driven business strategies, organizations need linked and comprehensive data that is available in real time. By consistently managing data across siloed systems and ensuring that data definitions are agreed and current, organizations can overcome the challenges presented by data being distributed across an increasingly disparate range of applications and data-processing locations. Maintaining data quality is a perennial data management challenge, often preventing...
Read More
Topics:
Data Management,
Data,
data operations
Data Operations (DataOps) has been part of the lexicon of the data market for almost a decade, with the term used to describe products, practices and processes designed to support agile and continuous delivery of data analytics. DataOps takes inspiration from DevOps, which describes a set of tools, practices and philosophy used to support the continuous delivery of software applications in the face of constant changes. DataOps describes a set of tools, practices and philosophy used to ensure...
Read More
Topics:
Data Governance,
Data Management,
Data,
data operations
As data continues to grow and evolve, organizations seek better tools and technologies to employ data faster and more efficiently. Finding and managing data remains a perennial challenge for most organizations, and is exacerbated by increasing volumes of data and an expanding array of data formats. At the same time, organizations must comply with a growing list of national and regional rules and regulations, such as General Data Protection Regulation and the California Consumer Privacy Act....
Read More
Topics:
Data Governance,
Data Management,
Data,
data operations
Success with streaming data and events requires a more holistic approach to managing and governing data in motion and data at rest. The use of streaming data and event processing has been part of the data landscape for many decades. For much of that time, data streaming was a niche activity, however, with standalone data streaming and event-processing projects run in parallel with existing batch-processing initiatives, utilizing operational and analytic data platforms. I noted that there has...
Read More
Topics:
Analytics,
Data,
Digital Technology,
Streaming Analytics,
Analytics & Data,
Streaming Data & Events,
analytic data platforms,
Operational Data Platforms
I have previously written about the importance of data democratization as a key element of a data-driven agenda. Removing barriers that prevent or delay users from gaining access to data enables it to be treated as a product that is generated and consumed, either internally by employees or externally by partners and customers. This is particularly important for organizations adopting the data mesh approach to data ownership, access and governance. Data mesh is an organizational and cultural...
Read More
Topics:
Cloud Computing,
Data Governance,
Data Management,
Data,
Digital Technology,
data operations,
Analytics & Data
I recently wrote about the potential use cases for distributed SQL databases as well as techniques being employed by vendors to accelerate adoption. Distributed SQL is a term that is used by several vendors to describe operational data platform products that combine the benefits of the relational database model and native support for distributed cloud architecture, including resilience that spans multiple data centers and/or cloud regions. I noted that compatibility with existing database tools...
Read More
Topics:
Cloud Computing,
Data,
Digital Technology,
Analytics & Data,
Operational Data Platforms
Organizations require faster analytics to continuously improve business operations and stay competitive in today’s market. However, many struggle with slow analytics due to a variety of factors such as slow databases, insufficient data storage capacity, poor data quality, lack of proper data cleansing and inadequate IT infrastructure. Challenges such as data silos can also decrease operational efficiency. And as the data grows, performing complex data modelling becomes challenging for users as...
Read More
Topics:
Data Management,
Data,
analytic data platforms
I have previously written about the functional evolutionand emerging use cases for NoSQL databases, a category of non-relational databases that first emerged 15 or so years ago and are now well established as potential alternatives to relational databases. NoSQL is a term that is used to describe a variety of databases that fall into four primary functional categories: key-value stores, wide column stores, document-oriented databases and graph databases. Each is worthy of further exploration,...
Read More
Topics:
Data,
Operational Data Platforms
The market for data and analytics products is constantly evolving, with the emergence of new approaches to data persistence, data processing and analytics. This enables organizations to constantly adapt data analytics architecture in response to emerging functional capabilities and business requirements. It can, however, also be a challenge. Investments in data platforms cannot be constantly written-off as organizations adopt new products for new approaches. Too little change can lead to...
Read More
Topics:
Data Governance,
Data Management,
Data,
data operations
Data observability was a hot topic in 2022 and looks likely to be a continued area of focus for innovation in 2023 and beyond. As I have previously described, data observability software is designed to automate the monitoring of data platforms and data pipelines, as well as the detection and remediation of data quality and data reliability issues. There has been a Cambrian explosion of data observability software vendors in recent years, and while they have fundamental capabilities in common,...
Read More
Topics:
Cloud Computing,
Data Management,
Data,
Digital Technology,
data operations,
Analytics & Data
I have written about the increased demand for data-intensive operational applications infused with the results of analytic processes, such as personalization and artificial intelligence-driven recommendations. I previously described the use of hybrid data processing to enable analytics on application data within operational data platforms. As is often the case in the data platforms sector, however, there is more than one way to peel an orange. Recent years have also seen the emergence of...
Read More
Topics:
Cloud Computing,
Data,
Digital Technology,
Analytics & Data,
analytic data platforms,
Operational Data Platforms
Organizations across various industries collect multiple types of data from disparate systems to answer key business questions and deliver personalized experiences for customers. The expanding volume of data increases complexity, and data management becomes a challenge if the process is manual and rules-based. There can be numerous siloed, incomplete and outdated data sources that result in inaccurate results. Organizations must also deal with concurrent errors – from customers to products to...
Read More
Topics:
Data Governance,
Data Management,
Data,
data operations,
analytic data platforms
Despite the emphasis on organizations being more data-driven and making an increasing proportion of business decisions based on data and analytics, it remains the case that some of the most fundamental questions about an organization are difficult to answer using data and analytics. Ostensibly simple questions such as, “how many customers does the organization have?” can be fiendishly difficult to answer, especially for organizations with multiple business entities, regions, departments and...
Read More
Topics:
Cloud Computing,
Data Management,
Data,
data operations,
Analytics & Data,
AI and Machine Learning
I am happy to share insights from our latest Ventana Research Value Index research, which assesses how well vendors’ offerings meet buyers’ requirements. The 2023 Analytic Data Platforms Value Index is the distillation of a year of market and product research by Ventana Research. Drawing on our Benchmark Research, we apply a structured methodology built on evaluation categories that reflect real-world criteria incorporated in a request for proposal to data platform vendors supporting the...
Read More
Topics:
Cloud Computing,
Data,
Digital Technology,
Analytics & Data,
analytic data platforms
Ventana Research recently published the 2023 Analytic Data Platforms Value Index. As organizations strive to be more data-driven, increasing reliance on data as a fundamental factor in business decision-making, the importance of the analytic data platform has never been greater. In this post, I’ll share some of my observations about how the analytic data platforms market is evolving.
Read More
Topics:
Cloud Computing,
Data,
Digital Technology,
Analytics & Data,
analytic data platforms,
Operational Data Platforms
I am happy to share insights from our latest Ventana Research Value Index research, which assesses how well vendors’ offerings meet buyers’ requirements. The 2023 Operational Data Platforms Value Index is the distillation of a year of market and product research by Ventana Research. Drawing on our Benchmark Research, we apply a structured methodology built on evaluation categories that reflect real-world criteria incorporated in a request for proposal to data platform vendors supporting the...
Read More
Topics:
Cloud Computing,
Data,
Digital Technology,
Analytics & Data,
operational data plaftforms
Ventana Research recently announced its 2023 Market Agenda for Data, continuing the guidance we have offered for two decades to help organizations derive optimal value and improve business outcomes.
Read More
Topics:
Cloud Computing,
Data Governance,
Data Management,
Data,
Digital Technology,
data operations,
Analytics & Data,
Streaming Data & Events,
analytic data platforms,
Operational Data Platforms
Ventana Research recently published the 2023 Operational Data Platforms Value Index. The importance of the operational data platform has never been greater as organizations strive to be more data-driven, incorporating intelligence into operational applications via personalization and recommendations for workers, partners and customers. In this post, I’ll share some of my observations on how the operational data platforms market is evolving.
Read More
Topics:
Cloud Computing,
Data,
Analytics & Data,
analytic data platforms,
operational data plaftforms
I am happy to share insights from our latest Ventana Research Value Index, which assesses how well vendors’ offerings meet buyers’ requirements. The 2023 Data Platforms Value Index is the distillation of a year of market and product research by Ventana Research. Drawing on our Benchmark Research, we apply a structured methodology built on evaluation categories that reflect real-world criteria incorporated in a request for proposal to data platform vendors that support the spectrum of...
Read More
Topics:
Cloud Computing,
Data,
Digital Technology,
Analytics & Data,
analytic data platforms,
Operational Data Platforms
Data observability is a hot topic and trend. I have written about the importance of data observability for ensuring healthy data pipelines, and have covered multiple vendors with data observability capabilities, offered both as standalone and part of a larger data engineering system. Data observability software provides an environment that takes advantage of machine learning and DataOps to automate the monitoring of data quality and reliability. The term has been adopted by multiple vendors...
Read More
Topics:
Cloud Computing,
Data Management,
Data,
Digital Technology,
data operations
Having recently completed the 2023 Data Platforms Value Index, I want to share some of my observations about how the market is evolving. Although this is our inaugural assessment of the market for data platforms, the sector is mature and products from many of the vendors we assess can be used to effectively support operational and analytic use cases.
Read More
Topics:
Cloud Computing,
Data,
Digital Technology,
Analytics & Data,
analytic data platforms,
Operational Data Platforms
The shift from on-premises server infrastructure to cloud-based and software-as-a-service (SaaS) models has had a profound impact on the data and analytics architecture of many organizations in recent years. More than one-half of participants (59%) in Ventana Research’s Analytics and Data Benchmark research are deploying data and analytics workloads in the cloud, and a further 30% plan to do so. Customer demand for cloud-based consumption models has also had a significant impact on the products...
Read More
Topics:
business intelligence,
Cloud Computing,
Data Management,
Data,
natural language processing,
data operations,
Analytics & Data,
analytic data platforms,
Operational Data Platforms,
AI and Machine Learning
There is always space for innovation in the data platforms sector, and new vendors continue to emerge at regular intervals with new approaches designed to serve specialist data storage and processing requirements. Factors including performance, reliability, security and scalability provide a focal point for new vendors to differentiate from established vendors, especially for the most demanding operational or analytic data platform requirements. It is never easy, however, for developers of new...
Read More
Topics:
Cloud Computing,
Data,
Operational Data Platforms
Earlier this year, I wrote about the increasing importance of data observability, an emerging product category that takes advantage of machine learning (ML) and Data Operations (DataOps) to automate the monitoring of data used for analytics projects to ensure its quality and lineage. Monitoring the quality and lineage of data is nothing new. Manual tools exist to ensure that it is complete, valid and consistent, as well as relevant and free from duplication. Data observability vendors,...
Read More
Topics:
business intelligence,
Cloud Computing,
Data Management,
Data,
data operations
One of the most significant considerations when choosing an analytic data platform is performance. As organizations compete to benefit most from being data-driven, the lower the time to insight the better. As data practitioners have learnt over time, however, lowering time to insight is about more than just high-performance queries. There are opportunities to improve time to insight throughout the analytics life cycle, which starts with data ingestion and integration, includes data preparation...
Read More
Topics:
business intelligence,
Data,
data operations,
analytic data platforms,
AI and Machine Learning
Organizations are increasingly utilizing cloud object storage as the foundation for analytic initiatives. There are multiple advantages to this approach, not least of which is enabling organizations to keep higher volumes of data relatively inexpensively, increasing the amount of data queried in analytics initiatives. I assert that by 2024, 6 in ten organizations will use cloud-based technology as the primary analytics data platform, making it easier to adopt and scale operations as necessary.
Read More
Topics:
Teradata,
Data Governance,
Data Management,
Data,
analytic data platforms,
operational data plaftforms,
Object storage,
vantage platforms
Almost all organizations are investing in data science, or planning to, as they seek to encourage experimentation and exploration to identify new business challenges and opportunities as part of the drive toward creating a more data-driven culture. My colleague, David Menninger, has written about how organizations using artificial intelligence and machine learning (AI/ML) report gaining competitive advantage, improving customer experiences, responding faster to opportunities and threats, and...
Read More
Topics:
Data Governance,
Data Management,
Data,
data operations,
Analytics & Data,
analytic data platforms,
AI and Machine Learning
I have previously written about growing interest in the data lakehouse as one of the design patterns for delivering hydroanalytics analysis of data in a data lake. Many organizations have invested in data lakes as a relatively inexpensive way of storing large volumes of data from multiple enterprise applications and workloads, especially semi- and unstructured data that is unsuitable for storing and processing in a data warehouse. However, early data lake projects lacked structured data...
Read More
Topics:
Business Intelligence,
Data Governance,
Data Management,
Data,
Streaming Data & Events,
analytic data platforms,
AI and Machine Learning
I have written recently about the similarities and differences between data mesh and data fabric. The two are potentially complementary. Data mesh is an organizational and cultural approach to data ownership, access and governance. Data fabric is a technical approach to automating data management and data governance in a distributed architecture. There are various definitions of data fabric, but key elements include a data catalog for metadata-driven data governance and self-service, agile data...
Read More
Topics:
business intelligence,
Cloud Computing,
Data Governance,
Data Management,
Data,
data operations,
operational data plaftforms,
AI and Machine Learning
In their pursuit to be data-driven, organizations are collecting and managing more data than ever before as they attempt to gain competitive advantage and respond faster to worker and customer demands for more innovative, data-rich applications and personalized experiences. As data is increasingly spread across multiple data centers, clouds and regions, organizations need to manage data on multiple systems in different locations and bring it together for analysis. As the data volumes increase...
Read More
Topics:
Data Management,
Data,
data operations,
analytic data platforms
I have written a few times in recent months about vendors offering functionality that addresses data orchestration. This is a concept that has been growing in popularity in the past five years amid the rise of Data Operations (DataOps), which describes more agile approaches to data integration and data management. In a nutshell, data orchestration is the process of combining data from multiple operational data sources and preparing and transforming it for analysis. To those unfamiliar with the...
Read More
Topics:
Data Management,
Data,
data operations,
Analytics & Data,
AI and Machine Learning
Ventana Research’s Data Lakes Dynamics Insights research illustrates that while data lakes are fulfilling their promise of enabling organizations to economically store and process large volumes of raw data, data lake environments continue to evolve. Data lakes were initially based primarily on Apache Hadoop deployed on-premises but are now increasingly based on cloud object storage. Adopters are also shifting from data lakes based on homegrown scripts and code to open standards and open...
Read More
Topics:
Business Intelligence,
Cloud Computing,
Data Governance,
Data Management,
Data,
data operations,
Analytics & Data,
analytic data platforms,
Operational Data Platforms,
AI and Machine Learning
I have written before about the continued use of specialist operational and analytic data platforms. Most database products can be used for operational or analytic workloads, and the number of use cases for hybrid data processing is growing. However, a general-purpose database is unlikely to meet the most demanding operational or analytic data platform requirements. Factors including performance, reliability, security and scalability necessitate the use of specialist data platforms. I assert...
Read More
Topics:
business intelligence,
Cloud Computing,
Data Management,
Data,
Analytics & Data,
analytic data platforms
Earlier this year I described the growing use-cases for hybrid data processing. Although it is anticipated that the majority of database workloads will continue to be served by specialist data platforms targeting operational and analytic workloads respectively, there is increased demand for intelligent operational applications infused with the results of analytic processes, such as personalization and artificial intelligence-driven recommendations. There are multiple data platform approaches to...
Read More
Topics:
Business Intelligence,
Cloud Computing,
Data,
Streaming Data & Events,
analytic data platforms,
Operational Data Platforms,
AI and Machine Learning
I have recently written about the organizational and cultural aspects of being data-driven, and the potential advantages data-driven organizations stand to gain by responding faster to worker and customer demands for more innovative, data-rich applications and personalized experiences. I have also explained that data-driven processes require more agile, continuous data processing, with an increased focus on extract, load and transform processes — as well as change data capture and automation...
Read More
Topics:
Cloud Computing,
Data Management,
Data,
data operations,
Analytics & Data
The data catalog has become an integral component of organizational data strategies over the past decade, serving as a conduit for good data governance and facilitating self-service analytics initiatives. The data catalog has become so important, in fact, that it is easy to forget that just 10 years ago it did not exist in terms of a standalone product category. Metadata-based data management functionality has had a role to play within products for data governance and business intelligence for...
Read More
Topics:
business intelligence,
Data Governance,
Data Management,
Data,
data operations,
Analytics & Data
I recently wrote about the need for organizations to take a holistic approach to the management and governance of data in motion alongside data at rest. As adoption of streaming data and event processing increases, it is no longer sufficient for streaming data projects to exist in isolation. Data needs to be managed and governed regardless of whether it is processed in batch or as a stream of events. This requirement has resulted in established data management vendors increasing their focus on...
Read More
Topics:
Big Data,
Cloud Computing,
Data Governance,
Streaming Analytics,
Streaming Data & Events
I have written recently about increased demand for data-intensive applications infused with the results of analytic processes, such as personalization and artificial intelligence (AI)-driven recommendations. Almost one-quarter of respondents (22%) to Ventana Research’s Analytics and Data Benchmark Research are currently analyzing data in real time, with an additional 10% analyzing data every hour. There are multiple data platform approaches to delivering real-time data processing and analytics...
Read More
Topics:
Cloud Computing,
Data,
Streaming Analytics,
Analytics & Data,
Streaming Data & Events,
analytic data platforms,
operational data plaftforms
I recently noted that as demand for real-time interactive applications becomes more pervasive, the use of streaming data is becoming more mainstream. Streaming data and event processing has been part of the data landscape for many decades, but for much of that time, data streaming was a niche activity. Although adopted in industry segments with high-performance, real-time data processing and analytics requirements such as financial services and telecommunications, data streaming was far less...
Read More
Topics:
Big Data,
Data,
Streaming Analytics,
Analytics & Data,
Streaming Data & Events
I have recently written about the importance of healthy data pipelines to ensure data is integrated and processed in the sequence required to generate business intelligence, and the need for data pipelines to be agile in the context of real-time data processing requirements. Data engineers, who are responsible for monitoring, managing and maintaining data pipelines, are under increasing pressure to deliver high-performance and flexible data integration and processing pipelines that are capable...
Read More
Topics:
Big Data,
Cloud Computing,
Data Management,
Data,
data operations
I recently explained how emerging application requirements were expanding the range of use cases for NoSQL databases, increasing adoption based on the availability of enhanced functionality. These intelligent applications require a close relationship between operational data platforms and the output of data science and machine learning projects. This ensures that machine learning and predictive analytics initiatives are not only developed and trained based on the relationships inherent in...
Read More
Topics:
Business Intelligence,
Data,
analytic data platforms,
Operational Data Platforms,
AI and Machine Learning
Streaming data has been part of the industry landscape for decades but has largely been focused on niche applications in segments with the highest real-time data processing and analytics performance requirements, such as financial services and telecommunications. As demand for real-time interactive applications becomes more pervasive, streaming data is becoming a more mainstream pursuit, aided by the proliferation of open-source streaming data and event technologies, which have lowered the cost...
Read More
Topics:
Data,
Streaming Analytics,
Streaming Data & Events,
operational data plaftforms
When joining Ventana Research, I noted that the need to be more data-driven has become a mantra among large and small organizations alike. Data-driven organizations stand to gain competitive advantage, responding faster to worker and customer demands for more innovative, data-rich applications and personalized experiences. Being data-driven is clearly something to aspire to. However, it is also a somewhat vague concept without clear definition. We know data-driven organizations when we see them...
Read More
Topics:
embedded analytics,
Analytics,
Business Intelligence,
Data Governance,
Data Integration,
Data,
Digital Technology,
natural language processing,
data lakes,
data operations,
Streaming Analytics,
Digital Business,
Data Platforms,
Analytics & Data,
Streaming Data & Events,
AI and Machine Learning
I recently wrote about the growing range of use cases for which NoSQL databases can be considered, given increased breadth and depth of functionality available from providers of the various non-relational data platforms. As I noted, one category of NoSQL databases — graph databases — are inherently suitable for use cases that rely on relationships, such as social media, fraud detection and recommendation engines, since the graph data model represents the entities and values and also the...
Read More
Topics:
business intelligence,
Analytics,
Cloud Computing,
Data,
Digital Technology,
Data Platforms,
Analytics & Data,
AI and Machine Learning
I previously described the concept of hydroanalytic data platforms, which combine the structured data processing and analytics acceleration capabilities associated with data warehousing with the low-cost and multi-structured data storage advantages of the data lake. One of the key enablers of this approach is interactive SQL query engine functionality, which facilitates the use of existing business intelligence (BI) and data science tools to analyze data in data lakes. Interactive SQL query...
Read More
Topics:
business intelligence,
Analytics,
Cloud Computing,
Data,
Digital Technology,
data lakes,
data operations,
Data Platforms,
Analytics & Data,
AI and Machine Learning
I previously explained how the data lakehouse is one of two primary approaches being adopted to deliver what I have called a hydroanalytic data platform. Hydroanalytics involves the combination of data warehouse and data lake functionality to enable and accelerate analysis of data in cloud storage services. The term data lakehouse has been rapidly adopted by several vendors in recent years to describe an environment in which data warehousing functionality is integrated into the data lake...
Read More
Topics:
Analytics,
Business Intelligence,
Data,
data lakes,
Data Platforms
As I recently described, it is anticipated that the majority of database workloads will continue to be served by specialist data platforms targeting operational and analytic workloads, albeit with growing demand for hybrid data processing use-cases and functionality. Specialist operational and analytic data platforms have historically been the since preferred option, but there have always been general-purpose databases that could be used for both analytic and operational workloads, with tuning...
Read More
Topics:
Analytics,
Business Intelligence,
Cloud Computing,
Data,
Digital Technology,
Data Platforms,
Analytics & Data
I recently wrote about the potential benefits of data mesh. As I noted, data mesh is not a product that can be acquired, or even a technical architecture that can be built. It’s an organizational and cultural approach to data ownership, access and governance. While the concept of data mesh is agnostic to the technology used to implement it, technology is clearly an enabler for data mesh. For many organizations, new technological investment and evolution will be required to facilitate adoption...
Read More
Topics:
Analytics,
Business Intelligence,
Data Governance,
Data Integration,
Data,
data operations,
Data Platforms,
Streaming Data & Events,
AI and Machine Learning
I recently described the use cases driving interest in hybrid data processing capabilities that enable analysis of data in an operational data platform without impacting operational application performance or requiring data to be extracted to an external analytic data platform. Hybrid data processing functionality is becoming increasingly attractive to aid the development of intelligent applications infused with personalization and artificial intelligence-driven recommendations. These...
Read More
Topics:
Analytics,
Business Intelligence,
Cloud Computing,
Data,
Digital Technology,
Data Platforms,
Analytics & Data,
AI and Machine Learning
The server is a key component of enterprise computing, providing the functional compute resources required to support software applications. Historically, the server was so fundamentally important that it – along with the processor, or processor core – was also a definitional unit by which software was measured, priced and sold. That changed with the advent of cloud-based service delivery and consumption models.
Read More
Topics:
Data,
Digital Technology,
Digital Business,
Data Platforms,
Analytics & Data
Over a decade ago, I coined the term NewSQL to describe the new breed of horizontally scalable, relational database products. The term was adopted by a variety of vendors that sought to combine the transactional consistency of the relational database model with elastic, cloud-native scalability. Many of the early NewSQL vendors struggled to gain traction, however, and were either acquired or ceased operations before they could make an impact in the crowded operational data platforms market....
Read More
Topics:
Business Continuity,
Cloud Computing,
Data,
Digital Technology,
Digital Business,
Data Platforms,
Analytics & Data
I recently wrote about the importance of data pipelines and the role they play in transporting data between the stages of data processing and analytics. Healthy data pipelines are necessary to ensure data is integrated and processed in the sequence required to generate business intelligence. The concept of the data pipeline is nothing new of course, but it is becoming increasingly important as organizations adapt data management processes to be more data driven.
Read More
Topics:
business intelligence,
Analytics,
Data Governance,
Data Integration,
Data,
Digital Technology,
Digital transformation,
data lakes,
data operations,
Digital Business,
Data Platforms,
Analytics & Data,
Streaming Data & Events,
AI and Machine Learning
Data governance is an issue that impacts all organizations large and small, new and old, in every industry, and every region of the world. Data governance ensures that an organization’s data can be cataloged, trusted and protected, improving business processes to accelerate analytics initiatives and support compliance with regulatory requirements. Not all data governance initiatives will be driven by regulatory compliance; however, the risk of falling foul of privacy (and human rights) laws...
Read More
Topics:
Analytics,
Data Governance,
Data
I recently described the growing level of interest in data mesh which provides an organizational and cultural approach to data ownership, access and governance that facilitates distributed data processing. As I stated in my Analyst Perspective, data mesh is not a product that can be acquired or even a technical architecture that can be built. Adopting the data mesh approach is dependent on people and process change to overcome traditional reliance on centralized ownership of data and...
Read More
Topics:
Business Continuity,
business intelligence,
Analytics,
Data Governance,
Data Integration,
Data,
Digital Technology,
data lakes,
Digital Business,
Data Platforms,
Analytics & Data
Data mesh is the latest trend to grip the data and analytics sector. The term has been rapidly adopted by numerous vendors — as well as a growing number of organizations —as a means of embracing distributed data processing. Understanding and adopting data mesh remains a challenge, however. Data mesh is not a product that can be acquired, or even a technical architecture that can be built. It is an organizational and cultural approach to data ownership, access and governance. Adopting data mesh...
Read More
Topics:
business intelligence,
Analytics,
Data Governance,
Data Integration,
Data,
Digital Technology,
Digital transformation,
data lakes,
data operations,
Digital Business,
Data Platforms,
Analytics & Data,
Streaming Data & Events
Despite widespread and increasing use of the cloud for data and analytics workloads, it has become clear in recent years that, for most organizations, a proportion of data-processing workloads will remain on-premises in centralized data centers or distributed-edge processing infrastructure. As we recently noted, as compute and storage are distributed across a hybrid and multi-cloud architecture, so, too, is the data it stores and relies upon. This presents challenges for organizations to...
Read More
Topics:
business intelligence,
Analytics,
Data Governance,
Data,
data operations,
Data Platforms,
AI and Machine Learning
I recently examined how evolving functionality had fueled the adoption of NoSQL databases, recommending that organizations evaluate NoSQL databases when assessing options for data transformation and modernization efforts. This recommendation was based on the breadth and depth of functionality offered by NoSQL database providers today, which has expanded the range of use cases for which NoSQL databases are potentially viable. There remain a significant number of organizations that have not...
Read More
Topics:
NoSQL,
Data,
Data Platforms,
Use Cases
The various NoSQL databases have become a staple of the data platforms landscape since the term entered the IT industry lexicon in 2009 to describe a new generation of non-relational databases. While NoSQL began as a ragtag collection of loosely affiliated, open-source database projects, several commercial NoSQL database providers are now established as credible alternatives to the various relational database providers, while all the major cloud providers and relational database giants now also...
Read More
Topics:
Analytics,
Data,
Data Platforms,
AI and Machine Learning
As businesses become more data-driven, they are increasingly dependent on the quality of their data and the reliability of their data pipelines. Making decisions based on data does not guarantee success, especially if the business cannot ensure that the data is accurate and trustworthy. While there is potential value in capturing all data — good or bad — making decisions based on low-quality data may do more harm than good.
Read More
Topics:
Data Governance,
Data Integration,
Data,
Digital Technology,
data lakes,
data operations,
Analytics & Data
I recently described the emergence of hydroanalytic data platforms, outlining how the processes involved in generating energy from a lake or reservoir were analogous to those required to generate intelligence from a data lake. I explained how structured data processing and analytics acceleration capabilities are the equivalent of turbines, generators and transformers in a hydroelectric power station. While these capabilities are more typically associated with data warehousing, they are now...
Read More
Topics:
Analytics,
Data Governance,
Data Integration,
Data,
data lakes,
data operations,
Data Platforms,
Streaming Data & Events,
AI and Machine Learning
As I stated when joining Ventana Research, the socioeconomic impacts of the pandemic and its aftereffects have highlighted more than ever the differences between organizations that can turn data into insights and are agile enough to act upon it and those that are incapable of seeing or responding to the need for change. Data-driven organizations stand to gain competitive advantage, responding faster to worker and customer demands for more innovative, data-rich applications and personalized...
Read More
Topics:
business intelligence,
Analytics,
Data Integration,
Data,
data lakes,
data operations,
Data Platforms,
Streaming Data & Events,
AI and Machine Learning
I recently described how the data platforms landscape will remain divided between analytic and operational workloads for the foreseeable future. Analytic data platforms are designed to store, manage, process and analyze data, enabling organizations to maximize data to operate with greater efficiency, while operational data platforms are designed to store, manage and process data to support worker-, customer- and partner-facing operational applications. At the same time, however, we see...
Read More
Topics:
embedded analytics,
Analytics,
Business Intelligence,
Data,
Digital Technology,
Data Platforms,
Analytics & Data,
Streaming Data & Events,
Natural Data,
AI and Machine Learning
Ventana Research recently announced its 2022 Market Agenda for Data, continuing the guidance we have offered for nearly two decades to help organizations derive optimal value and improve business outcomes.
Read More
Topics:
Data Governance,
Data Integration,
Data,
data lakes,
data operations,
Data Platforms,
Streaming Data & Events
Few trends have had a bigger impact on the data platforms landscape than the emergence of cloud computing. The adoption of cloud computing infrastructure as an alternative to on-premises datacenters has resulted in significant workloads being migrated to the cloud, displacing traditional server and storage vendors. Almost one-half (49%) of respondents to Ventana Research’s Analytics and Data Benchmark Research currently use cloud computing products for analytics and data, and a further...
Read More
Topics:
Analytics,
Business Intelligence,
Data Governance,
Data Integration,
Data
The need for data-driven decision-making requires organizations to transform not only the approach to business intelligence and data science but also accelerate the development of new operational applications that support greater business agility, enable cloud- and mobile-based consumption, and deliver more interactive and personalized experiences. To stay competitive, organizations need to prioritize the development of new, data-driven applications. As a result, many have been encouraged to...
Read More
Topics:
Analytics,
Cloud Computing,
Analytics & Data,
AI and Machine Learning
The term NoSQL has been a misnomer ever since it appeared in 2009 to describe a group of emerging databases. It was true that a lack of support for Structured Query Language (SQL) was common to the various databases referred to as NoSQL. However, it was always one of a number of common characteristics, including flexible schema, distributed data processing, open source licensing, and the use of non-relational data models (key value, document, graph) rather than relational tables. As the various...
Read More
Topics:
Business Continuity,
Analytics,
Data,
Digital Technology,
Digital Business,
Data Platforms
Data lakes have enormous potential as a source of business intelligence. However, many early adopters of data lakes have found that simply storing large amounts of data in a data lake environment is not enough to generate business intelligence from that data. Similarly, lakes and reservoirs have enormous potential as sources of energy. However, simply storing large amounts of water in a lake is not enough to generate energy from that water. A hydroelectric power station is required to harness...
Read More
Topics:
Analytics,
Business Intelligence,
Cloud Computing,
Data Governance,
Data Integration,
Data,
Digital Technology,
data lakes,
data operations,
Data Platforms,
AI and Machine Learning
As I noted when joining Ventana Research, the range of options faced by organizations in relation to data processing and analytics can be bewildering. When it comes to data platforms, however, there is one fundamental consideration that comes before all others: Is the workload primarily operational or analytic? Although most database products can be used for operational or analytic workloads, the market has been segmented between products targeting operational workloads, and those targeting...
Read More
Topics:
Analytics,
Business Intelligence,
Data,
data lakes,
data operations,
Data Platforms,
AI and Machine Learning
Breaking into the database market as a new vendor is easier said than done given the dominance of the sector by established database and data management giants, as well as the cloud computing providers. We recently described the emergence of a new breed of distributed SQL database providers with products designed to address hybrid and multi-cloud data processing. These databases are architecturally and functionally differentiated from both the traditional relational incumbents (in terms of...
Read More
Topics:
Cloud Computing,
Data,
Digital Technology,
Digital transformation,
Digital Business,
Data Platforms,
Analytics & Data
It has been clear for some time that future enterprise IT architecture will span multiple cloud providers as well as on-premises data centers. As Ventana Research noted in the market perspective on data architectures, the rapid adoption of cloud computing has fragmented where data is accessed or consolidated. We are already seeing that almost one-half (49%) of respondents to Ventana Research’s Analytics and Data Benchmark Research are using cloud computing for analytics and data, of which 42%...
Read More
Topics:
Data,
data lakes,
data operations,
Data Platforms
Enterprises looking to adopt cloud-based data processing and analytics face a disorienting array of data storage, data processing, data management and analytics offerings. Departmental autonomy, shadow IT, mergers and acquisitions, and strategic choices mean that most enterprises now have the need to manage data across multiple locations, while each of the major cloud providers and data and analytics vendors has a portfolio of offerings that may or may not be available in any given location. As...
Read More
Topics:
Analytics,
Cloud Computing,
Data Governance,
Data Integration,
Data,
Digital Technology,
data lakes,
data operations,
Data Platforms,
AI and Machine Learning
I am very happy to announce that I have joined Ventana Research to help lead the expertise area of Digital Technology, including Analytics and Data, Cloud Computing, Artificial Intelligence and Machine Learning, the Internet of Things, Robotic Automation, and Collaborative and Conversational Computing. While the breadth of applications and technology covered by our Digital Technology practice is broad, I will naturally make use of my decades of experience covering data platforms and analytics...
Read More
Topics:
Digital Technology