Matt Aslett's Analyst Perspectives

The adoption of cloud environments for analytic workloads has been a key feature of the data platforms sector in recent years. For two-thirds (66%) of participants in ISG’s Data Lake Dynamic Insights Research, the primary data platform used for analytics is cloud based. Many enterprises adopted cloud-based analytic data platforms with a view to improving operational efficiencies by reducing the need for upfront investment in physical infrastructure as well as the ability to scale cloud services up and down to match fluctuating requirements. As adoption has grown, some enterprises found that the theoretical advantages of data processing in the cloud can be more challenging to deliver in practice, with constant monitoring and manual intervention required to optimize resources and realize potential savings. This fostered the emergence of an ecosystem of software providers, including Capital One Software, with products designed to optimize the efficient use of cloud analytic data platforms.

Capital One Software was launched in 2022 to build a business around Capital One Slingshot. Slingshot is a data management software product initially developed by Capital One Financial Corporation to accelerate and manage its internal adoption of Snowflake’s cloud-based analytic data platform. The journey to the launch of Capital One Software can be traced back to 2013 and the adoption by Capital One’s internal IT team of a new agile development approach that embraced RESTful application programming interfaces, decentralized and collaborative development and cloud computing. Capital One began its transition to a cloud-first company in 2016 and completed its migration away from on-premises data centers to the cloud in 2020. Along the way, it adopted Snowflake’s AI Data Cloud and became an investor in the company in 2017.

The origins of Capital One Slingshot began with the need to develop internal tooling to ensure that the company realized potential business value improvements by managing costs and automating governance processes. Key goals were reducing the amount of time and resources spent on managing Snowflake warehouses and ensuring that Capital One’s central IT teams did not become a bottleneck that limited the adoption of Snowflake across the company. With the launch of Capital One Slingshot in 2022, the internal development work was made available to other enterprises. While Capital One Software has focused specifically on Snowflake AI Data Cloud environments, the company announced in June 2024 that it intends to adapt Slingshot to the Databricks’ Data Intelligence Platform to address cost management.

Cloud data warehouse services remove the requirement for upfront hardware and software licensing costs and, in theory, make it far simpler for enterprises to scale data warehouses up and down to maximize the efficient use of resources in response to usage requirements. I assert that through 2026, 8 in 10 enterprises will migrate on-premises analytics and data workloads to cloud environments, shifting focus to improving innovation and efficiency rather than maintaining existing systems.

Efficient usage of cloud services is easier said than done, however. Failure to make the most efficient use of cloud infrastructure can result in unexpected costs. Data engineers require a thorough understanding of usage requirements, the ability to accurately predict future needs, the time and expertise to monitor data warehouse environments to ensure efficient operation and the skills to take corrective action to address any inefficiencies. Even if data engineers have all these attributes and are initially successful, operational complexity is likely to grow as more workloads and users are added to the cloud data warehouse environment, resulting in operational overheads that may not have been initially identified and budgeted for.

There is an argument that while using cloud data warehouses can reduce upfront capital expenditure on data warehouse hardware and software, it can also increase ongoing operational expenditure on data engineering staff required to monitor and optimize cloud data warehouse deployments and ensure they are running efficiently. These were amongst the concerns that prompted the development of Capital One Slingshot. The product provides data operations and financial governance capabilities, including monitoring Snowflake costs, performance and usage, with granular reporting based on business domain, project or individual user, and alerting to notify database administrators about usage or cost anomalies. Capital One Slingshot also provides data warehouse sizing recommendations to meet cost and performance requirements, identifies inefficient queries and remediation suggestions and sets schedules for automatic provisioning of warehouses.

These capabilities are all available for Snowflake’s AI Data Cloud today and will soon be available in private preview for Databricks’ Data Intelligence Platform. Capital One Slingshot for Databricks will take advantage of Databricks’ system tables functionality to provide insight into the cost, performance and usage of Databricks’ Data Intelligence Platform. Enterprises using both Snowflake and Databricks will be able to monitor cost and performance information from both platforms in a single dashboard, providing the opportunity to compare use of the two platforms.

The move to expand its addressable market is a wise one for Capital One Software. While it has a solid partnership with Snowflake, the data platform vendor is adding capabilities for observability and financial operations. Databricks also has some capabilities for cost management, so the breadth and depth of its functionality, combined with the ability to monitor cost and performance information from Snowflake and Databricks, will likely be a key differentiator for Capital One Slingshot going forward. I recommend that enterprises using Snowflake and concerned about cost containment and performance efficiency should evaluate the potential benefits of using Capital One Slingshot. Enterprises using both Snowflake and Databricks should keep a watchful eye on the company’s plans.

Regards,

Matt Aslett