Organizations are dealing with ever-growing data volumes, diverse sources, and complex analytics needs. While platforms like Databricks Data Intelligence Platform provide an exceptional foundation for scalable, unified storage and powerful data processing, the next challenge is ensuring that this data is accessible and consistently defined across an entire organization. That’s where Cube’s universal semantic layer comes in.

[Databricks Data Intelligence Platform](https://www.databricks.com/ "Visit databricks.com") is a next-generation architecture that combines the best features of data lakes and data warehouses into a single platform. It provides a unified data storage layer for structured, semi-structured, and unstructured data. With Databricks, companies can store vast amounts of data and process it using high-performance data engineering, data science, and machine learning capabilities. It supports a variety of workloads—from real-time streaming to advanced analytics—while ensuring governance and data quality.

On the other hand, Cube offers a universal semantic layer that brings consistency, governance, and efficiency to how data is queried and analyzed. It acts as a bridge between raw data and end-user tools like AI, BI, spreadsheets, and embedded analytics. Cube abstracts complex data into easy-to-understand metrics and definitions, so business users can trust and easily explore data without needing to worry about inconsistent definitions.

## A Perfect Pair: Combining the Power of Databricks  with Cube’s Universal Semantic Layer
### Unify Your Data Architecture, Data Models, and Metrics
Databricks Data Intelligence Platform allows companies to store vast amounts of data from multiple sources, but accessing and interpreting this data can be complex, especially when involving multiple teams across an organization. Cube simplifies this complexity by creating a centralized semantic layer that defines metrics, dimensions, and KPIs in a consistent way. By integrating Cube with Databricks, all business units, whether marketing, sales, or finance, access the same governed data definitions, regardless of the tool they’re using.

### Improve Collaboration between Data and Business Teams
With Databricks Data Intelligence Platform, data engineering teams can efficiently manage data pipelines, ensuring data is cleansed, transformed, and stored. However, there’s often a gap when business teams try to make sense of this data for decision-making. Cube’s universal semantic layer bridges this gap, providing a user-friendly interface that democratizes data access. Analysts and business users can self-serve their analytics needs using BI platforms, such as Tableau, Looker, or Power BI, and spreadsheets, such as Microsoft Excel or Google Sheets, while relying on the consistent models and metrics powered by Cube. Connections are made via Cube's [native APIs](https://cube.dev/docs/product/apis-integrations), eliminating the need for direct database connections or data movement.

![](https://ucarecdn.com/1bf93e87-aa87-4715-8aee-4abbcfb48af3/-/quality/smart/-/format/auto/)

### Gain Real-Time Insights at Scale
Databricks is known for its real-time processing capabilities, enabling businesses to harness data streams for up-to-the-minute insights. But real-time data is only useful if it’s delivered with context and meaning. Cube delivers consistent metrics at scale, allowing for real-time analytics across a variety of technology and tools. Having Cube integrated with Databricks Data Intelligence Platform, organizations can leverage both real-time data processing and consistent data, ensuring that decision-makers are always acting on the latest, most accurate numbers.

### Achieve Faster Time to Insight and Lower Costs
The combination of Databricks’ scalable architecture and Cube’s ability to deliver simplified, governed data definitions means organizations can reduce time spent on data preparation and ensure faster delivery of insights. Cube’s caching and pre-aggregation features optimize query performance, reducing the computational burden on Databricks Data Intelligence Platform, lowering infrastructure costs, and improving the user experience.

### Establish Governance and Compliance Without Compromise
As organizations scale, maintaining data governance becomes critical. Databricks Data Intelligence Platform offers a strong foundation for data governance, security, and compliance. However, as data is exposed to different teams and tools, maintaining consistency in how this data is used and reported becomes a challenge. Cube’s universal semantic layer ensures that governance policies are enforced uniformly across all downstream analytics tools, eliminating the risk of data misuse and ensuring compliance with industry regulations.

## Together, Driving the Future of Data Analytics
By combining the power of Databricks Data Intelligence and Cube’s universal semantic layer, organizations can unlock the full potential of their data. Databricks ensures that data is stored, processed, and analyzed at scale, while Cube ensures that the data is consistently and easily accessible for the entire business.

With this “better together” solution, teams can:
- Break down data silos, enabling unified, democratized access to critical insights.
- Accelerate time to value by reducing the complexity of preparing and serving data.
- Ensure governance, security, and compliance across the entire data lifecycle, from ingestion to insight.

Whether you're building machine learning models, powering real-time dashboards, or generating self-service reports, the combination of Databricks and Cube helps you achieve faster, more reliable, and more scalable data insights—for everyone. [Contact sales](https://cube.dev/contact) to learn more about how Cube and Databricks work together.

Michael Hetrick

Databricks

How Cube's Universal Semantic Layer & Databricks Work Together

I am thrilled to announce that we have completed $25 million in funding. As part of this round, Nnamdi Okike, co-founder and Managing Partner at 645 Ventures joined us as a new board observer for Cube's Board of Directors. I am honored that Nnamdi and <a href="https://645ventures.com/" target="_blank" rel="noopener">645 Ventures</a> have doubled down on their investment and belief in what we can achieve at Cube. 

<a href="https://www.databricks.com/" target="_blank" rel="noopener">Databricks</a>, our strategic partner, joined this funding round. The Databricks team shares our vision that the semantic layer is a necessary component of the AI-enabled future of the data stack. With this new investment, we are deepening our relationship with Databricks and reaffirming our commitment to our roadmap to support an AI-enabled data intelligence future.  

> Andrew Ferguson, VP, Databricks Ventures, said it best “The semantic layer is critical to data intelligence and provides necessary context for AI agents to analyze data with higher accuracy.” 

## Solving data problems with the universal semantic layer

In 2019, Pavel and I started Cube to create a single place to manage data models, security, and caching. As we talked to more people and worked with customers, the problem that needed to be solved became clearer. Data models are spread across too many BI tools, and data engineers need an [open and universal semantic layer](https://cube.dev/use-cases/semantic-layer) to deliver trusted data. The data community needs to create a single source of truth that can feed metrics and defined data to any data app, BI tool or AI agent.

## How we got here
We launched the first version of open source Cube on HackerNews in 2019. Cube Core grew rapidly in the first several months, crossing 10,000 stars on GitHub and 1,000s of data engineers and developers joining the community almost overnight. Today, Cube’s open source community has grown to over 10,000 developers in the Cube Slack community, and it has 17,000 stars on GitHub. In 2021, the team built and launched the commercial version, [Cube Cloud](https://cubecloud.dev/auth), and signed up customers online or through a small sales team. Today, Cube Cloud is experiencing 3x growth year over year in both customers and bookings.

## What today's semantic layer looks like
Today, Cube Cloud is a universal semantic layer that is an independent yet interoperable part of the data stack between your data sources and data apps. And being universal is critical: Cube is vendor- and technology-agnostic. Cube Cloud connects to every cloud data warehouse on one side and to every data consumption tool on the other side – whether it is BI tools, Excel, embedded analytics, or AI agents –  using the same semantics and underlying data. This leads to consistent and trusted insights, ultimately driving informed business decisions. 

We are committed to being technology agnostic and helping our customers avoid a vendor lock-in. Implementing a universal semantic layer means you can connect to any cloud data source and any visualization tool, and are never locked into one technology. In the rapidly evolving modern data stack, flexibility must be a top priority because you need the ability to add, remove, or change solutions according to your evolving requirements. When business definitions are institutionalized in a universal semantic layer, migrations to new platforms are as simple as pointing to the new endpoint.

We keep building Cube Cloud with these critical core tenants:

- **Cube is code-first** and believes in empowering the data engineer by applying software engineering best practices and processes to data management, following CI/CD software practices, creating isolated environments, and delivering easy version control.
- **Cube is interoperable** so you can model once, deliver anywhere with a robust set of deployment options, data connectivity, and native APIs to any data endpoint.
- **Cube is fast** and optimizes data workloads with a combination of in-memory cache and pre-aggregations to enable interactive analytics with the flexibility of live connections to the cloud data warehouse.

## What's next
With new funding, we are doubling down on our roadmap in our core areas: data modeling, interoperability, and performance. We are also investing in supporting an AI-enabled data intelligence future:
- **Data modeling**. We’re expanding our data modeling capabilities to support more complicated calculated metrics and introducing low-code tools to empower more data professionals to build semantic models in Cube.
- **Interoperability**. We already support dozens of BI and data consumption tools. Earlier this year, we introduced support for the <a href="https://www.prnewswire.com/news-releases/cube-adds-support-for-the-microsoft-data-and-analytics-ecosystem-to-deliver-a-truly-universal-semantic-layer-to-enterprises-302080313.html" target="_blank" rel="noopener">Microsoft suite</a> of products. Integrations and interoperability remain our big priority, and more integrations are coming. This includes BIs, spreadsheet applications, and other data consumption tools. 
- **Performance**. We believe users shouldn’t need to choose between “extracts” and “live queries.” With our intelligent caching, we can provide the best of both worlds: interactive analytics with live drill-down capabilities to the cloud data warehouse.
- **AI-enabled data intelligence**. The semantic layer provides critical context for AI agents to analyze data with higher accuracy. We’ve recently released [AI API to enable AI agents](https://cube.dev/blog/introducing-the-ai-api-and-chart-prototyping-in-cube-cloud) to access data in the cloud data warehouses. For that, we've built a robust retrieval augmented generation (RAG) architecture on top of Cube’s semantic layer to provide the most relevant context about the data to LLMs. Looking into the future, there are a lot of opportunities in applying Gen AI not only to data consumption but to the development and maintaince of the semantic layer itself. 

Be on the lookout for what’s to come as we add more functionality with every release. I can’t wait to see what you build on top of Cube!

Come talk to us and we will [help you get started](https://cube.dev/contact).


Artyom Keydunov

All Things Cube

With strategic investment from Databricks

Cube raises $25 million

Working with Databricks and Cube is a great way to ensure your data engineering team can build and maintain a [unified semantic layer](https://cube.dev/use-cases/semantic-layer) for your organization. At a higher level, it is a better means to make cloud data accessible and consistent to every data consumer in and outside of your company. 

By combining Databricks’ unified analytics and data store with Cube’s data modeling, access control, caching, and APIs, it’s now possible to deliver consistent metrics securely to business intelligence tools, embedded analytics use cases, and AI agents.

## Why a Semantic Layer?

A semantic layer functions as a middleware component between all of your data sources, including cloud-based data lakehouses, such as Databricks, and downstream applications. Its primary role is to abstract complicated physical data models and present them in a more user-friendly manner with improved field labeling, field descriptions, and dynamically calculated metrics. It serves as a contextual bridge between the data sources and the diverse analytical tools businesses employ for data manipulation, slicing, dicing, and analysis.

![](https://ucarecdn.com/5bcadadb-60ee-4698-bc6e-1f1e2ad8aa71/)

## Why Cube and Databricks?

**Unify your data stack**. Databricks established itself as a high-performance Lakehouse for a **modern data stack**, unifying all data types. Combined with Cube, you will have consistently modeled metrics and performant queries for all your downstream data consumers. 

**Modern data stack** follows several foundational principles and architecture designs: 
* Using a Lakehouse like Databricks as a central data platform
* ELT vs. [ETL](https://www.databricks.com/glossary/extract-transform-load)
* Applying software engineering best practices to data management 

The Cube platform embodies software engineering best practices, empowering data engineers to [handle data models through code](https://cube.dev/docs/product/getting-started/databricks/create-data-model). This capability unlocks the potential to harness version control systems for secure data model evolution management, fostering collaborative efforts and streamlined code reviews. Additionally, it facilitates the integration of a semantic layer into the CI/CD pipeline and the creation of distinct development, staging, and production environments.

**Harness the capabilities and scalability offered by Databricks with Cube.** Cube seamlessly integrates with Databricks, enabling direct queries without the need for data extraction. This ensures that your data remains within the Databricks environment. Cube optimizes performance by leveraging the computing power of your Databricks instance, fully maximizing the scalability of your cloud data lakehouse. Additionally, you have the option to create [pre-aggregations](https://cube.dev/docs/product/caching/getting-started-pre-aggregations) within Cube, reducing the need for small, repetitive queries on your Databricks instance and allowing for background aggregate updates instead. Our aggregate-aware pre-aggregations are fundamentally different from common table or view materializing strategies often employed for performance reasons. With Cube pre-aggregations, users do not need to be aware of objects materialized for performance reasons; instead, they simply issue their queries, and Cube automatically chooses the optimal way to answer, from cache, from a pre-aggregate, or the source.

**Use your Databricks Lakehouse data in any application.** Cube supports your current and future use cases with first-class application programming interfaces (APIs). In addition to our SQL API, Cube also speaks REST and GraphQL, which are often preferred by application developers who are working with common front-end frameworks to develop embedded analytic applications. With Cube, you can take your investment in your Databricks ecosystem and build anything you can dream of.

## A step-by-step guide to building semantic layer with Cube and Databricks

In the below section, we’ll cover a quickstart step-by-step tutorial on how to start with Cube and Databricks. Remember: Cube Cloud is free to start (there is a sign-up button, up and to the right^). No credit card or calls with salespeople are required! :) 

Find a detailed getting started with [Cube Cloud and Databricks](https://cube.dev/docs/product/getting-started/databricks) guide in Cube documentation. 

### Create Databricks and Cube accounts
First, create Cube Cloud and Databricks accounts.

If you don’t have a dataset that you would like to use, you can follow [this guide](https://cube.dev/docs/product/getting-started/databricks/load-data) to create a test dataset in your Databricks instance.

### Connect Databricks to Cube

Please follow these steps to create your first deployment in Cube Cloud and connect it to your Databricks instance.

1. After you [sign in to your Cube Cloud account](https://cubecloud.dev/auth), click “Create Deployment”.

2. Give the deployment a name, select the cloud provider and region of your choice, and click “Next”.

3. Next, click Create to create a new project from scratch.

4. Select Databricks from the list of supported data sources and enter your Databricks credentials to connect to it. Get a [free Databricks account](https://www.databricks.com/try-databricks?utm=cube#account)

5. Once Databricks is connected, Cube can generate a basic data model from your data warehouse schema, which helps you get started with data modeling faster. We'll inspect these generated files in the next section and start changing them.

### Create your data model
Cube follows a dataset-oriented data modeling approach, which is inspired by and expands upon dimensional modeling. 

When building a [data mode](https://cube.dev/docs/product/getting-started/databricks/create-data-model) in Cube, you work with two dataset-centric objects: cubes and views. Cubes usually represent business entities such as customers, line items, and orders. In cubes, you define all the calculations within the measures and dimensions of these entities. 

To begin building your data model, click on “Enter Development Mode” in Cube Cloud. This will take you to your personal developer space.

Once you are in developer mode, navigate to the “Data Model” and click on any file located in the left sidebar to open it.

You can follow [this detailed guide](https://cube.dev/docs/product/getting-started/databricks/create-data-model) on how to make changes to your data model and build new measures and dimensions.

When you are ready to test updates to your data model, you can navigate to [Cube’s Playground](https://cube.dev/docs/product/workspace/playground). The Playground is a web-based tool that allows you to query your data without connecting any tools or writing any code.

### Query your data
You can query Cube using a BI or visualization tool through the Cube SQL API. For a better experience, we recommend mapping the BI's data model to Cube's semantic layer. This can be done automatically with Semantic Layer Sync or manually.

**Semantic Layer Sync**
[Semantic Layer Sync](https://cube.dev/docs/product/getting-started/databricks/query-from-bi) will synchronize all public cubes and views with connected BI tools.
1. Create a new sync by navigating to the “Semantic Layer Sync” tab on the “BI Integrations” page and clicking “+ Create Sync”. 
2. Follow the steps in the wizard to create a sync
3. Replace the fields for user, password, and URL with your BI tool credentials, then click on “Save All”. You can now go to the “BI Integrations” page and trigger the synchronization of your newly created semantic layer.
4. After running the sync, navigate to your BI tool instance. You should see the orders_view dataset that was created in the tool. Cube automatically maps all metrics and dimensions in the BI tool to measures and dimensions in the Cube data model.

**Manual setup**
Alternatively, you can connect to Cube and create all the mappings manually. 
1. You can find the credentials to connect to Cube on the “BI Integrations” page under the “SQL API Connection” tab.
2. After connecting, create a new dataset in the BI tool and select "orders_view" as a table.
3. Now you can map the BI tool metrics and columns to Cube's measures and dimensions.

That was a high level preview. You can create both you Databricks and Cube instances for free to try it out in your own environment.

Great work!

### More resources

Recorded workshop [Getting Started with Cube on Databricks](https://cube.dev/events/unlock-data-cube-snowflake)

Detail on [connecting to Databricks](https://cube.dev/docs/product/getting-started/databricks)


Deep Dives

Getting Started with a Databricks Semantic Layer

Today, we’re announcing our official integration with [Databricks][1]. Here’s how it works—and what it means.

## What is Databricks?

Databricks is a unified, open, web-based platform for working with Spark. Its multicloud infrastructure provides automated cluster management and IPython-style notebooks to unify data warehousing, AI, and analytics in a single platform. 

The Databricks [Data Lakehouse][2] brings the best of both worlds: the reliability, governance, and performance of data warehouses with the openness, flexibility, and ML capabilities of data lakes. The Data Lakehouse efficiently handles all data types while applying a consistent security and governance approach. 

Unifying data in a single platform fulfills the needs of all your data teams—engineering, analytics, BI, data science, and ML—and beautifully simplifies your modern data stack. 

## What is Cube?

[Cube][3]’s universal semantic layer makes lakehouse data consistent, performant, and accessible to every downstream application. We do this with [data modeling that defines your metrics up-stack][4]. 

Cube’s advanced [pre-aggregation][5] capabilities powerfully condense frequently-used queries to optimize response times. Cube then provides this data to every application with GraphQL, REST, and SQL APIs so that every tool—and every data consumer—can work with the same data.

![cube-schema][30]

## The Cube and Databricks Integration

By combining Databricks’ unified analytics and data store with Cube’s data modeling, access control, caching, and APIs, it’s now possible to build a new dashboard or embed analytics features into customer-facing apps in minutes—not weeks. 

Select Cube and Databricks users already have been enjoying this integration in production for use cases like creating dashboards for internal analytics. Here’s one example: by consuming operational data with Kafka, transforming it into insights with Databricks, then using Cube as an API and caching layer to power visualization layers, users have been able to find actionable insights and identify opportunities to optimize their operations.

It’s easy to connect a workspace to Cube using Databricks’ SQL endpoint. Here’s how.

For starters, we recommend using Cube with the [Databricks SQL Endpoint][6]. Follow [Databricks’ docs to create a SQL Endpoint][7]. 

### Creating a new SQL endpoint

 - Log in to your Databricks Workspace account.
 - Navigate to the **SQL Console** on the left hand bar and select **SQL** from the drop-down menu.

![select-sql-tab-in-databricks][17]

 - Click __API Endpoints__, and then on the top right, select __+New SQL Endpoint.__

![click-API-endpoint][18]
![create-new-SQL-endpoint][19]

 - Next, name your endpoint and choose a __Cluster Size__. Make sure to set the __Auto Stop__ to at least 60-minutes of no activity, then click __Create__.

![name-new-endpoint][20]

After you’ve configured data access on the SQL Endpoint, it’s time to get your credentials for the SQL Endpoint. 

 - In the __Databricks SQL Console__, go to __Endpoints__ and click on the endpoint you created.

![go-to-endpoints-in-console][21]
![click-on-endpoint-created][22]

 - Navigate to __Connection Details__ and copy the HTTP Path, Port, and Server Hostname:

![connection-details][23]

Once you have your credentials, [generate a Personal Token][8]. 

 - Navigate to the __Admin Console__, click on __Settings__, and select __User Settings__.

![user-settings][24]

 - Toggle over __Personal Access Tokens__ and click on __+Generate New Token__.

![personal-access-token][25]

 - Name your token and set the expiration to __“Lifetime”__, then click __Generate__.

![name-your-token][26]

 - Copy the generated JDBC URL.
 - Attach your newly-generated personal token to the JDBC URL you copied in step 6, e.g. `jdbc:spark://dbc-xxxxxxxx-3957.cloud.databricks.com:443:443/default;transportMode=http;ssl=1;AuthMech=3;httpPath=/sql/1.0/endpoints/0b27a1dfc18f82ae;UID=token;PWD=PERSON-TOKEN`
 - Finally, select the database you’d like to query with Cube. In our example, we’ll use “default”.

![select-database][27]

### Connecting Databricks to Cube

 - Sign into [Cube Cloud][9] and use the wizard to create a new deployment. Select Databricks from the database connection list.

![sign-into-Cube-Cloud][28]

 - Enter your Databricks JDBC URL and personal token as well as your database name to connect.

![databricks-jdbc-url][29]

 - If needed, copy Cube Cloud’s IP addresses and add them to your Databricks allow-list. ([Databricks’ documentation for IP access lists is here][10].)

That’s it! Now you are ready to use Cube on top of your Databricks to model your data, manage security rules, apply caching, and expose your data definitions to any downstream application via Cube’s REST, GraphQL, or SQL APIs. 

You can check out [more examples of integrations with visualization tools on our website][11].

## Try the Cube and Databricks Integration for yourself

To follow the tutorial above and experience everything this integration offers, check out our [docs][12]—and, if you don’t already have one, sign up for [a free Cube Cloud account][13].

## Tell us what you think

We’d love to hear from you, learn what you’re building, and strategize how to help. Find us via Cube’s [Slack][14] and [GitHub][15] communities—or [schedule a time][16] to chat 1:1.

[1]: https://databricks.com/
[2]: https://databricks.com/product/data-lakehouse
[3]: https://cube.dev/
[4]: https://cube.dev/blog/headless-bi
[5]: https://cube.dev/docs/schema/reference/pre-aggregations
[6]: https://docs.databricks.com/sql/admin/sql-endpoints.html
[7]: https://docs.databricks.com/sql/admin/sql-endpoints.html#create-a-sql-endpoint
[8]: https://docs.databricks.com/administration-guide/access-control/tokens.html#enable-or-disable-token-based-authentication-for-the-workspace
[9]: https://cube.dev/cloud
[10]: https://docs.databricks.com/security/network/ip-access-list.html#add-an-ip-access-list
[11]: https://cube.dev/examples
[12]: https://cube.dev/docs/config/databases/databricks/jdbc#databricks-jdbc
[13]: https://cube.dev/cloud/
[14]: https://slack.cube.dev/
[15]: https://github.com/cube-js/cube.js
[16]: https://cube.dev/contact
[17]: https://cubedev-blog-images.s3.us-east-2.amazonaws.com/39cfa0b3-c6b9-44dd-80a2-41bae6dea1a3.png
[18]: https://cubedev-blog-images.s3.us-east-2.amazonaws.com/38cfe617-ae91-4a85-a610-12eaba6c9dfe.png
[19]: https://cubedev-blog-images.s3.us-east-2.amazonaws.com/02a29df7-26dd-41ba-a442-7fe46cf58246.png
[20]: https://cubedev-blog-images.s3.us-east-2.amazonaws.com/f64ca517-e89e-437b-92e2-1c48752b8ca4.png
[21]: https://cubedev-blog-images.s3.us-east-2.amazonaws.com/5b5644d7-d5e1-4bff-87cd-abf40bbe459d.png
[22]: https://cubedev-blog-images.s3.us-east-2.amazonaws.com/f8bd03ad-4edf-4452-a866-9d3c3140dc42.png
[23]: https://cubedev-blog-images.s3.us-east-2.amazonaws.com/6acf6cd1-0423-45f9-8bb2-884e3115e689.png
[24]: https://cubedev-blog-images.s3.us-east-2.amazonaws.com/8fd9c163-ebae-4b1e-b846-6178df628781.png
[25]: https://cubedev-blog-images.s3.us-east-2.amazonaws.com/7877ce4f-0e2d-49c1-97f9-56769fef46c7.png
[26]: https://cubedev-blog-images.s3.us-east-2.amazonaws.com/9dd552af-3dd4-40b4-9b9a-b3fd6c2bc8d2.png
[27]: https://cubedev-blog-images.s3.us-east-2.amazonaws.com/580253c6-1077-4291-a4a3-13303d383cda.png
[28]: https://cubedev-blog-images.s3.us-east-2.amazonaws.com/cd5ae87c-f9d0-432b-9a98-b378580d8d1d.png
[29]: https://cubedev-blog-images.s3.us-east-2.amazonaws.com/7f3f30a5-33a4-4073-8669-08bc7082c651.png
[30]: https://cubedev-blog-images.s3.us-east-2.amazonaws.com/e758755f-d633-45da-8ae3-92bc510cc9a1.png
[31]:https://databricks.com/dataaisummit/?itm_data=mainnav-promo-dais2022
[32]:https://cube.dev/events/data-ai-summit/

Tamar Barlev

News

Announcing the Cube and Databricks integration

About 6 months ago, we witnessed what Andy Pavlo [wittily characterised](https://twitter.com/andy_pavlo/status/1459251061085118465) as an "old school database benchmark gang war". The timeline of events goes like this:

* [Databricks](https://databricks.com), a data lakehouse company founded by the creators of Apache Spark, published a [blog post](https://databricks.com/blog/2021/11/02/databricks-sets-official-data-warehousing-performance-record.html) claiming that it set a new data warehousing performance record in 100 TB [TPC-DS benchmark](https://www.tpc.org/tpcds/). It was also mentioned that Databricks was 2.7x faster and 12x better in terms of price performance compared to Snowflake.
* [Snowflake](https://snowflake.com), a data warehousing company founded by ex-Oracle and ex-VectorWise experts, responded with a [blog post](https://www.snowflake.com/blog/industry-benchmarks-and-competing-with-integrity/) that critically reviewed Databricks' findings, reported different results for the same benchmark, and claimed comparable price/performance to Databricks.
* After two days, Databricks followed up with another [blog post](https://databricks.com/blog/2021/11/15/snowflake-claims-similar-price-performance-to-databricks-but-not-so-fast.html) that confirmed that they stand by the initially reported results.

While this back-and-forth between respected data vendors was informative and amusing, its collateral damage was even more substantial. Databricks shared that they have [eliminated the anti-competitive *DeWitt clause*](https://databricks.com/blog/2021/11/08/eliminating-the-dewitt-clause-for-database-benchmarking.html) from their service terms. Snowflake followed up with a [similar update](https://www.snowflake.com/blog/industry-benchmarks-and-competing-with-integrity/) to their acceptable use policy.

But what is the *DeWitt clause*?

More importantly, why would you need to know if a DeWitt clause is included in terms of service in case you'd like to do and publish a performance benchmark? Let's explore.

## Backstory of the DeWitt Clause

In 1982, at the very inception of relational databases, [David DeWitt](https://www.linkedin.com/in/david-dewitt-30189b50/), a researcher at the Department of Computer Sciences at the University of Wisconsin-Madison, was working on measuring database performance. His team wrote the Wisconsin Benchmark, tested a number of databases, and [published the results](https://pages.cs.wisc.edu/~dewitt/includes/benchmarking/vldb83.pdf).

Apparently, some folks were displeased with the results. [Michael Stonebraker](https://en.wikipedia.org/wiki/Michael_Stonebraker), a researcher at the UC Berkeley and co-creator of Ingres, called DeWitt and expressed how upset he was. However, they remained friends anyway as DeWitt recounts: in his [talk on YouTube](https://www.youtube.com/watch?v=-TIUGC4X2q8&t=420s):

<iframe width="800" height="500" src="https://www.youtube.com/embed/-TIUGC4X2q8?start=420" title="YouTube video player" frameborder="0" allow="accelerometer; autoplay; clipboard-write; encrypted-media; gyroscope; picture-in-picture" allowfullscreen></iframe>

Oracle also didn’t perform very well and reacted hastily. [According to DeWitt](https://www.eweek.com/development/db-test-pioneer-makes-history/), Oracle CEO Larry Ellison was so displeased that he called the department chair and insisted, "You have to fire this guy.” Oracle also inserted a clause in their terms of use that boiled down to the fact that one can’t publish benchmarks without getting an explicit approval from Oracle.

## DeWitt Clause vs. DeWitt Embrace Clause

Nowadays, the DeWitt clause is a common provision in end-user license agreements for proprietary software that prevents from publishing information about the software without an explicit approval from the vendor. Over time, it have become widespread in the software industry and its implications are drastic:

* People avoid publishing benchmarks about software products with the DeWitt clause.
* Moreover, some won’t even bother doing a benchmark knowing they won’t be able to publish the results.

There are debates whether DeWitt clauses are [moral or legal](https://www.dwheeler.com/essays/dewitt-clause.html) because of the implications above as well as the overall corrosive effect they have on the industry. Once one supplier adds a DeWitt clause, the others can feel that they are at a disadvantage without one. We're lucky it can also work the other way round, like in the Databricks and Snowflake case. However, open source software (OSS) licenses don’t and can't have DeWitt clauses, meaning that the OSS can be legally critiqued, but not their proprietary competitors.

At the same time, some vendors do quite the opposite to the inclusion of a DeWitt clause. Instead, they change their license to the DeWitt Embrace Clause meaning that if you disclose benchmark results for them, you automatically allow them to do the same with you. In a way, DeWitt Embrace Clause is like copyleft for the DeWitt Clause. 

## Can You Benchmark %DATABASE%?

Now, do you wonder whether some vendor has or hasn't a DeWitt clause?

Shortly after the Databricks and Snowflake debate, [Mark Callaghan](https://twitter.com/MarkCallaghanDB), an ex-MongoDB and ex-Rockset database expert, checked if some database and public cloud vendors have DeWitt clauses in their terms of use.

In this blog post, I try to extend [his research](http://smalldatum.blogspot.com/2021/11/dewitt-clause-vs-public-cloud.html) and maintain an ever-green list of database and similar vendors with regard to the DeWitt clause. (If you spot any inaccuracy, which is totally possible, please get in touch via [igor@cube.dev](mailto:igor@cube.dev?subject=DeWitt), I'll be happy to update the post.)

*Last updated: June 2, 2022.*

### 😇 Open-source vendors (without the DeWitt clause)

Open source licenses usually grant users permission to [use open source software for any purpose](https://opensource.org/docs/osd). So, by definition, open source software can't contain DeWitt clauses (because if it did, it wouldn't be open source).

Yes, you can benchmark any of these tools:

* Apache [Drill](https://github.com/apache/drill), [Druid](https://github.com/apache/druid/), [Flink](https://github.com/apache/flink), [Hive](https://github.com/apache/hive), [Kafka](https://kafka.apache.org), [Spark](https://github.com/apache/spark)
* [appwrite](https://github.com/appwrite/appwrite)
* [ClickHouse](https://github.com/ClickHouse/ClickHouse) (they even maintain a [list of benchmarks](https://github.com/ClickHouse/ClickHouse/issues/22398) on GitHub)
* [CockroachDB](https://github.com/cockroachdb/cockroach) (licensed under BSL, not [OSI-approved](https://opensource.org/licenses/alphabetical))
* [Dremio](https://github.com/dremio/dremio-oss)
* [Elasticsearch](https://github.com/elastic/elasticsearch) (licensed under SSPL, not OSI-approved)
* [ksqlDB](https://ksqldb.io)
* [Materialize](https://github.com/MaterializeInc/materialize) (licensed under BSL, not OSI-approved)
* [MongoDB](https://github.com/mongodb/mongo) (licensed under SSPL, not OSI-approved)
* [PostgreSQL](https://www.postgresql.org)
* [Presto](https://github.com/prestodb/presto)
* [Redpanda](https://github.com/redpanda-data/redpanda) (licensed under BSL, not OSI-approved)
* [QuestDB](https://github.com/questdb/questdb)
* [TimescaleDB](https://github.com/timescale/timescaledb)
* [Trino](https://github.com/trinodb/trino)

Also, some open-source vendors collaboratively maintain benchmarking suites such as [Time Series Benchmark Suite](https://github.com/timescale/tsbs) to help choose the best tools for particular workloads.

### 😇 Vendors without the DeWitt clause

Cloud vendors can restrict you from benchmarking their services but sometimes they choose not to have the DeWitt clause.

Yes, you can benchmark any of these services:

* [Ahana Cloud](https://ahana.io/terms-of-service/)
* [Altinity Cloud](https://altinity.com/legal/)
* [appwrite Cloud](https://appwrite.io) (not launched to GA yet)
* [ClickHouse Cloud](https://clickhouse.com/cloud/) (not launched to GA yet)
* [Confluent Cloud](https://assets.confluent.io/m/103033bdcb9b3fcd/original/20220201-Confluent_Cloud_Services_Agreement.pdf)
* [Firebolt](https://d7umqicpi7263.cloudfront.net/eula/product/28a79f78-8008-4c02-a971-54ef6d8ed904/898eb1bc-7c2a-4829-b479-67f5a18a51ff.pdf)
* [Materialize Cloud](https://materialize.com/terms-and-conditions/)
* [MongoDB Cloud](https://www.mongodb.com/cloud-terms-and-conditions)
* [QuestDB Cloud](https://questdb.io/terms/) (not launched to GA yet)
* [Redpanda Cloud](https://redpanda.com/cloud/)
* [Snowflake](https://www.snowflake.com/legal/acceptable-use-policy/)
* [Starburst Galaxy](https://www.starburst.io/terms/galaxy-terms-of-service/)
* [Supabase](https://supabase.com/docs/company/terms)
* [Timescale Cloud](https://www.timescale.com/legal/timescale-cloud-terms-of-service)

However, many of these cloud vendors disallow *abusive benchmarking* in their acceptable use policies. So be careful and please don't get banned.

### 🙃 Vendors with the DeWitt Embrace clause

Some cloud vendors permit you to benchmark their service but require reciprocity: you must make the benchmark reproducible and allow benchmarking of your own service or tool in response.

Yes, you can benchmark any of these services and probably get away with it:

* [Amazon Web Services](https://aws.amazon.com/service-terms/), including [Athena](https://aws.amazon.com/athena/), [Aurora](https://aws.amazon.com/rds/aurora/), and [Redshift](https://aws.amazon.com/redshift/). *You may perform benchmarks or comparative tests or evaluations (each, a “Benchmark”) of the Services. If you perform or disclose, or direct or permit any third party to perform or disclose, any Benchmark of any of the Services, you (i) will include in any disclosure, and will disclose to us, all information necessary to replicate such Benchmark, and (ii) agree that we may perform and disclose the results of Benchmarks of your products or services, irrespective of any restrictions on Benchmarks in the terms governing your products or services.*
* [CockroachDB Serverless](https://www.cockroachlabs.com/acceptable-use-policy/). *You can only disclose benchmarking tests if you (i) provide us the necessary information to replicate the tests, (ii) disclose the methodology for the benchmarking tests along with the results, and (iii) allow us to run our own benchmarking tests against your products and services at our discretion.*
* [Databricks](https://databricks.com/blog/2021/11/08/eliminating-the-dewitt-clause-for-database-benchmarking.html). *You may perform benchmarks or comparative tests or evaluations (each, a “Benchmark”) of the Platform Services and may disclose the results of the Benchmark other than for Beta Services. If you perform or disclose, or direct or permit any third party to perform or disclose, any Benchmark of any of the Platform Services, you (i) will include in any disclosure, and will disclose to us, all information necessary to replicate such Benchmark, and (ii) agree that we may perform and disclose the results of Benchmarks of your products or services, irrespective of any restrictions on Benchmarks in the terms governing your products or services.*
* [SingleStore](https://www.singlestore.com/blog/eliminating-the-dewitt-clause-for-greater-transparency-in-benchmarking/). *Customer may perform industry standard benchmarks, comparative tests or evaluations (each, a "Performance Report") of the Managed Service. If Customer performs or discloses, or directs or permits any third party to perform or disclose, any Performance Report of any of the Managed Service, Customer (i) will include in any disclosure, and will disclose to SingleStore all information necessary to replicate such Performance Report and the data from the Performance Report, and (il) agree that SingleStore may perform and disclose the results of the Performance Report of Customer's products or services, irrespective of any restrictions on Performance Report.*

### 🤬 Vendors with the DeWitt clause

These vendors chose to restrict benchmarks with DeWitt clauses.

No, you can't benchmark any of these tools and get away with it:

* [Cloudera Data Platform](https://www.cloudera.com/legal/terms-and-conditions/cloudera-online-services-terms.html). *Restrictions. Customer may not: [...] access or use the Services for purposes of monitoring availability, performance or functionality of the Services, or for any benchmarking or competitive purposes.*
* [Decodable](https://www.decodable.co/legal/tos). *Restrictions. Customer may not, and may not cause or permit others to: [...] disclose results of any benchmark tests or performance tests of the Decodable services hereunder without Decodable’s prior written consent.*
* [Dremio Cloud](https://www.dremio.com/legal/terms-of-service/). *Restrictions. Customer shall not, and shall not cause or allow any Authorized User or third party to: [...] except with Dremio’s prior written permission, publish any performance or benchmark tests or analysis relating to Dremio Cloud.*
* [Elastic Cloud](https://www.elastic.co/agreements/global/cloud-services-monthly). *Restrictions [...] You shall not: [...] access or use any Cloud Service for purposes of monitoring its availability, performance or functionality, or for any other benchmarking or competitive purposes, including, without limitation, for the purpose of designing and/or developing any competitive services.*
* [Google Cloud Platform](https://cloud.google.com/terms/service-terms), including [BigQuery](https://cloud.google.com/bigquery) & [Firebase](https://firebase.google.com). *Benchmarking. Customer may conduct benchmark tests of the Services (each a "Test"). Customer may only publicly disclose the results of such Tests if it (a) obtains Google's prior written consent, (b) provides Google all necessary information to replicate the Tests, and (c) allows Google to conduct benchmark tests of Customer's publicly available products or services and publicly disclose the results of such tests. Notwithstanding the foregoing, Customer may not do either of the following on behalf of a hyperscale public cloud provider without Google's prior written consent: (i) conduct (directly or through a third party) any Test of the Services or (ii) disclose the results of any such Test.*
* [InfluxDB Cloud](https://www.influxdata.com/legal/terms-of-use/). *Restrictions. Customer will not: [...] publish or provide any benchmark, comparison or performance test results.*
* [MariaDB SkySQL](https://legal.mariadb.com/agreements/skysql/SkySQL_Terms_of_Service_V2_2020-12-16.pdf) and [MariaDB Xpand](https://legal.mariadb.com/agreements/xpand/xpand-license-terms-v2-2021-02-24.pdf). *Benchmarking. Customer may perform benchmarks or comparative tests or evaluations (each, a "Benchmark") of the SkySQL Services. However, Customer must obtain MariaDB's prior written approval to disclose to a third party the results of any Benchmark of the SkySQL Services.*
* [Microsoft SQL Server](https://www.microsoft.com/en-us/sql-server/). *BENCHMARK TESTING. You must obtain Microsoft’s prior written approval to disclose to a third party the results of any benchmark test of the software.*
* [Oracle](https://www.oracle.com/us/corporate/contracts/lic-online-toma-ph-de-eng-2817945.pdf). *RESTRICTIONS. [...] You may not: [...] disclose results of any Program benchmark tests without Oracle's prior written consent.*
* [PlanetScale](https://planetscale.com/legal/aup). *Customer will not, and will ensure that its End Users do not: [...] access any portion of the Product for the purpose of building a similar or competitive product or service, or monitor the Product for any benchmarking or competitive purpose.*

### 😶 Vendors for which the author didn't find the data

I wasn't able to identify where a few vendors stand with regard to the DeWitt clause:

* [MySQL HeatWave](https://www.mysql.com/cloud/)
* [Yellowbrick Data](https://www.yellowbrick.com/)

If you can provide any pointers, please get in touch via [igor@cube.dev](mailto:igor@cube.dev?subject=DeWitt), I'll be happy to update the post. Also, please reach out if you'd like more databases to be featured here.

## What about Cube?

[Cube](https://cube.dev) is an open-source [headless BI](https://cube.dev/blog/headless-bi) platform that [integrates with a lot of data sources](https://cube.dev/docs/config/databases): cloud data warehouses, query engines, and databases. Many of them are designed for low latency and some of them can provide high concurrency. However, sometimes they don't; you can read our [blog post](https://cube.dev/blog/data-warehouse-performance-and-how-cube-can-help) about that.

In any case, Cube will have no problem delivering your data, with sub-second latency and high concurrency, to any [data consumer](https://cube.dev/docs/config/downstream) out there: BI tools, data notebooks, front-end apps, etc. Thanks to [Cube Store](https://cube.dev/blog/introducing-cubestore), Cube's REST, GraphQL, and SQL APIs are able to respond to any request within 200-300 milliseconds and allow for 100+ concurrent requests—and that can scale.

Please explore how Cube integrates with [databases and other tools](https://cube.dev/blog/category/data) in the data space in our blog. Also, don't hesitate to try [Cube Cloud](https://cubecloud.dev/auth/signup), join our large [Slack community](http://slack.cube.dev), and give Cube a [star on GitHub](https://github.com/cube-js/cube.js) 🌟

Igor Lukanin

Performance

All database vendors who will either praise or punish you for doing a benchmark as well as an entertaining digression into the recent history of computer science

Databricks

How Cube's Universal Semantic Layer & Databricks Work Together

Cube raises $25 million

Get Cube updates to your inbox

Getting Started with a Databricks Semantic Layer

Announcing the Cube and Databricks integration

DeWitt clause, or Can you benchmark a database and get away with it

Upgrade your data stack today