Like any mature market, growth in the database market should be flattening; instead, it’s accelerating. Most of that growth is being gobbled up by cloud databases, but enterprises are spoiled for choice with hundreds of databases to choose from, cloud or otherwise.
Decades into its existence, the database market is hotter than ever, spawning startups with seemingly bottomless amounts of venture funding across an ever-widening array of database types, including vector, graph and time series.
Even as legacy database vendors like Oracle see growth slow, the category is booming. A big reason is cloud, but the bigger reason is simply that data keeps growing in importance to every enterprise, with diverse, unstructured data giving birth to new databases to manage it all. Oh, and of course AI has a starring role, with vector databases created to help support the trend.
What does the current database market look like?
In 2022, the global database market grew by, according to Gartner, an “unprecedented” 14.4%, hitting $91 billion. Now, in 2023, Gartner predicts the database market will clear $100 billion at a compound annual growth rate of 16.8%. According to the analyst firm, at this CAGR, the database market will surpass $203.6 billion by 2027, dramatically outpacing projected growth for the overall IT market of 4.3%.
That’s a lot of ones and zeroes being stored.
According to Ofer Bengal, CEO at Redis Labs, three big trends drive database market growth: open source, non-relational data and cloud. Of the three, cloud is the primary driver of this growing largesse, according to Gartner.
SEE: Explore the impact of cloud data systems and edge AI on data science.
In fact, cloud database spending (55%) now exceeds on-premises database spending (45%). This shift to cloud database services will only accelerate as AI, in particular, pushes enterprises to embrace the agility and flexibility that AI demands and cloud computing affords.
The database market moves to the cloud
This growth truly is unprecedented, as Gartner suggests, perhaps because the evolving nature of the database is somewhat unprecedented. Normally, product categories rise and then tend to dwindle over time, replaced by other things.
For example, Microsoft minted billions in the operating system market, but today, we don’t really care much about the OS. In addition, app servers, enterprise resource planning and enterprise content management used to be considered hot new markets. While companies still depend on these products or some variant of them, they’re not considered growth markets. Companies spend on them to maintain past investments, but they’re not considered growth drivers.
Databases, by contrast, are growth drivers. Relational databases were born in the early 1970s, and Oracle, Microsoft and IBM started up massive businesses to sell and support them. For their time, they were exceptionally innovative and paved the way for modern enterprises. Most tech markets rise on the promise of changing some corner of the world, but databases have lived up to that promise and are doing so again.
SEE: Discover the uses for different data model types.
While these legacy vendors have seen their database revenue growth slow, the market as a whole has done anything but. We’ve seen an explosion of companies expanding on taking relational databases like PostgreSQL to the cloud, including AWS, Timescale, Aiven and Neon.
Non-relational databases find an audience
There has been a profound and sustained rise in NoSQL databases like MongoDB, Amazon DynamoDB, Neo4j and Apache Cassandra. The “NoSQL” moniker isn’t perfect — after all, some of these databases have added support for SQL-style queries. But it has helped to demarcate a new breed of databases that put a premium on schema flexibility, horizontal scalability and an ability to manage the explosion in unstructured data.
Indeed, relational databases like Oracle, Microsoft SQL Server and MySQL, with the prominent exception of PostgreSQL, have declined relative to non-relational databases over the last nine years, as measured by DB-Engines (Figure A).
Figure A
That’s not to suggest that SQL/relational use is on the wane. In fact, SQL’s importance, as measured by job postings, keeps increasing. Enterprises are increasing their interest in developers who can query the databases that have been running their enterprises for years using comfortable, widely used SQL.
SEE: Download our cheat sheet on how to become a database administrator.
At the same time, enterprises are just as clearly looking for developers that can help them query new data types and sources, which often won’t involve SQL.
It’s not an either/or decision, in other words. For enterprises of any reasonable size, it’s a matter of “and.” Enterprises are simply trying to make the best use of their data and turn to the right database for the job.
Database’s new growth category: Vectors
According to Zilliz’s Frank Liu, “A vector database is a fully managed, no-frills solution for storing, indexing and searching across a massive dataset of unstructured data that leverages the power of embeddings from machine learning models.”
As such, vector databases prove useful in things like image search or searching within video, audio or other forms of unstructured data to understand the content, not the keywords associated with that content. Vectors are increasingly critical to productively building AI-driven applications. In so doing, they offer yet another reason for enterprises to spend heavily on databases.
About a year ago, most people wouldn’t have heard of vectors. Today, they’re all the rage. Zilliz, the company behind the open-source vector database Milvus, raised $60 million in 2022, to add to the $43 million raised in 2020. More recently, Pinecone, another vector database startup, raised $100 million.
My point isn’t to offer a tutorial in vector databases. Rather, it’s to show that with the continued growth of structured and especially unstructured data, the database market will continue to balloon. At the same time, we’ll see new approaches to databases crop up, like vectors. Given the ever-increasing importance of data, and of AI that depends on data, Gartner’s projections for database market growth may, if anything, be understated.
Disclosure: I work for MongoDB but the views expressed herein are mine.
