Database Servers and Embedded Databases: A Comparison of Architecture and I/O Characteristics

Database Server

A database server refers to a database that runs as an independent server process and processes application requests over a network. Most RDBMS products familiar to us, such as MySQL, PostgreSQL, and Oracle DB, fall under this model. Database servers offer the following advantages.

• Centralized management: Data is managed in one place, which facilitates consistency, security, backup, and recovery.
• Resource independence: Dedicated CPU, memory, and storage are allocated, ensuring stable performance regardless of the state of the application.
• High concurrency and scalability: They are optimized for simultaneous access by many users, and horizontal scaling through clustering or replication is relatively straightforward.

Owing to these characteristics, database servers have established themselves as the standard of software infrastructure over the past several decades. However, due to technological advances and changes in the environment, the drawbacks of database servers have begun to surface in some settings, and embedded databases are receiving renewed attention as an alternative.

A New Trend: Embedded Databases

An embedded database is a database engine that, without a separate server process, is included in an application as a library and runs within the same process. SQLite, LevelDB, and RocksDB are representative examples of embedded databases. The background against which embedded databases have emerged as a strong alternative in the modern technological environment is as follows.

• The advent of SSDs

  In the earlier HDD-centered system environment, random access performance was markedly lower than sequential read and write performance. Accordingly, minimizing disk I/O and maximizing memory caching were central to performance optimization. Database servers, in response, operated large-scale caches and processed write operations in batches, thereby overcoming the physical limitations of HDDs.

  However, the advent of SSDs has completely overturned this premise. SSDs offer random I/O performance thousands of times faster than HDDs, and their latency is dramatically shorter. As a result, the conventional perception that "disk I/O is the greatest bottleneck" has disappeared, and latency arising from network access to remote DB servers has instead begun to emerge as a new bottleneck.

• MSA and data independence

  In the past, it was common for a single large monolithic application to depend on one centralized database. Although this approach was simple, as the scale of services grew, changes to the data schema became difficult, and because all services shared the same DB, coupling became excessively strong.

  In modern microservice architectures, the "Database per Service" principle is emphasized to resolve these problems. When each service owns and encapsulates its own dedicated data store, independence between services is guaranteed, and fault isolation and scaling become easier. In this context, embedded DBs become an ideal choice as service-level data stores. They are lightweight and fast, and because they can be deployed together with service code, they fit very well with MSA environments.

  Representatively, in Kubernetes environments, the practice of placing an embedded DB beside the application container through the sidecar pattern to optimize local data processing is widely used. This reduces network bottlenecks and minimizes data access latency.

• Simplification of operational environments

  Database servers require complex operational procedures such as installation, patching, backup, failure recovery, replication, and performance monitoring, and for these purposes dedicated DBAs and specialized operations personnel are necessary. Particularly in large-scale environments, this management burden is very substantial.

  Embedded DBs greatly alleviate these problems. When the database is integrated inside the application, unit testing, building, version control, and deployment are carried out together, so no separate server operation is required. Furthermore, when an application scales out, the embedded DB scales along with it, and automated management through DevOps culture and CI/CD pipelines becomes possible. Particularly in startups, small-scale services, and prototype development environments, embedded databases dramatically reduce operational complexity and significantly accelerate development and deployment.

• Maturation of high-performance systems languages and ecosystems

  In the past, it was common for databases and applications to be written in different languages. For example, high-performance database engines were primarily written in C and C++, while applications were developed in languages such as Java, Python, and PHP. Although this approach was effective for performance optimization, it had limitations including memory safety issues, complex concurrency handling, latency, and difficulties in library integration and deployment.

  However, with the rise of modern systems programming languages such as Go and Rust, these limitations have been substantially mitigated. In particular, Go is suitable not only for high-performance applications but also for writing low-level programs such as database engines, making it possible to handle databases and applications together within a single language ecosystem.

  In fact, in Golang, high-performance embedded databases such as BadgerDB and PebbleDB are actively used. This signifies that embedded databases have moved beyond being merely lightweight alternatives to database servers and have established themselves as important options with their own competitive strengths.

Conclusion

Under conditions requiring centralized management of large-scale data, complex transactions, and high concurrency, the database server remains the most powerful solution. However, for individual services in MSA environments, IoT edge devices, rapid prototyping, and projects where operational efficiency is important, embedded databases can be a more efficient and rational choice. Ultimately, what matters is selecting the optimal tool that best fits the given situation and requirements, and the development of embedded databases is providing us with broader and more diverse options.