Database Servers and Embedded Databases
Database Server
A database server refers to a database that operates as an independent server process and handles application requests over a network. Most familiar RDBMS, such as MySQL, PostgreSQL, and Oracle DB, fall into this model. A database server offers the following advantages:
- Centralized Management: Data is centrally and integrally managed, facilitating consistency, security, backup, and recovery.
- Resource Independence: It is allocated its own CPU, memory, and storage, ensuring stable performance regardless of the application's state.
- High Concurrency and Scalability: It is optimized for concurrent access by numerous users, and horizontal scaling through clustering or replication is relatively straightforward.
Owing to these characteristics, database servers have been established as the standard for software infrastructure for several decades. However, due to technological advancements and environmental changes, the disadvantages of database servers have begun to emerge in certain environments, and embedded databases are re-emerging as an alternative.
New Trend: Embedded Databases
An embedded database is a database engine that is included as a library within an application and executes within the same process, without a separate server process. SQLite, LevelDB, and RocksDB are representative embedded databases. The background for these embedded databases becoming a powerful alternative in the latest technological environments is as follows:
Emergence of SSDs
In past system environments dominated by HDDs, random access performance was significantly inferior compared to sequential data read/write performance. Therefore, minimizing disk I/O and maximizing memory caching were central to performance optimization. Database servers adapted to this by operating large caches and processing write operations in batches, thereby overcoming the physical limitations of HDDs.
However, the advent of SSDs completely altered this premise. SSDs offer random I/O performance thousands of times faster than HDDs and significantly shorter latency. As a result, the conventional perception that "disk I/O is the biggest bottleneck" has dissipated, and instead, latency arising from remote DB server access via the network has begun to emerge as a new bottleneck.
MSA and Data Independence
In the past, it was common for a single monolithic application to rely on a single centralized database. While this approach was simple, as service scales grew, it became challenging to modify data schemas, and the problem of excessive coupling arose as all services shared the same database.
In modern microservices architecture, the "Database per Service" principle is emphasized to address these issues. By ensuring that each service owns and encapsulates its dedicated data store, independence between services is guaranteed, and fault isolation and scaling become easier. In this process, embedded databases become an ideal choice for service-unit data storage. They are lightweight, fast, and can be deployed with service code, making them highly compatible with MSA environments.
For instance, in Kubernetes environments, the sidecar pattern is widely used to optimize local data processing by placing an embedded DB alongside the application container. This results in reduced network bottlenecks and minimized data access latency.
Simplified Operational Environment
Database servers necessitate complex operational procedures, including installation, patching, backup, disaster recovery, replication, and performance monitoring, requiring dedicated DBAs and specialized operational personnel. Particularly in large-scale environments, this management burden is substantial.
Embedded databases significantly alleviate these issues. When a database is integrated within an application, unit testing, build, version control, and deployment occur concurrently, eliminating the need for separate server operations. Furthermore, when an application scales out, the embedded DB scales with it, and automated management through DevOps culture and CI/CD pipelines becomes feasible. Especially in startups, small-scale services, and prototype development environments, embedded databases dramatically reduce operational complexity and greatly accelerate development and deployment speeds.
Advancement of High-Performance System Languages and Ecosystems
Historically, it was common for databases and applications to be written in different languages. For example, database engines requiring high performance were primarily written in C, C++, while applications were developed in languages such as Java, Python, and PHP. While this approach was effective for performance optimization, it had limitations, including memory stability issues, complex concurrency handling, latency, and difficulties in library integration and deployment.
However, with the recent rise of modern system programming languages like Go and Rust, these limitations have been significantly mitigated. Go, in particular, is suitable for writing high-performance applications as well as low-level programs like database engines, enabling databases and applications to be handled together within a single language ecosystem.
In fact, in Golang, high-performance embedded databases such as BadgerDB and PebbleDB are actively utilized. This signifies that embedded databases are not merely lightweight alternatives to database servers but have established themselves as important choices with their own competitive advantages.
Conclusion
For conditions requiring centralized management of large-scale data, complex transactions, and high concurrency, database servers remain the most robust solution. However, for individual services in an MSA environment, IoT edge devices, rapid prototyping, and projects where operational efficiency is crucial, embedded databases can be a more efficient and rational choice. Ultimately, the key is to select the optimal tool that best fits the given situation and requirements, and the evolution of embedded databases provides us with a broader and more diverse range of options.