Stateful Databases: A Deep Dive with Cassandra & TiDB

What is a Stateful Database?

A stateful database is a type of database system that retains information about past interactions or states and uses this information to process subsequent requests. In essence, it remembers the "state" of the data over time. This means that data written to the database persists and is available for future reads and modifications, maintaining its integrity across operations and sessions.

Simply put, it saves user data permanently so you can seamlessly read and update it whenever you reconnect.

In short, it retains user modifications persistently, allowing you to access and update the exact same data across future sessions.

It means user data is persistently stored and ready to be read or modified upon any future request.

Key characteristics of stateful databases include:

• Persistence: Data is stored durably and survives system restarts or power failures.
• Consistency: They often strive to maintain data consistency, ensuring that data is valid according to predefined rules.
• Transaction Management: They support transactions, which are sequences of operations performed as a single logical unit of work, ensuring atomicity and isolation.
• Data Integrity: Mechanisms are in place to prevent data corruption and ensure the reliability of stored information.

Most traditional databases, including relational databases, are inherently stateful, as are many NoSQL databases designed for persistent storage.

Yes, probably most of you guys already experienced a stateful database such as MySQL, Postgresql, BigQuery, etc.

How is it different from a Relational Database?

While all relational databases are stateful, the question implies a comparison of common stateful database types with the relational model. Here's how relational databases, often serving as a benchmark, compare to other stateful database approaches (which often fall under the "NoSQL" umbrella, like Cassandra, or "NewSQL" like TiDB, aiming for different scaling or consistency models):

Relational Databases (e.g., MySQL, PostgreSQL, Oracle):

• Data Model: Based on the relational model, organizing data into tables with predefined schemas, rows, and columns. Relationships between tables are defined using foreign keys.
• Schema: Generally "schema-on-write," meaning the schema must be defined before data can be inserted, and changes typically require schema migrations.
• Consistency Model: Primarily adhere to ACID properties (Atomicity, Consistency, Isolation, Durability), prioritizing strong consistency.
• Scalability: Traditionally scale vertically (more powerful hardware), though horizontal scaling (sharding) is possible but often complex to manage.
• Query Language: Standardized SQL (Structured Query Language).
• Strengths: Excellent for complex queries, data integrity, transactions, and applications requiring strong consistency.

Other Stateful Databases (often NoSQL or NewSQL, like Cassandra, MongoDB, TiDB):

• Data Model: Varies widely, including key-value, document, column-family, graph, or distributed SQL. They might offer more flexible ways to store and structure data.
• Schema: Can be "schema-on-read" (flexible, schema defined when reading) or "schema-on-write" with flexible column definitions (Cassandra), or even full SQL schema (TiDB).
• Consistency Model: Often prioritize availability and partition tolerance over immediate strong consistency, offering tunable consistency (Cassandra) or strong consistency with distributed ACID transactions (TiDB). They may adhere to BASE properties (Basically Available, Soft state, Eventually consistent).
• Scalability: Designed for horizontal scalability from the ground up, allowing easy distribution of data and workload across many commodity servers.
• Query Language: Can vary from specific query languages (e.g., CQL for Cassandra), API calls, or standard SQL (e.g., TiDB's MySQL compatibility).
• Strengths: High availability, massive scalability (horizontal), ability to handle unstructured or semi-structured data, high throughput for specific access patterns, resilience to failures.

In essence, while both store persistent state, relational databases emphasize rigid structure and strong consistency, whereas other stateful databases often prioritize flexibility, massive scale, and availability, sometimes by relaxing consistency guarantees or adopting a distributed SQL approach.

Information for Cassandra and TiDB

Example System Architecture using Cassandra

Cassandra's architecture is a distributed, decentralized, and peer-to-peer system designed for high availability and linear scalability.

Explanation:

• Client Applications: Interact with the Cassandra cluster through application services.
• Load Balancer / DataStax OpsCenter Driver: Clients (or application services) connect to the Cassandra cluster. While a generic load balancer can be used, Cassandra's client drivers are often intelligent enough to handle load balancing and fault tolerance by themselves, connecting directly to multiple nodes.
• Cassandra Cluster (Ring Topology):
  • Nodes: Each Cassandra node in the cluster runs identical software and can handle read/write requests.
  • Ring Topology: Nodes are organized into a ring. Data is partitioned and distributed across the nodes.
  • Replication: Data is replicated across multiple nodes (determined by the Replication Factor) to ensure high availability and fault tolerance. In a multi-datacenter setup, data is replicated across different DCs for geo-redundancy and disaster recovery.
  • Gossip Protocol: Nodes communicate using a gossip protocol to discover each other, track state, and detect failures without a central master.

This architecture ensures no single point of failure, allowing the system to continue operating even if multiple nodes or entire data centers become unavailable.

Example System Architecture using TiDB

TiDB is a NewSQL database designed for large-scale, strongly consistent, and horizontally scalable relational workloads, often used for Hybrid Transactional/Analytical Processing (HTAP). It separates the SQL layer, the scheduling layer, and the distributed key-value storage layer.

Explanation:

• Client Applications: Connect to TiDB using standard MySQL clients or connectors.
• Load Balancer: Distributes incoming MySQL-compatible SQL queries across multiple TiDB Server instances.
• TiDB Servers: These are the SQL layer. They parse SQL, optimize queries, and interact with the PD and TiKV clusters to execute transactions and retrieve/store data. They are stateless and can be scaled independently.
• PD Cluster (Placement Driver): The "brain" of the TiDB cluster. It stores metadata, performs global timestamp allocation for transactions, and schedules data distribution and load balancing across the TiKV nodes. It typically runs as a small, highly available cluster (e.g., 3-5 nodes) using Raft for consensus.
• TiKV Cluster (Distributed K-V Storage): The distributed transactional key-value store. It stores all the actual data in sorted key-value pairs. Data is automatically sharded into "regions" and replicated across multiple TiKV nodes using Raft consensus for strong consistency and high availability. It scales horizontally by adding more TiKV nodes.
• TiFlash Cluster (Optional): An analytical column store engine. It replicates data from TiKV in a columnar format to accelerate analytical queries, enabling HTAP capabilities.

This layered architecture allows for independent scaling of compute (TiDB Servers), storage (TiKV/TiFlash), and scheduling (PD), providing flexibility and high performance for both transactional and analytical workloads.