Enterprise-Scale 100k+ TPS Order Tracking System Architecture

System Architecture Overview

To handle 100,000+ Transactions Per Second (TPS) for order inquiries, the system must decouple write operations from read operations. A traditional relational database (RDBMS) cannot handle this scale of read traffic directly.

This architecture adopts the CQRS (Command Query Responsibility Segregation) pattern, separating the Write Database (OLTP) from the Read Database (OLAP/Cache), and utilizes multi-level caching and asynchronous event-driven synchronization.

1. Resolving Database Bottlenecks

To handle 100k TPS without degrading database performance, read traffic must be diverted away from the primary transactional database.

Multi-Tier Caching Strategy

• L1 Cache (In-Memory JVM Cache): Use local caches like Caffeine Cache within the application instances for extremely hot data (e.g., popular merchant orders, system-wide configurations). This reduces network latency to 0ms.
• L2 Cache (Distributed Cache): Deploy a Redis Enterprise Cluster with sharding.
  • Sizing: Scale Redis to handle 90%+ of simple order status inquiries.
  • Data Structure: Use Redis Hashes to store serialized order JSONs.
  • High Availability: Use Redis Sentinel or Cluster with master-replica replication across Multiple Availability Zones (Multi-AZ).

Read-Optimized NoSQL & Search Engine

• Elasticsearch Cluster: For complex, multi-conditional inquiries (e.g., searching orders by date range, item name, and status), sync RDBMS data to Elasticsearch. Elasticsearch shards can scale horizontally to handle high-throughput search queries.
• DynamoDB / Cassandra: For historical order lookups, migrate orders older than 30 days to a highly scalable NoSQL database optimized for key-value reads.

Database Sharding and Read Replicas

• RDBMS (PostgreSQL/MySQL): Maintain write-heavy data here.
• Read Replicas: Deploy multiple read replicas behind a load balancer (e.g., AWS Aurora Reader Endpoints) to handle fallback read traffic that misses the cache.
• Database Sharding: Shard the database horizontally by User_ID or Order_Date to prevent any single database instance from becoming a bottleneck during peak write periods, which indirectly improves read replication lag.

2. Concurrency Control & Traffic Management

Handling 100k concurrent requests requires robust traffic shaping and non-blocking I/O architectures.

Asynchronous, Non-Blocking Edge Layer

• Reactive Frameworks: Implement the API layer using non-blocking, event-driven frameworks like Spring WebFlux (Project Reactor) or Go (Goroutines). Traditional thread-per-request models (like Tomcat) will experience thread pool exhaustion at 100k TPS.
• Connection Pooling: Optimize connection pools (e.g., HikariCP for SQL, LetTuce for Redis) to handle high-frequency recycling of connections.

Traffic Throttling and Rate Limiting

• API Gateway (Kong / AWS API Gateway): Implement rate limiting using the Token Bucket or Leaky Bucket algorithm at the user/IP level.
• Load Shedding: When system load exceeds a critical threshold (CPU > 85%), automatically reject low-priority requests (e.g., historical order searches) with an HTTP 429 (Too Many Requests) to protect core inquiry paths.

3. Transactional Integrity & Data Consistency

Because read and write paths are separated (CQRS), maintaining data consistency between the transactional database and the read stores (Redis/Elasticsearch) is critical.

Event-Driven Synchronization (CDC)

To prevent write-path latency, do not update caches synchronously during the write transaction. Use Change Data Capture (CDC).

1. Debezium + Apache Kafka: Captures row-level changes (Insert/Update) from the RDBMS transaction logs (e.g., MySQL binlog) in real-time.
2. Kafka Message Broker: Publishes change events to an order-events topic partitioned by Order_ID to guarantee sequential processing.
3. Cache Sync Consumers: Lightweight consumer applications read from Kafka and update Redis and Elasticsearch. This keeps the data sync lag under 100 milliseconds.

Addressing Distributed Transaction Issues

• Transactional Outbox Pattern: To guarantee that an order creation/update is both saved to the DB and published to Kafka without distributed transactions (2PC), write the event to an outbox table within the same local RDBMS transaction. A relay process then reads the outbox table and publishes to Kafka.
• Idempotency: Consumers updating the read stores must inspect the version or updated_at timestamp of the incoming message. If a message with an older or equal version than what is currently in Redis/Elasticsearch arrives, it is discarded to prevent out-of-order updates.

Cache Consistency Patterns

• Cache-Aside with TTL: For standard orders, read from cache; on miss, read from replica DB and write to cache with a Time-To-Live (TTL) (e.g., 10 minutes).
• Write-Through via CDC: For active, high-frequency orders (e.g., newly placed orders), the CDC pipeline immediately pushes updates to Redis, ensuring the cache is populated before the first inquiry request arrives.

4. Disaster Recovery & High Availability

At 100k TPS, even a few seconds of downtime affects thousands of users. The infrastructure must be self-healing.

Circuit Breakers and Fallbacks

• Resilience4j Integration: Implement circuit breakers on the cache layers.
• Fallback Path: If the Redis Cluster fails, the system must degrade gracefully. Instead of crashing the RDBMS with 100k TPS, the API layer should:
  1. Direct a throttled percentage of reads to RDBMS Read Replicas.
  2. Serve slightly stale data from a backup static CDN or Elasticsearch.
  3. Return a user-friendly error message asking the user to retry, protecting the master database.

Multi-Region Active-Active Deployments

• Global Load Balancing (GSLB): Use AWS Route 53 or Cloudflare to route user traffic to the geographically nearest Data Center/Region.
• Cross-Region Replication: Deploy the system in an Active-Active setup across two regions. Use globally replicated databases (e.g., AWS Aurora Global Database) and cross-region Redis replication to ensure instant failover capability.