{"id":29005,"date":"2026-07-30T23:56:51","date_gmt":"2026-07-30T21:56:51","guid":{"rendered":"https:\/\/blog.mi.hdm-stuttgart.de\/?p=29005"},"modified":"2026-07-30T23:56:54","modified_gmt":"2026-07-30T21:56:54","slug":"polyglot-data-storage-in-ultra-large-scale-systems-a-workload-driven-architecture-for-a-ride-hailing-platform","status":"publish","type":"post","link":"https:\/\/blog.mi.hdm-stuttgart.de\/index.php\/2026\/07\/30\/polyglot-data-storage-in-ultra-large-scale-systems-a-workload-driven-architecture-for-a-ride-hailing-platform\/","title":{"rendered":"Polyglot Data Storage in Ultra-Large-Scale Systems: A Workload-Driven Architecture for a Ride-Hailing Platform"},"content":{"rendered":"\n

Abstract:<\/strong> The assumption that a single type of data storage can serve every workload no longer holds at ultra-large scale. This paper examines storage in a globally distributed ride-hailing platform, where each workload is mapped to a storage engine whose guarantees match its access pattern, consistency requirement, and query shape. The mapping is justified through the PACELC framework<\/strong>, which labels each operational database by its behaviour both during network partitions and during normal operation. Object storage is treated separately as a shared layer for blobs, cold history copies, and lakehouse files. This paper pursues three primary goals. First, it establishes a PACELC-labelled reference mapping of the platform\u2019s operational databases, grounded in published components of the Uber stack. Second, it demonstrates how cross-service correctness, via the saga<\/strong> and transactional-outbox<\/strong> patterns, and the analytical pipeline can reuse common change-data-capture<\/strong> infrastructure while separating operational saga topics from analytical replication topics. Third, it provides a scoped assessment of convergence trends that reduce the number of distinct systems an infrastructure must operate. The analysis is a conceptual synthesis argued from published system properties rather than an empirical benchmark.<\/p>\n\n\n\n

Index Terms:<\/strong> ultra-large-scale systems, polyglot persistence, PACELC, distributed databases, sharding, change data capture, saga pattern, lakehouse<\/p>\n\n\n\n

1 Introduction<\/h2>\n\n\n\n

The relational database was historically applied across a wide range of workloads, from interactive requests to long-running reports. Stonebraker and C\u00b8 etintemel argued early on that a single database architecture cannot satisfy all requirements. They predicted a split into specialised storage systems tailored for distinct workload categories such as online transaction processing, data warehousing, stream processing, and text search [1]<\/a>. Twenty years later, per-workload engine selection is standard practice in large systems. This paper examines the resulting design space, in which a storage engine is chosen per workload, and uses a globally distributed ride-hailing platform as a running example.<\/p>\n\n\n\n

This platform qualifies as an ultra-large-scale system. Northrop et al. define such a system as an infrastructure whose scale in at least one dimension makes conventional software methods inadequate, and describe it as a socio-technical ecosystem rather than a single application [2, pp. 11, 125]<\/a>. Four of their defining criteria bear directly on storage. Control is decentralised, with no central authority over the data or its daily operation [2, p. 13]<\/a>. The elements are heterogeneous, independently developed, and changing [2, p. 16]<\/a>. The system evolves and deploys continuously while it keeps running [2, p. 15]<\/a>, and hardware and software failures are routine rather than rare [2, p. 19]<\/a>. A ride-hailing platform shows each of these traits. Its features are split across independently owned services, so no single component controls the full data model. Each service selects a storage engine suited to its own workload. The platform runs across several regions without pause, and at that scale individual node and network failures are expected. These four criteria shape the storage decisions in the sections that follow.<\/p>\n\n\n\n

The main division in this storage design space is between operational and analytical systems. Operational systems handle short keyed reads and writes for interactive requests. Analytical systems scan and aggregate larger data sets for reporting and business analysis. These patterns are commonly called OLTP (Online Transaction Processing)<\/strong> and OLAP (Online Analytical Processing)<\/strong> [3, p. 5]<\/a>. The difference is not only in the queries. Operational and analytical systems require distinct internal data layouts because they handle different data types, access patterns, and user groups [3, p. 25]<\/a>. One physical representation cannot serve both well.<\/p>\n\n\n\n

The same tension appears inside the operational side. Sadalage and Fowler build on Neal Ford\u2019s 2006 idea of polyglot programming and apply it to storage. They note that a store which is highly available and scalable enough for a shopping cart cannot answer relationship queries such as products bought by a customer\u2019s friends, and they call this mixed approach polyglot persistence<\/strong> [4]<\/a>. The point generalises: session data, cart data, and order data do not require the same availability, consistency, or backup guarantees.<\/p>\n\n\n\n

A ride-hailing platform exhibits both extremes. On one side, constant location updates from millions of vehicles create high throughput demand, measured as requests or data volume processed per second. On the other side, payments occur less often but must commit with low response time and durable correctness. Response time includes all delays observed by the client and is best treated as a distribution rather than a single number [3, p. 41]<\/a>. The higher percentiles, or tail latencies<\/strong>, require optimisation because they influence the responsiveness experienced by end users [3, p. 40]<\/a>, and this effect is amplified when one end-user request fans out to several backend calls [3, p. 41]<\/a>. No single engine can simultaneously optimise for millions of short-lived location writes per second and strict transactional correctness for payments.<\/p>\n\n\n\n

Getting this choice right affects reliability<\/strong> and maintainability<\/strong>. Reliability means that the system continues to work correctly under faults, while maintainability covers operability, simplicity, and evolvability [3, pp. 43, 52\u201356]<\/a>. Storage design determines many of these qualities.<\/p>\n\n\n\n

The remainder of the paper treats storage as one problem spanning both planes. Section 2 establishes the consistency lens (CAP and PACELC). Section 3 develops the operational (OLTP) storage landscape as a polyglot mapping and adds a treatment of sharding. Section 4 addresses correctness across stores through the saga and transactional-outbox patterns. Section 5 extends the picture to the analytical (OLAP) plane with change data capture, a lakehouse, and a shared object-storage layer. Section 6 covers convergence trends, Section 7 states limitations, and Section 8 concludes. The mapping is argued from published system properties rather than from new measurements.<\/p>\n\n\n\n

2 Consistency Foundations: CAP and PACELC<\/h2>\n\n\n\n

Choosing a storage engine per workload is only useful if the guarantee each engine gives up can be stated precisely. In a distributed system the central property being traded is consistency, and the classic tool for reasoning about it is the CAP theorem<\/strong>. It is a useful starting point, but on its own it lacks the resolution needed for the design decisions in later sections, so this section establishes a more complete lens.<\/p>\n\n\n\n

Brewer initially proposed CAP as a conjecture. Gilbert and Lynch later provided the formal proof, demonstrating that an asynchronous network cannot simultaneously ensure data availability and linearizability<\/strong> during message drops [5]<\/a>. Atomic consistency here means linearizability. That is, there exists a total order on all operations such that each read observes the most recent completed write, as if there were a single copy of the data [5]<\/a>. The popular summary of CAP claiming that systems can only pick two out of three guarantees is misleading [3, p. 415]<\/a>. Brewer later clarified that selecting two out of three guarantees is an inaccurate model. The theorem only rules out achieving perfect availability and consistency during rare network partitions, leaving the rest of the design space open [6]<\/a>.<\/p>\n\n\n\n

A further limitation concerns its scope. The formal CAP theorem covers only linearizability and network partitions. It ignores common operational issues such as latency and node failures, which limits its practical value for modern system design [3, p. 415]<\/a>. CAP constrains the design only while a partition is actually occurring. It is silent about the trade-off a system faces during normal operation, which is most of the time.<\/p>\n\n\n\n

During normal operation the trade-off is between consistency and latency. Keeping replicas strongly consistent has a cost paid on every request, not only during failures. Strong consistency imposes an acceptable cost in data centres connected by fast and reliable networks [3, p. 401]<\/a>. The performance penalty never completely disappears. A linearizable read from a quorum system requires synchronous read repair [3, p. 412]<\/a>. Geographically distributed setups force follower regions to forward writes and linearizable reads synchronously over the network to the central leader [3, p. 414]<\/a>. Strong consistency incurs a latency cost during normal operation, not only during failures.<\/p>\n\n\n\n

Abadi extends CAP into the PACELC framework<\/strong> to cover normal operation. PACELC defines how a system balances availability against consistency during a partition, and how it balances latency against consistency when the network works normally [7]<\/a>. It addresses an oversight in CAP, because the consistency and latency trade-off of replicated systems is present at all times, whereas CAP applies only to rare network partitions [7]<\/a>.<\/p>\n\n\n\n

PACELC therefore attaches two labels to each store instead of one. A store is described by a pair: its partition-time behaviour (PA or PC) and its normal-time behaviour (EL or EC), written together as, for example, PC\/EC or PA\/EL. Abadi classifies default Dynamo, Cassandra, and Riak as PA\/EL, and fully ACID systems as PC\/EC [7]<\/a>, categories that recur in Section 3.<\/p>\n\n\n\n

Eventual consistency<\/strong> provides no strict bounds on replication delay, allowing replicas to fall behind indefinitely [3, p. 209]<\/a>. Replication lag typically stays below a second during normal operation and increases to minutes under heavy load or network degradation [3, p. 209]<\/a>. The relevant design concern is the kind of guarantee being relaxed. Kleppmann and Riccomini distinguish between timeliness<\/strong> and integrity<\/strong>. An eventually consistent store may relax timeliness by serving stale reads, while the application must still preserve integrity and avoid data corruption or loss [3, p. 571]<\/a>. This distinction permits placing a store on the EL side without giving up correctness. The EL side accepts stale reads, but it does not accept lost or corrupted data.<\/p>\n\n\n\n

The remainder of the paper applies this framework to each operational database. Each is placed on the PACELC map according to what its workload can tolerate. Payments and order state sit on the PC\/EC side, paying latency for correctness. Live location and active-ride state sit on the PA\/EL side, accepting staleness for speed and availability. Fig. 1<\/a> places the platform\u2019s main stores on the two PACELC axes; the full per-store mapping follows in Section 3.<\/p>\n\n\n\n

\"\"<\/a>
Fig. 1:<\/strong> PACELC placement of the platform’s main stores. The horizontal axis is normal-time behaviour (EC to EL). The vertical axis is partition-time behaviour (PC to PA). Correctness-first stores occupy the PC\/EC quadrant and latency-first stores occupy the PA\/EL quadrant. Ratings sits at the tolerant edge of the PA\/EL group. The figure previews the mapping in Table 1<\/a>.<\/figcaption><\/figure>\n\n\n\n

3 Polyglot Operational Storage<\/h2>\n\n\n\n

A. Ownership and Boundaries<\/em><\/h3>\n\n\n\n

Polyglot persistence<\/strong> is the practice of a single application using several stores, each chosen for a different job [4]<\/a>. To apply it, the boundaries between stores must first be decided. In a microservice architecture the boundary is the service itself. Each service usually owns its database and exposes its data through its API [3, pp. 21\u201322]<\/a>.<\/p>\n\n\n\n

This isolation exists because shared tables couple services through schema, performance, and deployment. If several services depend on the same database structure, schema changes and independent deployment become harder [3, pp. 21\u201322]<\/a>, [8, pp. 224, 551]<\/a>. This keeps storage decisions behind the service boundary.<\/p>\n\n\n\n

Where these boundaries should be drawn is a modelling question, and Domain-Driven Design gives it a name. A bounded context<\/strong> defines the exact boundaries of a domain model. This ensures development teams agree on internal consistency rules and explicit external integration points [9, p. 413]<\/a>. Each bounded context keeps its own model consistent internally and integrates with others through explicit translation at the boundary. Storage is split along this same line, giving one service, one bounded context, and one owned store. While this split is ideal, a bounded context may span multiple services and a single service may own more than one store if needed. Nonetheless, this work uses a one-to-one mapping because it shows the per-workload storage choice clearly. It also establishes that consistency is expected to hold inside a store, not automatically across stores.<\/p>\n\n\n\n

Richardson\u2019s Database per Service<\/strong> pattern formalises this ownership model by assigning each service its own data store [10, pp. 34, 46, 58]<\/a>. Once storage is private per service, each owner is free to select the engine that fits its workload, which turns Database per Service into polyglot persistence in practice.<\/p>\n\n\n\n

This freedom carries a cost, because splitting data across private stores removes two guarantees a single database provided automatically. The first is cross-store querying. Data distributed across multiple operational systems becomes difficult to query together, creating data silos [3, p. 6]<\/a>. Section 5 addresses this problem with analytics and a lakehouse. The second is the cross-store transaction. Private service databases move the burden of data consistency to the application layer, and standard distributed transactions are generally avoided in microservices because they create tight coupling [3, p. 21]<\/a>. The Database per Service architecture therefore needs the saga pattern to maintain data consistency [10, p. 46]<\/a>.<\/p>\n\n\n\n

B. Mapping workloads to stores<\/em><\/h3>\n\n\n\n

With ownership settled, each service selects the engine that fits its workload, and the PACELC framework<\/strong> from Section 2 supplies the terms that justify each choice. Table 1<\/a> lists the platform\u2019s operational stores and Fig. 2<\/a> arranges them into two bands by consistency behaviour on a shared object-storage layer. Each PACELC label shows the behaviour of the specific configuration assumed here. The same storage solution under a different configuration may be best classified using a different label.<\/p>\n\n\n\n

\n\t
\n\t\t\n\t\t\t\n\t\t\t\t\n\t\t\t\t\t\n\t\t\t\t\n\t\t\t\t\t\n\t\t\t\t\t\n\t\t\t\t\t\n\t\t\t\t\t\n\t\t\t\t\t\n\t\t\t\t\t\n\t\t\t\t\t\n\t\t\t\t\t\n\t\t\t\t\t\n\t\t\t\t\t\n\t\t\t\t\t\n\t\t\t\t\t
#<\/th>\n\t\t\t\t\tService \/ data<\/th>\n\t\t\t\t\tAccess pattern<\/th>\n\t\t\t\t\tEngine (class)<\/th>\n\t\t\t\t\tPACELC \/ class<\/th>\n\t\t\t\t\tMain reason<\/th>\n\t\t\t\t\tAccepted cost<\/th>\n\t\t\t\t<\/tr>\n\t\t\t<\/thead>\n\t\t\t
1<\/td>\n\t\t\t\t\tPayment<\/td>\n\t\t\t\t\trare writes, durable commit<\/td>\n\t\t\t\t\tDistributed SQL1<\/sup><\/td>\n\t\t\t\t\tPC\/EC<\/strong><\/td>\n\t\t\t\t\tdurable commit, correct final state<\/td>\n\t\t\t\t\thigher latency<\/td>\n\t\t\t\t<\/tr>\n\t\t\t\t
2<\/td>\n\t\t\t\t\tOrder<\/td>\n\t\t\t\t\tsaga participant, keyed by transaction<\/td>\n\t\t\t\t\tDistributed SQL<\/td>\n\t\t\t\t\tPC\/EC<\/strong><\/td>\n\t\t\t\t\tsaga participant, correct ride state<\/td>\n\t\t\t\t\thigher latency<\/td>\n\t\t\t\t<\/tr>\n\t\t\t\t
3<\/td>\n\t\t\t\t\tMatching Service<\/td>\n\t\t\t\t\tauthoritative match state<\/td>\n\t\t\t\t\tDistributed SQL<\/td>\n\t\t\t\t\tPC\/EC<\/strong><\/td>\n\t\t\t\t\tone ride reserves one driver<\/td>\n\t\t\t\t\tlower availability under partition<\/td>\n\t\t\t\t<\/tr>\n\t\t\t\t
4<\/td>\n\t\t\t\t\tUser<\/td>\n\t\t\t\t\tprofile \/ authentication<\/td>\n\t\t\t\t\tDistributed SQL<\/td>\n\t\t\t\t\tPC\/EC<\/strong><\/td>\n\t\t\t\t\tcorrect profiles and authentication<\/td>\n\t\t\t\t\thigher read\/write latency<\/td>\n\t\t\t\t<\/tr>\n\t\t\t\t
5<\/td>\n\t\t\t\t\tVehicle<\/td>\n\t\t\t\t\tfleet registry (seat count, class)<\/td>\n\t\t\t\t\tDistributed SQL<\/td>\n\t\t\t\t\tPC\/EC<\/strong><\/td>\n\t\t\t\t\tcorrect eligibility checks<\/td>\n\t\t\t\t\thigher read\/write latency<\/td>\n\t\t\t\t<\/tr>\n\t\t\t\t
6<\/td>\n\t\t\t\t\tDriver<\/td>\n\t\t\t\t\tdriver profile \/ status<\/td>\n\t\t\t\t\tDistributed SQL<\/td>\n\t\t\t\t\tPC\/EC<\/strong><\/td>\n\t\t\t\t\tcorrect driver status<\/td>\n\t\t\t\t\thigher read\/write latency<\/td>\n\t\t\t\t<\/tr>\n\t\t\t\t
7<\/td>\n\t\t\t\t\tLive Location<\/td>\n\t\t\t\t\tmillions of GPS writes per second<\/td>\n\t\t\t\t\tRedis2<\/sup><\/td>\n\t\t\t\t\tPA\/EL<\/strong><\/td>\n\t\t\t\t\tvery low latency for transient writes<\/td>\n\t\t\t\t\tstale values after failover<\/td>\n\t\t\t\t<\/tr>\n\t\t\t\t
8<\/td>\n\t\t\t\t\tActive Rides<\/td>\n\t\t\t\t\tfrequently accessed ride session state<\/td>\n\t\t\t\t\tRedis<\/td>\n\t\t\t\t\tPA\/EL<\/strong><\/td>\n\t\t\t\t\tlow-latency session state<\/td>\n\t\t\t\t\tstate rebuild after failover<\/td>\n\t\t\t\t<\/tr>\n\t\t\t\t
9<\/td>\n\t\t\t\t\tRoute<\/td>\n\t\t\t\t\ttrajectory, high write volume<\/td>\n\t\t\t\t\tCassandra3<\/sup><\/td>\n\t\t\t\t\tPA\/EL<\/strong><\/td>\n\t\t\t\t\thigh write volume, append-heavy<\/td>\n\t\t\t\t\tstale reads under relaxed quorum<\/td>\n\t\t\t\t<\/tr>\n\t\t\t\t
10<\/td>\n\t\t\t\t\tCompleted ride history copy<\/td>\n\t\t\t\t\trecent trip history, then archive<\/td>\n\t\t\t\t\tCassandra; object storage for old history<\/td>\n\t\t\t\t\tPA\/EL<\/strong><\/td>\n\t\t\t\t\tlow-cost trip-history reads and archive<\/td>\n\t\t\t\t\tcopy may lag behind the authoritative record<\/td>\n\t\t\t\t<\/tr>\n\t\t\t\t
11<\/td>\n\t\t\t\t\tRatings<\/td>\n\t\t\t\t\tappend, staleness-tolerant<\/td>\n\t\t\t\t\tCassandra<\/td>\n\t\t\t\t\tPA\/EL<\/strong><\/td>\n\t\t\t\t\tappend-heavy, staleness-tolerant<\/td>\n\t\t\t\t\tstale reads under relaxed quorum<\/td>\n\t\t\t\t<\/tr>\n\t\t\t\t
12<\/td>\n\t\t\t\t\tMedia \/ large objects<\/td>\n\t\t\t\t\tobject upload, read-mostly<\/td>\n\t\t\t\t\tObject storage4<\/sup><\/td>\n\t\t\t\t\tShared layer<\/td>\n\t\t\t\t\tlow-cost durable storage for large files<\/td>\n\t\t\t\t\tno cross-object transactions, asynchronous cross-region replication<\/td>\n\t\t\t\t<\/tr>\n\t\t\t<\/tbody>\n\t\t<\/table>\n\t\t
Table 1:<\/strong> PACELC classification of the platform’s operational databases and the\nshared object-storage layer, with the reason for each choice and the cost the\narchitecture accepts.\n

1<\/sup> e.g. Google Spanner, CockroachDB, YugabyteDB.
2<\/sup> Redis Cluster with hash-based sharding and asynchronous replication. This topology can lose acknowledged writes on failover, which is why the store carries a PA\/EL label.
3<\/sup> Apache Cassandra.
4<\/sup> Amazon S3 and Ceph are examples. Object storage is treated as a shared layer outside the database PACELC labels. Amazon S3 provides strong read-after-write and list consistency within a region [11]<\/a>, while cross-region replication [12]<\/a> and lifecycle transitions [13]<\/a> are asynchronous.<\/p><\/figcaption>\n\t<\/figure>\n<\/div>\n\n\n\n

\"\"<\/a>
Fig. 2:<\/strong> Consistency-cut view of the operational tier. A service strip sits above two storage bands on a shared object-storage layer. The solid arrow shows completed ride history copies moving from Cassandra into object storage. The dashed arrow indicates the forward data flow to the analytical tier of Section 5<\/a>.<\/figcaption><\/figure>\n\n\n\n

The stores fall into two groups. The correctness-first group (PC\/EC) holds data where a stale or divergent read would compromise correctness or cause business harm: payments, orders, Matching Service state, and the registries for users, vehicles, and drivers. These stores pay the latency cost identified in Section 2, namely synchronous replication and reads served from an authoritative replica, because the alternative is double-booked drivers or financial loss. They are served by a distributed SQL<\/strong> engine such as Spanner, CockroachDB, or YugabyteDB, which provides strongly consistent transactions across nodes. The Matching Service depends on several of these stores at once. It reads authoritative vehicle data, such as seat count and vehicle class, and driver status to assign exactly one driver to one rider, which is why those registries take the PC\/EC side rather than tolerating staleness.<\/p>\n\n\n\n

The latency-first group (PA\/EL) holds data where freshness matters less than speed and availability. Live location absorbs millions of GPS writes per second, and active-ride session state must be read with minimal latency. Both are served by an in-memory store (Redis<\/strong>), where a one-second-stale location value does not affect correctness. Route trajectories, completed ride history copies, and Ratings are high-volume and append-heavy, and are served by Cassandra<\/strong>, whose leaderless, tunable replication, detailed in Section 3-C, trades linearizability for write throughput and availability. Ratings tolerate the replication lag quantified in Section 2: a rating that appears a few seconds late does not affect correctness, so it belongs on the EL side.<\/p>\n\n\n\n

The main completed ride record used for fare calculation, receipts, payment links, payouts, and disputes remains in the Order and Payment stores. Cassandra stores the history copy for rider and driver trip-history views and downstream analytics. A delay in this copy does not change the paid fare, payout, or final ride state.<\/p>\n\n\n\n

These are not only hypothetical assignments. Uber operates Apache Cassandra across tens of thousands of nodes to manage data for diverse operational workloads with varying read and write ratios [14]<\/a>. On the correctness-first side, Uber\u2019s Docstore is a distributed database built on top of MySQL, in front of which an integrated read-through cache called CacheFront handles extremely high read volumes while ensuring consistency via binlog change events instead of manual invalidation [15]<\/a>. CacheFront illustrates how very high read volumes can be served through an integrated cache while database change-log events keep cached data aligned with the source of truth. Its use of the database change log for cache invalidation is also an instance of the change-data-capture infrastructure discussed in Sections 4 and 5.<\/p>\n\n\n\n

Object storage<\/strong> (Amazon S3, Ceph) is the shared layer for large files, cold history copies, and lakehouse data. Completed ride history copies stay in Cassandra for recent trip-history reads and are archived here once they age. Amazon S3 gives strong consistency for object reads and listings within a region [11]<\/a>, while cross-region replication [12]<\/a> and lifecycle movement [13]<\/a> are asynchronous. The analytics pipeline in Section 5 draws on the same storage.<\/p>\n\n\n\n

Table 1<\/a> therefore turns the abstract PACELC trade-off into concrete choices. It labels each operational database, states the reason for each storage choice, and names the cost the architecture accepts. Object storage is listed as the shared layer for media, cold history, and lakehouse files rather than as a PACELC-labelled database.<\/p>\n\n\n\n

C. How each band keeps its guarantee<\/em><\/h3>\n\n\n\n

The two bands in Table 1<\/a> reach opposite PACELC labels because they replicate data in opposite ways. The correctness-first band routes all writes through a leader while the latency-first band lets any replica accept a write. Each choice explains both the guarantee and its cost.<\/p>\n\n\n\n

Leader-based replication (PC\/EC).<\/strong> Single-leader replication designates one replica to handle all client writes. This leader records the new data locally first [3, p. 198]<\/a>. The followers then receive the updates through a replication log [3, p. 199]<\/a>. Synchronous replication ensures the follower maintains an exact, current copy of the leader\u2019s data [3, p. 200]<\/a>. This incurs the expected normal-time latency cost because the leader halts new writes until the replica confirms receipt [3, p. 200]<\/a>. Fault-tolerant leader-based replication requires consensus. Common consensus algorithms such as Raft and Paxos follow a leader-based model and add automatic leader election and failover [3, p. 442]<\/a>. Paxos, introduced by Lamport, established consensus as agreement on a single value among unreliable processes [16]<\/a>. Raft manages a replicated log by electing one leader and giving that leader responsibility for the log [17]<\/a>. Log entries flow in one direction only, from leader to followers. An entry becomes durable once the leader has replicated it on a majority of the servers [17]<\/a>. This majority-commit rule lets a distributed SQL engine remain linearizable while surviving a minority of node failures, and it is why every store in the PC\/EC band accepts the leader-blocking latency as the cost required for correctness.<\/p>\n\n\n\n

Leaderless replication (PA\/EL).<\/strong> The latency-first band omits the leader role and permits any replica to accept client writes [3, p. 229]<\/a>. A quorum<\/strong> condition governs consistency. In a system of n replicas, reads require r responses and writes require w confirmations. If the sum of r and w exceeds n, the read and write sets overlap. In the simple case without concurrent writes or failed writes, this overlap lets a read include at least one replica that has seen the latest successful write [3, pp. 231\u2013232]<\/a>. This inequality is the parameter operators configure. Configuring the read and write quorums to sum to less than the total replica count increases the risk of stale reads, but it improves overall system availability and reduces latency [3, p. 233]<\/a>, which is the EL end of the trade-off. Availability under partition is preserved by additional mechanisms. Read repair writes fresh values back to stale replicas during reads, a background anti-entropy process reconciles differences, and a sloppy quorum allows reachable replicas to accept writes even when they are not normally responsible for that key [3, pp. 231, 236]<\/a>. This keeps the system writeable when a strict quorum cannot be formed. Together these mechanisms form the PA response to partition.<\/p>\n\n\n\n

Dynamo pioneered this design by prioritising high availability over consistency during failures [18]<\/a>. It functions as an eventually consistent store that relies on the identical overlapping quorum condition used in modern production systems [18]<\/a>. Dynamo\u2019s designers justified this by the tail-latency reasoning introduced in Section 1. The designers discarded median and average latency metrics, arguing that extreme percentiles are necessary to ensure consistent performance. They based their service level agreements strictly on the 99.9th percentile [18]<\/a>. Cassandra, which serves the Route store, the completed ride history copy, and the Ratings store in Table 1<\/a>, inherits this model directly. The system copies each item to a configured number of hosts. Depending on the client\u2019s consistency level, writes may wait for one replica, a quorum, or all replicas, while reads can likewise be served by one replica or by a quorum [19]<\/a>.<\/p>\n\n\n\n

The quorum condition shows that the PA\/EL label is a configuration choice. Setting w + r > n<\/em> gives the read and write quorums an overlapping replica and improves the chance of reading the latest write under normal operation. Even with w + r > n<\/em>, edge cases such as concurrent writes, a write that succeeds on fewer than w replicas, and reads during node recovery can still return stale values [3, pp. 233\u2013234]<\/a>. Setting w + r \u2264 n<\/em> trades that overlap for lower latency and higher availability. An operator applies this configuration per store, and in Cassandra per query, matching each store\u2019s consistency to the workload it serves. The PC\/EC and PA\/EL labels in Table 1<\/a> therefore state a default position for each store, and for the leaderless band that position remains adjustable toward stronger consistency when a given query requires it.<\/p>\n\n\n\n

Because a single replication choice fixes both PACELC labels together, the platform uses only the two aligned categories, PC\/EC and PA\/EL. The intermediate categories arise when a system decouples its partition-time stance from its normal-time one, as in MongoDB (PA\/EC) or PNUTS (PC\/EL) in Abadi\u2019s classification [7]<\/a>. No workload in Table 1<\/a> requires that decoupling, so neither category appears here.<\/p>\n\n\n\n

The analysis so far assumes replicas within a single region. Geographic distribution raises the normal-time cost described above. When the replicas of a PC\/EC store sit in data centres on separate continents, every linearizable read or write that must reach the leader carries the inter-region round-trip time, so the EC latency has a lower bound set by the propagation delay between regions [3, p. 414]<\/a>. Spanner accepts this cost and manages it with synchronised clocks while sharding data across Paxos groups in data centres spread worldwide [20]<\/a>. CockroachDB lowers cross-region latency by assigning Range-level leases to specific replicas. This leaseholder becomes the sole node authorised to serve strict reads and writes [21, p. 3]<\/a>. Administrators can bind leaseholders to the geographical region of primary access to avoid long network hops [21, p. 4]<\/a>. Stale read queries bypass the leaseholder entirely and resolve against the nearest local replica [21, pp. 6\u20137]<\/a>. YugabyteDB offers similar geographic controls by assigning tablet leaders to specific preferred regions to handle local client traffic [22]<\/a>.<\/p>\n\n\n\n

The PA\/EL band faces a related choice of region topology. Multi-leader setups process writes locally and replicate them asynchronously across regions [3, p. 217]<\/a>. This maintains write availability during network partitions since regional leaders function independently, but introduces consistency conflicts when concurrent writes occur [3, pp. 217, 222]<\/a>. Adly et al. confirm that asynchronous replication handles heavy loads and provides superior performance at the acceptable cost of occasionally serving stale data [23, p. 108]<\/a>. This multi-leader option applies to the Cassandra stores, which support active-active replication across regions. The Redis stores instead run a single primary per shard with failover, so each region reads from local replicas while writes route to the primary. On the platform the live-location and active-ride stores tolerate the resulting divergence, while the payment and order stores keep a single authoritative region and pay the cross-region latency for correctness.<\/p>\n\n\n\n

D. Partitioning for scale<\/em><\/h3>\n\n\n\n

Replication explains how each band keeps its consistency guarantee, but it does not by itself explain how a single store sustains that write volume. Horizontal partitioning, also called sharding<\/strong>, provides that capacity. Partitioning divides the dataset so that every distinct record belongs to exactly one shard [3, p. 251]<\/a>. Sharding and replication are normally combined by storing copies of each shard on multiple nodes [3, p. 251]<\/a>. The replication analysis of Section 3-C therefore applies within each shard, while sharding distributes shards across the cluster.<\/p>\n\n\n\n

In practice, two schemes are primarily used. Key-range sharding keeps keys sorted within each shard, which makes range scans efficient, but it can create a hot shard when many writes target nearby keys [3, p. 257]<\/a>. Hash sharding distributes skewed data evenly across nodes using a hash function [3, p. 258]<\/a>. This approach makes range queries inefficient because keys within a specific range end up scattered across multiple shards [3, p. 261]<\/a>. Skewed distributions overload individual shards and create hot spots [3, pp. 255\u2013256]<\/a>. Changes to the node count require the system to rebalance the data distribution and force clients to update their request routing paths [3, pp. 255, 265]<\/a>.<\/p>\n\n\n\n

Both PACELC bands rely on sharding, using the two schemes. In the PC\/EC band, CockroachDB applies range partitioning to split data into ordered chunks of roughly 64 MiB and assigns a Raft consensus group to each block [21]<\/a>. Spanner similarly distributes partitioned data geographically across independent Paxos state machines [20]<\/a>. In each case the per-shard consensus group is the same mechanism described in Section 3-C, applied once per shard. In the PA\/EL band, Dynamo implements consistent hashing to map keys onto a ring structure. It utilises virtual nodes to ensure new hardware assumes a proportional share of the existing load [18]<\/a>. Cassandra partitions data through a similar consistent hashing topology that assigns ring tokens to individual nodes [19]<\/a>.<\/p>\n\n\n\n

The hot-spot problem is concrete for a ride-hailing platform because location data is partitioned geographically. Uber\u2019s H3 system projects a hexagonal grid onto the map and converts geographic coordinates into 64-bit indexes [24]<\/a>. The system selects hexagons because the uniform distance to all neighbouring cell centres simplifies spatial analysis [24]<\/a>. The structure operates hierarchically across 15 different resolutions [25]<\/a>. An H3 cell index can serve both as a geospatial index and as a shard key. This design also makes the skew concrete: dense urban cells receive disproportionately many location writes and therefore become hot shards, which is the instance of the hot-spot problem that partitioning must manage. At the storage layer, Uber manages trip data using Schemaless, an append-only MySQL datastore that delegates the specific shard routing logic directly to the application layer [26]<\/a>.<\/p>\n\n\n\n

Partitioning sits below the service boundary of Section 3-A. It is an implementation choice inside a store, invisible across the service API. It is nonetheless the mechanism that lets the PA\/EL stores sustain their write volume and the PC\/EC stores scale horizontally while preserving per-shard linearizability.<\/p>\n\n\n\n

E. Comparison with single-engine designs<\/em><\/h3>\n\n\n\n

Two single-engine designs show the cost of the polyglot approach. A single distributed SQL deployment would give every workload strong consistency and one transaction boundary, which removes the need for sagas, but it would force the PC\/EC latency cost onto the live-location and route writes that can tolerate stale data, and its per-shard consensus would cap their write throughput. A single Cassandra deployment would handle that write volume and stay available under partition, but it would push payment and Matching Service correctness into application-level conflict resolution, which brings back the lost-update and double-assignment risks that the PC\/EC band removes. The polyglot mapping accepts the cost of running several engines in return for matching each workload to the guarantee it needs.<\/p>\n\n\n\n

4. Maintaining Consistency Across Services: The Saga Pattern<\/h2>\n\n\n\n

Because each service owns a private database, a cross-service operation cannot rely on a single ACID transaction, and consistency across services becomes an application-level concern [10, p. 46]<\/a>. The saga pattern<\/strong> is the standard response.<\/p>\n\n\n\n

Two-phase commit (2PC)<\/strong> provides atomicity across services but imposes costs that microservice architectures reject. In a two-phase commit, participants keep resources locked while waiting for the final decision. A coordinator crash leaves these resources blocked until manual or automatic recovery occurs [3, p. 331]<\/a>. Distributed transactions degrade performance due to extra disk synchronisation and network communication [3, p. 329]<\/a>. Furthermore, their tight synchronous integration often causes minor component errors to trigger broader system outages [3, p. 549]<\/a>. Newman advises using sagas instead of distributed transactions whenever possible. Distributed transactions introduce severe system complexity, create problematic failure modes, and often fail to yield the expected results [8, p. 554]<\/a>.<\/p>\n\n\n\n

A saga coordinates a sequence of local ACID transactions via messages to maintain consistency within each store [10, pp. 114, 121]<\/a>. Garcia-Molina and Salem introduced the concept by splitting long-lived transactions into subtransactions and pairing each with a compensating transaction to undo its effects [27]<\/a>. Sagas commit steps locally and cannot execute automatic rollbacks. A failure in a later step triggers compensating transactions in reverse order to undo the completed steps [10, pp. 122, 126]<\/a>. This independent execution of discrete steps removes the lengthy resource locking required by two-phase commits [8, p. 182]<\/a>.<\/p>\n\n\n\n

A saga\u2019s coordination logic follows one of two structures [10, p. 131]<\/a>. Choreography relies on decentralised event exchange between participants. Orchestration<\/strong> centralises the logic in a designated controller that commands the participants sequentially [10, p. 132]<\/a>. This paper uses orchestration, which suits a ride\u2019s request flow because that flow crosses several PC\/EC stores that a central coordinator can drive in a defined order. A confirmed ride spans three of the PC\/EC stores in Table 1<\/a>: Order, Payment, and Matching Service. The orchestrator issues the steps in sequence, that is, create the ride record (Order), authorise the fare (Payment), and assign exactly one driver (Matching Service), each a local ACID transaction in its owning service. If driver assignment fails because none is available, the orchestrator triggers compensating transactions in reverse by releasing the payment authorisation, then cancelling the ride record. The system reaches a consistent end state without any transaction spanning the three databases. Fig. 3<\/a> shows this forward path and its compensation.<\/p>\n\n\n\n

\"\"<\/a>
Fig. 3:<\/strong> Confirmed-ride saga across three PC\/EC stores. Solid black arrows show the forward orchestration (Order, Payment, Matching Service), each a local ACID transaction. If the Matching Service step cannot assign a driver, marked by the failure above it, the orchestrator runs the dashed compensating transactions in reverse order, cancelling the payment authorisation and then the ride record, and reaches a consistent end state without a transaction spanning the three databases.<\/figcaption><\/figure>\n\n\n\n

A saga is only as reliable as the messaging beneath it. Because a saga is message-driven, each local transaction must both commit its state change and emit the message that advances the saga. Performing these as two independent operations admits the dual-write failure in which the commit succeeds but the message is lost. The transactional outbox<\/strong> removes this gap. It is a table in the service database that stores messages to be published, and the service writes the business update and the outbox row in one local ACID transaction [10, p. 98]<\/a>. A message relay publishes these rows to the broker. At low scale, the relay can poll the outbox table. At higher scale, a log tailer reads the database commit log, such as a binlog or write-ahead log, and publishes committed outbox entries [10, pp. 98\u201399]<\/a>. The relay guarantees at-least-once delivery, so a consumer such as Payment must be idempotent<\/strong> to avoid a double authorisation. Storing the message identifier ensures idempotence. The system can safely retry failed messages without duplicating the operational side effects [3, p. 334]<\/a>.<\/p>\n\n\n\n

Sagas provide atomicity, consistency, and durability but omit isolation [10, p. 114]<\/a>. Intermediate updates become visible immediately upon local commit [10, p. 152]<\/a>. This visibility admits three anomalies. Lost updates occur when sagas overwrite unread changes. Dirty reads happen when a transaction observes uncompleted saga updates. Nonrepeatable reads manifest when two steps read the same data and obtain different results [10, p. 156]<\/a>. Each local transaction, including the one on the Matching Service store, is still fully ACID under distributed SQL. The isolation gap appears only between steps that commit at different times over the course of the saga. In the ride flow, a dirty read would occur if a rider-facing view reported a ride as confirmed after the Matching Service step but before a later compensation cancelled it.<\/p>\n\n\n\n

Applications implement specific countermeasure techniques to mitigate the impact of these concurrency anomalies [10, p. 114]<\/a>. Richardson lists six countermeasures, summarised in Table 2<\/a> [10, p. 162]<\/a>. For the ride flow, the Matching Service marks a candidate driver as RESERVED<\/em> while the saga is in progress, so other matching sagas exclude that driver and no second assignment occurs. The Order service keeps the ride in a PENDING<\/em> state until Payment and Matching Service have both completed, and only then emits RideConfirmed<\/em>. Together these remove the lost-update risk on the driver and prevent the rider-facing dirty read described above, since no view reports the ride as confirmed while it can still be compensated<\/p>\n\n\n\n

Countermeasure<\/th>Mechanism<\/th>Illustrative use<\/th><\/tr><\/thead>
Semantic lock<\/td>lock held until saga ends<\/td>lock assigned driver<\/td><\/tr>
Commutative update<\/td>order-independent writes<\/td>fare adjustments<\/td><\/tr>
Pessimistic view<\/td>reorder to cut risk<\/td>assign before charge<\/td><\/tr>
Reread value<\/td>verify before overwrite<\/td>re-check driver free<\/td><\/tr>
Version file<\/td>log updates, reorder later<\/td>ride status log<\/td><\/tr>
By value<\/td>pick mechanism per risk<\/td>per-fare risk choice<\/td><\/tr><\/tbody><\/table>
Table 2:<\/strong> Saga isolation countermeasures [10, p. 162]<\/a>. The final column gives one illustrative ride-flow application per countermeasure; the semantic lock is developed in depth in the text.<\/figcaption><\/figure>\n\n\n\n

The remaining five countermeasures address the same anomalies by other means. Commutative updates eliminate lost updates by designing operations that can execute in any order. The pessimistic view reorders a saga\u2019s steps to minimise the business risk of a dirty read. The reread value countermeasure prevents lost updates by verifying that a record is unchanged before overwriting it. The version file records the operations performed on a record so that noncommutative operations can be reordered into commutative ones. Finally, the by value countermeasure selects the concurrency mechanism per request from its business risk, choosing between a saga and a distributed transaction [10, p. 162]<\/a>.<\/p>\n\n\n\n

5. Analytical Access to Operational Data: From Data Warehouse to Lakehouse<\/h2>\n\n\n\n

The operational services described earlier serve OLTP workloads. Analytical queries have a different shape. These queries process massive record sets to generate business intelligence reports [3, p. 5]<\/a>. In a ride-hailing platform, demand forecasting, driver-utilisation reporting, and surge-pricing analysis are OLAP workloads. Executing them against the operational stores would degrade the point-query latency those stores are provisioned for, so the two workloads are separated.<\/p>\n\n\n\n

The standard target for analytical workloads is a data warehouse<\/strong>, a separate read-only copy of operational data used for queries and reporting without affecting OLTP operations [3, p. 7]<\/a>. Data reaches the warehouse through extract, transform, and load processes. The system extracts data from operational databases via periodic dumps or continuous streams, transforms it into an analysis-friendly format, and loads it into the data warehouse [3, p. 7]<\/a>. This consolidates the platform\u2019s polyglot operational stores into one queryable copy.<\/p>\n\n\n\n

Continuous extraction uses the same kind of update stream that underpins the outbox relay of Section 4. This mechanism is change data capture (CDC)<\/strong>, which observes database changes and exports them so they can be replicated to other systems [3, p. 503]<\/a>. The same CDC infrastructure can serve both saga coordination and analytical replication. Saga events advance the ride workflow, for example ride created, payment authorised, and driver assigned. Analytical events build the lakehouse copy. They may use different fields, retention periods, and filtering. The shared parts are the outbox, log tailer, broker, event identifiers, and schema registry<\/strong>. Fig. 4<\/a> shows this shared infrastructure.<\/p>\n\n\n\n

\"\"<\/a>
Fig. 4:<\/strong> Shared change-data-capture infrastructure. The log tailer reads committed outbox entries and publishes them to the broker. The broker exposes operational topics for saga participants and analytical topics for the lakehouse. Delivery is at least once, so consumers deduplicate by event identifier.<\/figcaption><\/figure>\n\n\n\n

With ordered event logs and idempotent consumers, heterogeneous services can use a common integration mechanism [3, p. 549]<\/a>. A schema registry beside the broker stores valid schema versions and checks compatibility [3, p. 190]<\/a>. CDC therefore replaces periodic bulk dumps with a low-latency, continuously maintained analytical copy.<\/p>\n\n\n\n

The warehouse commonly organises this data as a star schema. At its centre is a fact table whose rows represent events. Dimension tables describe entities such as rider, driver, location, and time, and are used for filtering and grouping [3, pp. 77\u201379]<\/a>. Fact tables record the numerical measurements of business events [28, p. 10]<\/a>. Dimension tables store the attributes required to filter and group those measurements [28, p. 40]<\/a>. For the platform, a completed-ride fact table keyed on rider, driver, location, and time dimensions supports the analytical queries above.<\/p>\n\n\n\n

A warehouse imposes a schema at load time while a data lake<\/strong> relaxes this. A data lake operates as a centralised repository for untransformed files. It drops the strict schemas of data warehouses and permits downstream consumers to parse the raw data exactly as needed [3, pp. 9\u201310]<\/a>.<\/p>\n\n\n\n

Operating a lake and a warehouse together is costly. In a two-tier design, data is first moved into the lake and later transformed again for the warehouse, which adds complexity, delay, and more failure modes [29]<\/a>. The extra jobs across both architectures can also introduce failures and bugs [29]<\/a>. The lakehouse<\/strong> unifies these architectures by applying database management capabilities, such as ACID transactions and indexing, directly over inexpensive object storage [29]<\/a>. The enabling mechanism is architectural. The architecture writes to an object store using standard formats like Apache Parquet and relies on an independent metadata layer to enforce transactional guarantees [29]<\/a>. Management features reside in the metadata layer while the bulk of the data stays in the object store [29]<\/a>. Because the platform already operates an object-storage layer, the lakehouse can reuse that same storage while adding its own transactional metadata and query layer.<\/p>\n\n\n\n

Analytical processing over this data takes two forms. Batch processing<\/strong> transforms bounded inputs of finite size into discrete outputs [3, pp. 451, 487]<\/a>. Stream processing<\/strong> manages continuous data arrivals and evaluates events immediately [3, pp. 487\u2013488, 544]<\/a>. The former suits periodic reporting over completed rides while the latter suits latency-sensitive analytics such as live surge computation over the streaming location and ride-status events.<\/p>\n\n\n\n

6. Convergence and Future Directions<\/h2>\n\n\n\n

The preceding treatment kept the operational and analytical tiers apart, joined only by a copy. The analytical replica, whether warehouse or lakehouse, was maintained separately and fed from the operational store by a change-data-capture pipeline. The copy is the source of the analytical latency in the design. Queries observe the operational state only after extraction, transformation, and loading. A convergence trend directly targets this separation.<\/p>\n\n\n\n

Hybrid transactional\/analytical processing (HTAP)<\/strong> denotes systems that support both OLTP and OLAP within one system. Ozcan et al. note that the term is now applied more broadly to systems that combine high-rate ingestion with analytical queries [30]<\/a>. The motivation is the freshness gap left by the copy. The demand for immediate analytical processing stems from real-time requirements and the high ingestion rates of mobile and IoT workloads [30]<\/a>. The ride-hailing platform fits this profile. Surge pricing is an analytical computation over the continuous flow of location and ride-status events whose value decays within seconds of the events that produce it. Unified database architectures directly address the data freshness problem as analytical queries can run closer to the latest operational commits, reducing or removing separate transformation and transfer steps [30]<\/a>. HTAP therefore removes the separately maintained, change-data-capture-fed analytical replica as a mandatory element, collapsing the extraction step rather than optimising it.<\/p>\n\n\n\n

Databricks Lakebase is an example of this convergence trend in a lakehouse setting [31]<\/a>. It is presented as a Lake Transactional\/Analytical Processing (LTAP)<\/strong> architecture that brings a managed, serverless, PostgreSQL-compatible operational database into the Databricks lakehouse environment [31]<\/a>. Rather than merging OLTP and OLAP inside one monolithic engine, the design unifies them through shared open storage and lakehouse metadata. Operational writes are made available in open columnar formats such as Delta Lake and Apache Iceberg, so analytical queries can access the same data without a separate ETL pipeline or duplicate operational copy [31]<\/a>. The same platform boundary also allows governance, security, and access control through Unity Catalog to apply across both transactional and analytical workloads [31]<\/a>. For the ride-hailing architecture considered here, such an LTAP design would mainly reduce the CDC-fed copy between selected operational stores and the lakehouse. It would not remove the need to classify workloads by latency, consistency, and query shape.<\/p>\n\n\n\n

A second convergence follows the same pattern of a general-purpose engine supporting a workload that previously justified a dedicated system. High-dimensional similarity searches over learned embeddings for tasks like fraud scoring and rider-driver matching traditionally required specialised vector databases<\/strong> like Milvus [32]<\/a>. Modern architectures increasingly extend established relational engines to handle vector search. This approach eliminates the operational overhead of managing an entirely separate system within the data infrastructure [33]<\/a>. The pgvector<\/strong> extension for PostgreSQL is the representative instance, exposing similarity search through the existing SQL interface rather than a separate query path [33]<\/a>, [34]<\/a>. The consolidation is thus visible along two axes at once, the operational\/analytical divide and the relational\/specialised divide, each reducing the number of distinct systems an infrastructure must operate.<\/p>\n\n\n\n

These trends narrow the polyglot argument without ending it. Convergence lowers the cost of running a given specialisation by folding it into a system that is already in place. It does not remove the differences in access pattern, consistency requirement, and query shape that made the specialisation necessary. Where one engine can serve two workloads well, the number of distinct systems drops. Where their requirements stay far apart, separate systems remain.<\/p>\n\n\n\n

7. Limitations<\/h2>\n\n\n\n

Several limitations need to be kept in mind when evaluating this paper. The work is a conceptual synthesis, and no benchmarking was performed, so the PACELC placements rest on the published properties of each system and not on measurements taken under the platform\u2019s own workload. The mapping in Table 1<\/a> is one defensible assignment among several. A team that prioritised reporting convenience might place the Ratings store on a relational engine, for instance. The Uber components cited in Section 3 serve as evidence that the classes of store discussed are operated at scale, and they are not a claim that the architecture presented here matches Uber\u2019s internal design. Operational figures reported in engineering blogs, such as the read rates cited for CacheFront, are self-reported by the operator and depend on hardware and configuration that the sources do not fully specify. Data security, regulatory compliance such as data-residency and deletion requirements, backup and disaster recovery, and cost modelling fall outside the scope of this paper.<\/p>\n\n\n\n

8. Conclusion<\/h2>\n\n\n\n

This paper examined the storage systems a ride-hailing platform requires, using the concrete demands of that domain to ground the design choices. The trip lifecycle, the driver-location stream, the surge computation, and the fraud and billing paths do not share a single access pattern, consistency requirement, or query shape, and no single engine serves them all without compromise.<\/p>\n\n\n\n

Several consequences follow from that starting point. Under geographic distribution the consistency and latency trade-off is paid continuously, whether or not a network partition is present, and PACELC exposes this cost where the CAP theorem is silent. Assigning each workload to a matched store then splits the operational tier into a correctness-first band under leader-based replication and a latency-first band under leaderless quorum, with horizontal partitioning in both to reach the required scale. The price of that split is the loss of a single cross-store transaction boundary, which sagas and the transactional outbox recover at the application level. The same outbox and log-tailing infrastructure can publish saga events and analytical replication events for the warehouse or lakehouse.<\/p>\n\n\n\n

The convergence trends surveyed in Section 6 reduce the total number of systems required, but they do not eliminate workload distinctions. HTAP and LTAP architectures execute both transactional and analytical queries on the same data. Similarly, in-engine vector search eliminates the need for dedicated vector databases. Despite this consolidation, engineers must still assign each workload to a storage engine based on its specific consistency guarantees and latency limits.<\/p>\n\n\n\n

References<\/h2>\n\n\n\n

1] M. Stonebraker and U. C\u00b8 etintemel, \u201cOne Size Fits All: An Idea Whose Time Has Come and Gone,\u201d in Proc. 21st Int. Conf. Data Engineering (ICDE \u201905), Tokyo, Japan, 2005, pp. 2\u201311. \u21a9<\/a><\/p>\n\n\n\n

[2] L. Northrop et al., Ultra-Large-Scale Systems: The Software Challenge of the Future. Pittsburgh, PA: Software Engineering Institute, Carnegie Mellon University, 2006. \u21a9<\/a><\/p>\n\n\n\n

[3] M. Kleppmann and C. Riccomini, Designing Data-Intensive Applications: The Big Ideas Behind Reliable, Scalable, and Maintainable Systems, 2nd ed. Sebastopol, CA: O\u2019Reilly Media, 2026. \u21a9<\/a><\/p>\n\n\n\n

[4] P. J. Sadalage and M. Fowler, NoSQL Distilled: A Brief Guide to the Emerging World of Polyglot Persistence. Upper Saddle River, NJ: Addison-Wesley, 2012. \u21a9<\/a><\/p>\n\n\n\n

[5] S. Gilbert and N. Lynch, \u201cBrewer\u2019s Conjecture and the Feasibility of Consistent, Available, Partition-Tolerant Web Services,\u201d ACM SIGACT News, vol. 33, no. 2, pp. 51\u201359, 2002. \u21a9<\/a><\/p>\n\n\n\n

[6] E. A. Brewer, \u201cCAP Twelve Years Later: How the \u2018Rules\u2019 Have Changed,\u201d Computer, vol. 45, no. 2, pp. 23\u201329, 2012. \u21a9<\/a><\/p>\n\n\n\n

[7] D. J. Abadi, \u201cConsistency Tradeoffs in Modern Distributed Database System Design: CAP is Only Part of the Story,\u201d Computer, vol. 45, no. 2, pp. 37\u201342, 2012. \u21a9<\/a><\/p>\n\n\n\n

[8] S. Newman, Building Microservices: Designing Fine-Grained Systems, 2nd ed. Sebastopol, CA: O\u2019Reilly Media, 2021. \u21a9<\/a><\/p>\n\n\n\n

[9] E. Evans, Domain-Driven Design: Tackling Complexity in the Heart of Software. Boston, MA: Addison-Wesley, 2003. \u21a9<\/a><\/p>\n\n\n\n

[10] C. Richardson, Microservices Patterns: With Examples in Java. Shelter Island, NY: Manning Publications, 2018. \u21a9<\/a><\/p>\n\n\n\n

[11] Amazon Web Services, \u201cAmazon S3 data consistency model,\u201d Amazon S3 User Guide, 2026. [Online]. Available: https:\/\/docs.aws.amazon.com\/AmazonS3\/latest\/userguide\/<\/a>. [Accessed: Jul. 09, 2026]. \u21a9<\/a><\/p>\n\n\n\n

[12] Amazon Web Services, \u201cReplicating objects within and across Regions,\u201d Amazon S3 User Guide, 2026. [Online]. Available: https:\/\/docs.aws.amazon.com\/AmazonS3\/latest\/userguide\/replication.html<\/a>. [Accessed: Jul. 09, 2026]. \u21a9<\/a><\/p>\n\n\n\n

[13] Amazon Web Services, \u201cTroubleshooting Amazon S3 Lifecycle issues,\u201d Amazon S3 User Guide, 2026. [Online]. Available: https:\/\/docs.aws.amazon.com\/AmazonS3\/latest\/userguide\/troubleshoot-lifecycle.html<\/a>. [Accessed: Jul. 09, 2026]. \u21a9<\/a><\/p>\n\n\n\n

[14] Uber Technologies, \u201cHow Uber Optimized Cassandra Operations at Scale,\u201d Uber Blog, 2022. [Online]. Available: https:\/\/www.uber.com\/de\/en\/blog\/how-uber-optimized-cassandra-operations-at-scale\/<\/a>. [Accessed: Jul. 06, 2026]. \u21a9<\/a><\/p>\n\n\n\n

[15] Uber Technologies, \u201cHow Uber Serves over 150 Million Reads per Second from Integrated Cache with Stronger Consistency Guarantees,\u201d Uber Blog, 2024. [Online]. Available: https:\/\/www.uber.com\/us\/en\/blog\/how-uber-serves-over-150-million-reads\/<\/a>. [Accessed: Jul. 06, 2026]. \u21a9<\/a><\/p>\n\n\n\n

[16] L. Lamport, \u201cThe Part-Time Parliament,\u201d ACM Trans. Computer Systems, vol. 16, no. 2, pp. 133\u2013169, 1998. \u21a9<\/a><\/p>\n\n\n\n

[17] D. Ongaro and J. Ousterhout, \u201cIn Search of an Understandable Consensus Algorithm,\u201d in Proc. 2014 USENIX Annual Technical Conf. (USENIX ATC \u201914), Philadelphia, PA, 2014, pp. 305\u2013319. \u21a9<\/a><\/p>\n\n\n\n

[18] G. DeCandia et al., \u201cDynamo: Amazon\u2019s Highly Available Key-value Store,\u201d in Proc. 21st ACM SIGOPS Symp. Operating Systems Principles (SOSP \u201907), Stevenson, WA, 2007, pp. 205\u2013220. \u21a9<\/a><\/p>\n\n\n\n

[19] A. Lakshman and P. Malik, \u201cCassandra: A Decentralized Structured Storage System,\u201d ACM SIGOPS Operating Systems Review, vol. 44, no. 2, pp. 35\u201340, 2010. \u21a9<\/a><\/p>\n\n\n\n

[20] J. C. Corbett et al., \u201cSpanner: Google\u2019s Globally Distributed Database,\u201d ACM Trans. Computer Systems, vol. 31, no. 3, pp. 1\u201322, 2013. \u21a9<\/a><\/p>\n\n\n\n

[21] R. Taft et al., \u201cCockroachDB: The Resilient Geo-Distributed SQL Database,\u201d in Proc. 2020 ACM SIGMOD Int. Conf. Management of Data, 2020, pp. 1493\u20131509. \u21a9<\/a><\/p>\n\n\n\n

[22] YugabyteDB, \u201cKey Concepts,\u201d Yugabyte, 2026. [Online]. Available: https:\/\/docs.yugabyte.com\/stable\/architecture\/key-concepts\/<\/a>. [Accessed: Jul. 04, 2026]. \u21a9<\/a><\/p>\n\n\n\n

[23] N. Adly, M. Nagi, and J. Bacon, \u201cPerformance Evaluation of a Hierarchical Replication Protocol: Synchronous versus Asynchronous,\u201d Univ. of Cambridge Computer Laboratory, 1995, pp. 102\u2013108. \u21a9<\/a><\/p>\n\n\n\n

[24] Uber Technologies, \u201cH3: Uber\u2019s Hexagonal Hierarchical Spatial Index,\u201d Uber Blog, 2018. [Online]. Available: https:\/\/www.uber.com\/blog\/h3\/<\/a>. [Accessed: Jul. 06, 2026]. \u21a9<\/a><\/p>\n\n\n\n

[25] Uber Technologies, \u201cH3: A Hexagonal Hierarchical Geospatial Indexing System,\u201d h3geo.org documentation, 2024. [Online]. Available: https:\/\/h3geo.org\/<\/a>. [Accessed: Jul. 06, 2026]. \u21a9<\/a><\/p>\n\n\n\n

[26] Uber Technologies, \u201cDesigning Schemaless, Uber Engineering\u2019s Scalable Datastore Using MySQL,\u201d Uber Blog, 2016. [Online]. Available: https:\/\/www.uber.com\/blog\/schemaless-part-one-mysql-datastore\/<\/a>. [Accessed: Jul. 06, 2026]. \u21a9<\/a><\/p>\n\n\n\n

[27] H. Garcia-Molina and K. Salem, \u201cSagas,\u201d in Proc. 1987 ACM SIGMOD Int. Conf. Management of Data (SIGMOD \u201987), San Francisco, CA, 1987, pp. 249\u2013259. \u21a9<\/a><\/p>\n\n\n\n

[28] R. Kimball and M. Ross, The Data Warehouse Toolkit: The Definitive Guide to Dimensional Modeling, 3rd ed. Indianapolis, IN: Wiley, 2013. \u21a9<\/a><\/p>\n\n\n\n

[29] M. Armbrust, A. Ghodsi, R. Xin, and M. Zaharia, \u201cLakehouse: A New Generation of Open Platforms that Unify Data Warehousing and Advanced Analytics,\u201d in Proc. 11th Conf. Innovative Data Systems Research (CIDR \u201921), 2021. \u21a9<\/a><\/p>\n\n\n\n

[30] F. \u00a8Ozcan, Y. Tian, and P. T\u00a8oz\u00a8un, \u201cHybrid transactional\/analytical processing: A survey,\u201d in Proc. 2017 ACM SIGMOD Int. Conf. Management of Data, 2017, pp. 1771\u20131775. \u21a9<\/a><\/p>\n\n\n\n

[31] K. Cummiskey and S. Pierce, \u201cAzure Databricks Lakebase is Generally Available,\u201d Databricks Blog, 2026. [Online]. Available: https:\/\/www.databricks.com\/blog\/azure-databricks-lakebase-generally-available<\/a>. [Accessed: Jul. 19, 2026]. \u21a9<\/a><\/p>\n\n\n\n

[32] J. Wang et al., \u201cMilvus: A Purpose-Built Vector Data Management System,\u201d in Proc. 2021 ACM SIGMOD Int. Conf. Management of Data, 2021, pp. 2614\u20132627. \u21a9<\/a><\/p>\n\n\n\n

[33] J. J. Pan, J. Wang, and G. Li, \u201cSurvey of vector database management systems,\u201d The VLDB Journal, vol. 33, no. 5, pp. 1591\u20131615, 2024. \u21a9<\/a><\/p>\n\n\n\n

[34] A. Kane, \u201cpgvector: Open-source vector similarity search for Postgres,\u201d GitHub repository, 2024. [Online]. Available: https:\/\/github.com\/pgvector\/pgvector<\/a>. [Accessed: Jul. 13, 2026]. \u21a9<\/a><\/p>\n","protected":false},"excerpt":{"rendered":"

Abstract: The assumption that a single type of data storage can serve every workload no longer holds at ultra-large scale. This paper examines storage in a globally distributed ride-hailing platform, where each workload is mapped to a storage engine whose guarantees match its access pattern, consistency requirement, and query shape. The mapping is justified through […]<\/p>\n","protected":false},"author":1315,"featured_media":0,"comment_status":"open","ping_status":"closed","sticky":false,"template":"","format":"standard","meta":{"_jetpack_newsletter_access":"","_jetpack_dont_email_post_to_subs":false,"_jetpack_newsletter_tier_id":0,"_jetpack_memberships_contains_paywalled_content":false,"_jetpack_feature_clip_id":0,"_jetpack_memberships_contains_paid_content":false,"footnotes":"","jetpack_post_was_ever_published":false},"categories":[1,655,650,21,651,223],"tags":[1240,1242,1241,1247,1244,91,1246,1245,1239,1238,1243,415,1233],"ppma_author":[1200],"class_list":["post-29005","post","type-post","status-publish","format-standard","hentry","category-allgemein","category-design-patterns","category-scalable-systems","category-system-architecture","category-system-designs","category-ultra-large-scale-systems","tag-cap","tag-change-data-capture","tag-distributed-databases","tag-htap","tag-lakehouse","tag-microservices","tag-olap","tag-oltp","tag-pacelc","tag-polyglot-persistance","tag-saga-pattern","tag-sharding","tag-ultra-large-scale-systems-2"],"aioseo_notices":[],"aioseo_head":"\n\t\t\n\t\n\t\n\t\n\t\n\t\n\t\n\t\t\n\t\t\n\t\t\n\t\t\n\t\t\n\t\t\n\t\t\n\t\t\n\t\t\n\t\t\n\t\t\n\t\t\n\t\t\n\t\t\n\t\t\n\t\t\n\t\t