Bigtable is used by Google for gmail, Google docs and other products. They store 2.5 exabytes of data, and do 600 million operations per second. HBase is used by Facebook for its messaging (1 million ops/s), Bloomberg, Xiaomi and others. Cluster size goes from 15 to 2000 nodes.

I will cover mostly how HBase is available (not as in CAP) and strongly consistent even in case of failure, and the some of the challenges in multi sites deployments. HBase uses replicated Write-Ahead-Logs, failure detectors and replay the WAL in parallel on failure. Failure detectors bring the strong consistency property, but impact negatively the latency (interestingly, Chandra is a coauthor of many papers on failure detectors and also a coauthor of the Bigtable paper at Google). I will introduce some tricks that HBase added to minimize the Mean-Time-To-Recover. HBase, as Bigtable, is eventually consistent between datacenters. This breaks the initial design goal of HBase, and introduces a dataloss risk. Synchronous replication is not that easy: Google initially added another component (Megastore) on top of Bigtable, then developed Spanner. Spanner supports natively synchronous replication. It seems that adding cross site synchronous replication has been tried at all the possible layers (disk, file system, distributed file system, distributed database, 'meta database', application). Is there a best-of-class approach that one should just copy? Here I will just present the existing asynchronous mechanism in HBase.