Tungsten Replicator has offered shard-based
parallel replication to slaves since late 2010. The initial
implementation uses in-memory queues. Working purely in
memory keeps latency low and throughput high. On the other
hand, working in memory consumes valuable RAM. It also
forces us to buffer all in-flight transactions and therefore
greatly limits the span of time permissible between the slowest
and fastest shard.
Hence our newest improvement: on-disk parallel queues.
In this article I will cover how parallel replication works
in general, how on-disk queues help with parallel replication,
and then show how to set up from the latest builds.
First, let's review the basic mechanics. Parallel …
Jul
03
2011