TokuDB’s loader uses the available multicore computing resources of the machine to presort and insert the data. In the last couple of posts (here and here), Rich and Dave presented performance results of TokuDB’s loader. Comparing load times with TokuDB 2.1.0, Rich found a 2.1x speedup on a 2 core machine, and a 4.2x speedup on an 8 core machine. Comparing load times with TokuDB 3.1, Dave found an 8.2x speedup on Amazon Web Services c1.large node with 8 cores while loading a table with 256 byte rows.
This leads to these natural questions: how does one use the TokuDB loader? Under what scenarios is it used?
The loader has two purposes:
-
- to ease migration of data from other sources (e.g. …