TokuDB has a big advantage over B-trees when trickle loading data into existing tables. However, it is possible to preprocess the data when bulk loading into empty tables or when new indexes are created. TokuDB release 4 now uses a parallel algorithm to speed up these types of bulk insertions. How does the parallel loader performance compare with the serial loader? We use the Air Traffic Control (ATC) data and queries described in a Percona blog and also used in an experiment with TokuDB 2.1.0 to gain some insight.

Our ATC data is about 122M rows in size, is stored in a 40GiB CSV file, and can be found in our Amazon S3 public bucket. See the end of this blog for details. We …

Tokutek is pleased to announce immediate availability of TokuDB for MySQL, version 4.1.1. It is ideally suited for delivering fast response times for complex / high-volume Web applications that must simultaneously store and query large volumes of rapidly arriving data:

• Social Networking
• Real-time clickstream analysis
• Logfile Analysis
• eCommerce Personalization
• High-speed Webcrawling

TokuDB v4.1.1 replaces TokuDB v4.1.0 and is recommended for all users. (We found a bug in v4.1.0 and have withdrawn it from our website). The new version has all of v4.1.0′s new features, including support for SAVEPOINT and an even better Fast Loader. As always, this release uses our high-performance Fractal Tree™ indexing to provide a unique combination of capabilities:

• 10x-50x faster indexing …

Tokutek is pleased to announce immediate availability of TokuDB for MySQL, version 4.1. It is designed for continuous querying and analysis of large volumes of rapidly arriving and changing data, while maintaining full ACID properties.

New in TokuDB v4.1 includes important improvements, most notably support for SAVEPOINT and improved Fast Loader performance (introduced in v4.0).

This new release builds on TokuDB’s unique combination of capabilities:

• 10x-50x faster indexing for faster querying
• Full support for ACID transactions
• Short recovery time (seconds or minutes, not hours or days)
• Immunity to database aging to eliminate performance degradation and maintenance headaches
• 5x-15x data compression for reduced disk use and lower storage costs

Because of its high …

In posts on June 30 and July 6, I explained how implementing the commands “replace into” and “insert ignore” with TokuDB’s fractal trees data structures can be two orders of magnitude faster than implementing them with B-trees. Towards the end of each post, I hinted at that there are some caveats that complicate the story a little. On July 21st I explained one caveat, secondary keys, and on August 3rd, Rich explained another caveat. In this post, I explain the other …

In posts on June 30 and July 6, we explained how implementing the commands “replace into” and “insert ignore” with TokuDB’s fractal trees data structures can be two orders of magnitude faster than implementing them with B-trees. Towards the end of each post, we hinted at that there are some caveats that complicate the story a little. In this post, we explain one of the complications: the calculation of affected rows.

MySQL returns the number of rows affected by a “replace” or “insert” statement to the client. For the “replace” statement, the number of affected rows is defined to be the sum of the number of rows …

Tokutek tests its TokuDB Fractal Tree storage engine with multiple MySQL distributions. We make extensive use of the MySQL Sandbox in our test automation. We tweaked the regular expressions that match binary tarball names in the MySQL Sandbox so that MariaDB releases can be easily loaded by the MySQL Sandbox. These changes can be found in launchpad lp:~prohaska7/mysql-sandbox/mariadb.

In posts on June 30 and July 6, I explained how implementing the commands “replace into” and “insert ignore” with TokuDB’s fractal trees data structures can be two orders of magnitude faster than implementing them with B-trees. Towards the end of each post, I hinted at that there are some caveats that complicate the story a little. In this post, I explain one of the complications: secondary indexes.

Secondary indexes act the same way in TokuDB as they do in InnoDB. They store the defined secondary key, and the primary key as a pointer to the rest of the row. So, say the table foo has the following schema:

create …

In my post on June 18th, I explained why the semantics of normal ad-hoc insertions with a primary key are expensive because they require disk seeks on large data sets. I previously explained why it would be better to use “replace into” or to use “insert ignore” over normal inserts. In this post, I explain why another alternative to normal inserts, “insert … on duplicate key update” is no better in MySQL, because the command incurs disk seeks.

The reason “insert ignore” and “replace into” can be made fast with TokuDB’s fractal trees is that the semantics of what to do in case a duplicate key is found is simple. …

In my post from three weeks ago, I explained why the semantics of normal ad-hoc insertions with a primary key are expensive because they require disk seeks on large data sets. Towards the end of the post, I claimed that it would be better to use “replace into” or “insert ignore” over normal inserts, because the semantics of these statements do NOT require disk seeks. In my post last week, I explained how the command “replace into” can be fast with TokuDB’s fractal trees. Today, I explain how “insert ignore” can be fast, using a strategy that is very similar to what we do with “replace into”.

The semantics …

Tokutek is pleased to announce immediate availability of TokuDB for MySQL, version 4.0. It is designed for continuous querying and analysis of large volumes of rapidly arriving and changing data, while maintaining full ACID properties.

New in TokuDB v4.0 is our multi-threaded Fast Loader. Capable of utilizing all available CPU cores, the Fast Loader greatly accelerates creating a new index or initializing a database with a bulk load. On an 8 core machine, loading operations speed up by over 4x and with 16 cores the acceleration is even greater.

Other improvements in v4.0 include:

• Shorter Logfiles – The logs needed for ACID transactions are now considerably
  smaller. In addition to saving disk space, smaller logs means faster recovery in many cases.
• Faster Logfile Reclamation – When a transaction commits, the …