Last week I wrote about our 10x insertion performance increase with MongoDB. We’ve continued our experimental integration of Fractal Tree® Indexes into MongoDB, adding support for clustered indexes.  A clustered index stores all non-index fields as the “value” portion of the index, as opposed to a standard MongoDB index that stores a pointer to the document data.  The benefit is that indexed lookups can immediately return any requested values instead of needing to do an additional lookup (and potential disk IOs) for the requested fields.

To create a clustered index you just need to add “clustering:true” as in the following example (note that version 2 indexes are Fractal Tree Indexes): …

The challenge of handling massive data processing workloads has spawned many new innovations and techniques in the database world, from indexing innovations like our Fractal Tree® technology to a myriad of “NoSQL” solutions (here is our Chief Scientist’s perspective). Among the most popular and widely adopted NoSQL solutions is MongoDB and we became curious if our Fractal Tree indexing could offer some advantage when combined with it. The answer seems to be a strong “yes”.

Earlier in the summer we kicked off a small side project and here’s what we did: we implemented a “version 2” IndexInterface as a Fractal Tree index and ran some benchmarks. Note that our integration only affects MongoDB’s secondary indexes; primary indexes continue to rely on MongoDB’s indexing code. All the changes we made to the MongoDB source …

Businesses increasingly operate in a 24×7 environment, where complex analytics must be performed on live, continuously incoming “Big Data.” To address this, TokuDB has developed Fractal Tree®  technology, a revolutionary new indexing capability that enables SQL databases running advanced web applications to grow from gigabytes to terabytes while improving insert speed, query performance, compression, and enabling zero-downtime schema changes.

Date: September 5th
Time: 2 PM EST / 11 AM PST

REGISTER TODAY

TokuDB is used by MySQL and MariaDB customers worldwide to increase their database performance by 20x-80x on Big Data applications that conventional RDBMS’s cannot handle. Instead of waiting hours or even days to run queries or …

Next week I (Bradley) will be traveling to FROSCON near Bonn, Germany, and then on to VLDB in Istanbul.

At FROSCON I’ll be talking about fast data structures for maintaining indexes. The talk will share some content with my upcoming MySQL Connect talk.

At VLDB, Dzejla Medjedovic will be presenting a talk on our paper on SSD-friendly Bloom-filter-like data structures. The paper is

Michael A. Bender, Martin Farach-Colton, Rob Johnson, Russell Kraner, Bradley C. Kuszmaul, Dzejla Medjedovic, Pablo Montes, Pradeep Shetty, Richard P. Spillane, and Erez Zadok.
Don’t Thrash: How to Cache Your Hash on Flash. PVLDB 5(11):1627-1637, 2012.

An earlier version of the paper appeared at …

A few weeks ago Bradley Kuszmaul and I attended the Dagstuhl Seminar on Database Workload Management.

The Dagstuhl computer science research center is (remotely) located in the countryside in Saarland, Germany. The actual building is an 18th Century Manor House, first retooled as an old-age home, and then a computer science research center. Workshop participants typically spend the whole week talking and working together.

Dagstuhl Computer Science Center

Shivnath Babu (Duke University), Goetz Graefe (Hewlett Packard), and Harumi Kuno (Hewlett Packard) did a great job organizing. …

Benchmarking is a tricky thing, especially when it comes to compression. Some data compresses quite well while other data does not compress at all. Storing jpeg images in a BLOB column produces 0% compression, but storing the string “AAAAAAAAAAAAAAAAAAAA” in a VARCHAR(20) column produces extremely high (and unrealistic) compression numbers.

This week I was assisting a TokuDB customer understand the insertion performance of TokuDB versus InnoDB and MyISAM for their actual data. The table contained a single VARCHAR(50), multiple INTEGER, one SET, one DECIMAL, and a surrogate primary key.  To support a varied query workload they needed 6 indexes.

Here is an obfuscated schema of the table:

col1 varchar(50) NOT NULL,
col2 int(40) NOT NULL DEFAULT '0',
col3 int(10) NOT NULL DEFAULT '0',
col4 int(10) NOT NULL DEFAULT '0',
col5 int(10) NOT NULL DEFAULT '0',
col6 set('val1', 'val2', ..., ‘val19’, 'val20',) NOT NULL DEFAULT …

I’ll be presenting TokuDB at the Emerging Business Technology Meetup in Boston this Thursday night at 6pm at 290 Congress St (near South Station).  The meeting topic is “NewSQL: SQL Technologies Keeping NoSQL Promises” and also includes presentations from Akiban and VoltDB.  Each of us will be highlighting our technology and use-cases, with a panel discussion and open Q&A to follow.

This is a well attended group with 102 currently registered for the meeting and room for 48 more.  If you want to come and haven’t yet signed up you can RSVP …

Three rules on making indexes around queries to provide good performance

Application performance often depends on how fast a query can respond and query performance almost always depends on good indexing. So one of the quickest and least expensive ways to increase application performance is to optimize the indexes. This talk presents three simple and effective rules on how to construct indexes around queries that result in good performance.

Time: 2PM EDT / 11AM PDT

This webinar is a general discussion applicable to all databases using indexes and is not specific to any particular MySQL® storage engine (e.g., InnoDB, TokuDB®, etc.). The rules are explained using a simple model that does NOT rely on understanding B-trees, Fractal Tree® indexing, …

TokuDB v6.1 is now generally available and can be downloaded here.

New features include:

• Added support for MariaDB 5.5 (5.5.25)
  • The TokuDB storage engine is now available with all the additional functionality of MariaDB 5.5.
• Added HCAD support to our MySQL 5.5 version (5.5.24)
  • Hot column addition/deletion was present in TokuDB v6.0 for MySQL 5.1 and MariaDB 5.2, but not in MySQL 5.5.  This feature is now present in all MySQL and MariaDB versions of TokuDB.
• Improved in-memory point query performance via lock/latch refinement
  • TokuDB has always been a great performer on range scans and workloads where the size of the working data set is significantly larger than RAM.  TokuDB v6.0 improved the …

We found a simple XA transaction that crashes MySQL 5.5 replication. This simple transaction inserts a row into an InnoDB table and a TokuDB table. The bug was caused by a flaw in the logging code exposed by the transaction’s use of two XA storage engines (TokuDB and InnoDB). This bug was fixed in the TokuDB 6.0.1 release.

Here are some details.  Suppose that a database contains the following tables.

create table t1 (a int) engine=InnoDB
create table t2 (a int) engine=TokuDB

 The following transaction

begin
insert into t1 values (1)
insert into t2 values (2)
commit

causes the replication slave to crash.

The crash occurs when mysqld tries to dereference a NULL pointer.

#4  0x000000000088e203 in MYSQL_BIN_LOG::log_and_order (this=0x14b8640, thd=0x7f7758000af0, xid=161, all=true, need_prepare_ordered=false, need_commit_ordered=true) at …