Recently, we’ve seen a few people ask us about building TokuMX from scratch. While it’s best if you just use the binaries you can get from us (they have all the right optimizations, we’ve tested them, and we can interpret coredumps they generate), we recognize there are other reasons you might need to do a custom build.

Since we actually build six distinct products all using the Fractal Tree indexing® library (community and enterprise versions of TokuDB for MySQL, TokuDB for MariaDB, and TokuMX), our build process is pretty complicated, compared to software packages that might, for example, just involve one source repository and link against a few standard libraries. Our TokuMX builds involve four git repositories, three separate build stages, two different build tools, and …

On Wednesday night, the Boston MongoDB User group was kind enough to have me speak about TokuMX Internals. I spoke about Fractal Tree® indexes and the technical reasons behind the benefits they provide to MongoDB applications. Although the talk mostly references TokuMX and MongoDB, all the theory applies to TokuDB and MySQL as well.

My slides are on our technology overview page, along with other great content.

Opportunities to present technical material to an engaged audience asking tough questions is rare, and much appreciated. So thank you to the Boston MongoDB User group for having …

A lot is said about the differences in the data between MySQL and MongoDB. Things such as “MongoDB is document based”, “MySQL is relational”, “InnoDB has a clustering key”, etc.. Some may wonder how TokuDB, our MySQL storage engine, and TokuMX, our MongoDB product, fit in with these data layouts. I could not find anything describing the differences with a simple google search, so I figured I’d write a post explaining how things compare.

So who are the players here? With MySQL, users are likely familiar with two storage engines: MyISAM, the original default up until …

Before creating a unique index in TokuMX or TokuDB, ask yourself, “does my application really depend on the database enforcing uniqueness of this key?” If the answer is ANYTHING other than yes, do not declare the index to be unique. Why? Because unique indexes may kill your write performance. In this post, I’ll explain why.

Unique indexes are a strange beast: they have no impact on standard databases that use B-Trees, such as MongoDB and MySQL, but may be horribly painful for databases that use write optimized data structures, like TokuMX’s Fractal Tree(R) indexes. How? …

In my last post, I showed what a Fractal Tree® index is at a high level. Once again, the Fractal Tree index is the data structure inside TokuMX and TokuDB, our MongoDB and MySQL products. One of its strengths is the ability to get high levels of compression on the stored data. In this post, I’ll explain why that is.

At a high level, one can argue that there isn’t anything special about our compression algorithms. We basically do this: we take large chunks of data, use known compression methods (e.g. zlib, lzma, …

CCM Benchmark is one of the leading forum provider on the web,  ROI is a major concern for them  and historically MyISAM was used on the forum replication cluster.  Reason is that MyISAM gave better ROI/performance on data that is hardly electable to cache mechanism.

This post is for MySQL users at scale,  if the number of servers or datacenter cost is not an issue for you, better get some more memory or flash storage and ou will found Lucifer server to demonstrate that your investment is not a lost of money or just migrate to Mongo.  

Quoting Damien Mangin, CTO at CCM "I like my data to be small, who want's to get to a post where the question is not popular and have no answer. Despite cleaning we still get more data than what commodity hardware memory can offer and storing all post in memory would be a major waste of money".

Like many other big web players at an other scale, …

With our recent release of TokuMX 1.0, we’ve made some bold claims about how fast TokuMX can run MongoDB workloads. In this post, I want to dig into one of the big areas of improvement, write performance and reduced I/O.

One of the innovations of TokuMX is that it eliminates a long-held rule of databases: to get good write performance, the working set of your indexes should fit in memory. The standard reasoning goes along the lines of: if your indexes’ working set does not fit in memory, then your writes will induce I/O, you will become I/O bound, and performance will suffer. So, either make sure your indexes fit in …

As promised, the Enterprise Edition of TokuDB®, Version 7, is ready. TokuDB Version 7, Enterprise Edition, introduces Hot Backup. You can now back up all your TokuDB tables directly from MySQL or MariaDB, with no down time. In addition, TokuDB Enterprise Edition comes with a support package.

TokuDB v7 Enterprise Edition maintains all our established advantages: hot schema changes, excellent compression, fast trickle load, fast bulk load, fast range queries through clustering indexes, no fragmentation, and full MySQL/MariaDB compatibility for ease of installation.

For details on pricing and supported MySQL and MariaDB versions, please see our FAQ.

To learn more about TokuDB:

• Download executables here.
• The source code for the Community Edition …

Shard-Query examines INFORMATION_SCHEMA.PARTITIONS to determine if a table is partitioned.  When a table is partitioned, Shard-Query creates multiple background queries, each limited to a single partition.  Unfortunately, it seems that examining INFORMATION_SCHEMA.PARTITIONS causes query plans to change after the view is accessed.

I have reported bug 69179 to MySQL AB  Oracle Corporation(old habits die hard).

Be careful: If you have automated tools (like schema management GUI tools) then make sure they don’t examine INFORMATION_SCHEMA.PARTITIONS or you may get bad plans until you analyze your tables or restart the database, even if using persistent stats.

I can only get the bug to happen when a WHERE clause is issued that limits access to a single partition.  It may be that the per partition statistics …

Following are benchmark results comparing Tokutek TokuDB and Percona XtraDB at scale factor 10 on the Star Schema benchmark. I’m posting this on the Shard-Query blog because I am going to compare the performance of Shard-Query on the benchmark on these two engines. First, however, I think it is important to see how they perform in isolation without concurrency.

Because I am going to be testing Shard-Query, I have chosen to partition the “fact” table (lineorder) by month. I’ve attached the full DDL at the end of the post as well as the queries again for reference.

I want to note a few things about the results:
First and foremost, TokuDB was configured to use quicklz compression (the default) and InnoDB compression was not used. No tuning of TokuDB was performed, which means it will use up to 50% of memory by default. Various InnoDB tuning options were set (see the end of the post) but the most important is that the …