For those interested In SSD or Wafflegrid I will be presenting on both topics at the 2009 MySQL User Conference!  I want to keep these fresh, so their will be more then just a rehash of my blog here, but their will be some overlap.  Interested in something I have not talked about yet?  Drop me a line!  Always looking for good ideas.

As one of the co-founders of the Waffle Grid project, I beam with pride every time I get a stellar benchmark or every time I find a new use for the Waffle.  But as a professional I still have to be critical of all solutions I would recommend or deploy.  One of the big goals of Waffle Grid is to replace disk IO which should be slow with remote memory which should be much faster.  But what happens when the disk is no longer slow?  This leads me to ask myself, is Waffle Grid only good for servers with slower disk?  Or is this a solution that can also help systems with fast disk?  So which should you deploy SSD -vs- Waffle?  Are they competitors?  Or are they complementary technologies?

I am going to say this, in these tests latency is king.  The faster the drives can deliver data, the higher the benchmarks should be.  Basically if my interconnect can deliver faster then the drive can serve up …

Maybe I should make the sub-tag line, make your MySQL database run up to 300% faster by using Waffle Grid…   Ahhh its all about marketing.

Had an interesting holiday, snuck away several times to benchmark and test out the new Waffle Grid release… nothing says turkey day like Waffles and a really fast database  ( Hey! Yes my wife knew she married a complete and utter nerd, she just might not have understood the depths of my nerdom ).  As I mentioned before with the release of MySQL 5.1.30 I switched over all my testing over to this version, and in my tests this combination is running really well.  Additionally I had a hardware issue which compelled me to retest some of the tests I did last week. So who loves benchmarks!  I DO I DO!!!  I like to go fast, so lets get right down to it.

Ok, Yves pushed up two patches onto launchpad the other Day. I wanted to get a chance to fully test out the patches before blogging about them, and as I mentioned in my previous blog post, I have been dealing with some hardware issues. I spent the day testing out these patches, running them through the paces looking for potential slowdowns or issues, and everything looks good. I am going to limit this post to just the new features, as I am still finalizing a few benchmarks as I write this.

Whats new?

First the patches work against 5.1.30. I have been running all my recent tests against these without any issues so far ( knock on wood ).

Second you will notice 2 patches. There is a patch to memcached which will only work in 1.2.5 right now ( 1.2.6 changes how the memcached LRU queue gets …

I knew something was wrong with my previous numbers, and there is an optimization that should improve it.  So I was trying to test out the performance difference between a Waffle running localhost memcached and one with a remote memcached when I noticed something.  I noticed that the disk IO, specifically the reads per second where higher then when the database was using Waffle then when I was not using a Waffle.  Why this was odd was that both of these tests should have 100% of the data in the buffer pool or in remote memcached, so their should be limited read activity.  Then it hit me, the memcached LRU is going to mess us up, let me explain:

The idea behind Waffle grid is when you hit the internal innodb LRU, put the data that was to be discarded into memcached, so the next time its needed you can retrieve it from memcached instead of disk.

Continuing on with the discussion on Waffle Grid, I have put together a few interesting benchmarks and wanted to expand on some of the possible deployment scenarios using Waffle.

My home laboratory is rather small, so my datasets are limited in size for the moment. To compensate I have shrunk the data sizes and database setting to simulate as best as I can the performance on a larger system. For this first test I decided to go with a 20 Warehouse DBT2 test, which equates to about 3GB of data in the database. In order to get a better feel for the performance benefit of fitting all most of your data in local memory + remote I setup my local machine with 768M allocated to the innodb buffer pool, and setup a remote machine with 768M allocated to memcached. This should scale up at a somewhat similar rate. What I mean by this is the increase seen with a 3GB dataset with 768/768 local/remote split I should also see with a 6GB dataset with a …

Yesterday we announced what the Waffle Grid project is.  Some people ignored it, others I am sure read it with curiosity, but i figure their is a small subset of folks out their who said: “WOW!!!  That is really cool.  I want a Waffle Grid now!”… I wonder if these are the same people who run out and buy the latest SSD harddrive because its fast and the latest nvidia card because they can one up all their friends…  To those who want it now, but do not know where to begin I let me give you a quick install tutorial.  Some of this already exists on the wiki, but hopefully I will make it clearer here.

Get two servers….  I will wait.

A few months ago I was at dinner with Yves Trudeau discussing what all consultants discuss in the late hours after a long day of hard work… how to improve performance and scalability.  I brought up an idea to him to utilize memcached as an L2 cache for innodb.  At first he was skeptical, but as we talked he was more and more intrigued by the idea.  The idea was simple, add a set to memcached when something hit the LRU…  then issue a get from memcached when you do not find the data locally stored in the buffer pool but before you read from disk.  Starting from that point, you can work out any of the issues that would be sure to follow.   So Yves continued emailing me asking me questions…  then he sent me a note that he had made huge progress with the idea. Huge progress means that he wrote version 0.1 and had it working.    That’s when the Idea really turned into a project.

We called …