Today, I was reminded of a Jim Starkey quote on the Random Query Generator:

“The Colonoscopy of Database Software”
– Jim Starkey

If your project does not have something that you can adapt that quote to, odds are your testing is inadequate.

In the MySQL QA teams in Oracle we have been using a tool called the Random Query Generator (or "RQG" for short) for some time now. The main RQG testing effort has been on new server development, including regression testing. The recent MySQL 5.6 GA is a result of a couple of years of hard work, including a lot of RQG testing and bugfixing as a result of that.

I can easily say that the RQG has helped making MySQL 5.6 a better release than it would otherwise be. It is of course not our only testing tool, but there are plenty of bugs this tool has uncovered that would likely not have been seen in our other testing. Such issues include:

• crashes and asserts
• memory management bugs (valgrind errors)
• wrong results

The main clue to the power of the RQG is its …

One of the tools we use to qualify MySQL Server is the Random Query Generator (RQG for short). This is an SQL generator (and more) to test database systems such as MySQL. I plan to post more on that later.

The RQG is an open source tool written in Perl, and is available on Launchpad. Up until now the RQG documentation has been part of the MySQL Forge wiki. As you may have seen from this blog post, MySQL Forge is going away soon (August 1st). Unlike other parts of the Forge wiki, the Random Query Generator documentation has moved to github, and will continue to be maintained by the RQG community:

…

So while I was at the MySQL UC, The Xeround database came to my attention.  It bills itself as database as a service for MySQL systems and a seamless replacement for standard MySQL.

Of course, since I am a QA Engineer, I could not resist the urge to try to break it >:)  As my friend and former MySQL colleage, Kostja says, “QA Engineers are a unique breed…they like to push all the buttons” : )  I would say that the QA mindset goes a bit further than that, but it is something I will delve into in another post.  I will only say that there is a reason that Microsoft recognizes QA software engineering as a distinct and specialized discipline.

So, let’s get back to Xeround.  It was the first database as a service that caught my eye and I just had to test it!  They are currently …

My presentation from the MySQL UC didn’t give a lot of detail on the actual tool I have hacked up, nor did it go into how to play with it / try it out.  I figured I should rectify that (at least one person seemed interested in trying it out <g>)

To begin with, you should have the random query generator installed (see the docs for handling that).  Besides being *the* cutting edge, production-ready testing tool in the open-source dbms world, it comes with a handy data generator.

One of the key features of kewpie, is that it can easily generate test queries against any test bed.  A standard randgen practice is to develop grammars and gendata files (which generates a user-specified …

So, back when we released our first beta in September, one of the many responses was this

The comments about the reliability / durability of the log definitely struck me as testing we needed.

It’s taken a while (we had this GA thing we were working on…), but we finally have crash and recover testing of the innodb transaction log and the slave plugin.

Here is what happens for the innodb-based log:

• Set up the test servers and start the randgen with the trx_log grammars.  I’ll point you …