Once again, I was unable to attend all of the sessions I wanted to at this year's User Converence, but I was happy to make it to Bob Burgess' talk on bash scripting with mysql. The slides and examples aren't up yet, but when they are (which may be as you read this, check the last link), they would probably also be a great tutorial.

So, I got bore\^D\^D\^D\^D inspired later that day to put some of the practices into use, and worked up a script to run mysqlslap in various ways against a server, and then added a couple funcitons to try it out on each storage engine. The script is below in its entirety - bash scripters, please be kind in your comments. No, I didn't write all this just for the pun in the …

Once again, I was unable to attend all of the sessions I wanted to at this year's User Converence, but I was happy to make it to Bob Burgess' talk on bash scripting with mysql. The slides and examples aren't up yet, but when they are (which may be as you read this, check the last link), they would probably also be a great tutorial.

So, I got bore\^D\^D\^D\^D inspired later that day to put some of the practices into use, and worked up a script to run mysqlslap in various ways against a server, and then added a couple funcitons to try it out on each storage engine. The script is below in its entirety - bash scripters, please be kind in your comments. No, I didn't write all this just for the pun in the …

DTrace has been something that I’ve been trying to get into the MySQL server for more than a year now.

After a combination of my own patches and working with Mikael Ronstrom and Alexey Kopytov we finally have a suite of probes in MySQL 6.0.8. Better still, after a short hiatus while I was busy working on a million-and-one other things, the documentation for those probes is now available: Tracing mysqld with DTrace.

The documentation is comparatively light and deep all at the same time. It’s lightweight from the perspective that I’ve added very little detail on the mechanics of DTrace itself, since there is no need to replicate the excellent guides that Sun already provide on the topic. At the same time, I’ve tried to provide at least one (and sometimes two) D script examples for each of the groups of probes in the 6.0.8 release.

Lots of blogs list a bunch of stuff that happened in the year just past, and I have done a year-in-review post before, but in looking back at posts on this blog and elsewhere, what strikes me most is not the big achievements that took place in technology in 2008, but rather the questions that remain unanswered. So much got started in 2008 — I’m really excited to see what happens with it all in 2009!

Cloud Computing

Technically, the various utility or ‘cloud’ computing initiatives started prior to 2008, but in my observation, they gained more traction in 2008 than at any other time. At the beginning of 2008, I was using Amazon’s S3, and testing to expand into more wide use of EC2 during my time as Technology Director for AddThis.com (pre-buyout). I was also investigating tons of other technologies that take different approaches to the higher-level problem these things all try to solve: owning, and housing (and …

My MythTV store lives on an LVM volume that is spread over 2 disks, one of them is an external USB disk. So the cleaninglady seems to have touched a cable and after coming back from holiday I had a read-only filesystem that afer a remount had about 350Gb in lost+found with irrelevant filenames.

1. total 337407844
2. drwx------ 2 tv tv 4096 Dec 17 22:47 .
3. drwxrwxrwx 15 tv tv 4096 Dec 17 22:44 ..
4. -rw-r--r-- 1 root root 423343556 Dec 14 07:10 I303109.RCN
5. -rw-r--r-- 1 root root 2990538924 Dec 13 19:05 I303107.RCN
6. -rw-r--r-- 1 root root 1023691768 Dec 13 08:10 I319494.RCN
7. -rw-r--r-- 1 root root 1023622348 Dec 13 07:45 I327684.RCN
8. -rw-r--r-- 1 root root 423735892 Dec 13 07:10 I327682.RCN
9. -rw-r--r-- 1 root root 466749476 Dec 12 15:43 I135169.RCN
10. -rw-r--r-- 1 root root 1023314212 Dec 12 07:45 I098309.RCN
11. -rw-r--r-- 1 root root …

One of the best ways to keep up with your field and network at the same time is to attend conferences. It’s one of the things I look forward to every year. After learning that O’Reilly has decided to commit blasphemy and *not* hold OSCON in Portland, Oregon the same week as the Oregon Brewers Festival, I was inspired to look around at what other conferences I might attend in 2009. Turns out, this is a huge pain in the ass, because I can’t find a single, central place that lists all of the conferences I’m likely to be interested in.

So… I created a public Google Calendar. It’s called “US Technical Conferences”. It needs more conferences, but I’ve listed the interesting ones I found. In order to keep the calendar from getting overwhelmingly crowded, I’ve decided that conferences on the list should:

• Deal with open source technology in some way. This is purposely broad.
• Be at least 3 days in …

A compilation of news of interest:

Note that I’m talking about using these tools in some kind of professional way, and more specifically, I’m talking about using Excel as a database, and using VPS hosting to host “professional” web sites. By “professional”, I mean something other than your personal blog, picture gallery, or other relatively inconsequential site.

Excel is not a database

Here’s the thing: Excel isn’t a database. Most people who don’t work in IT don’t seem to understand this, and they’re deathly afraid to actually communicate with anyone in IT, so they take matters into their own hands, and create problems so big that IT is forced to get involved, because at some point this spreadsheet becomes “critical” to some business function. Then IT gets even more bitter toward the non-IT folk, validating some of the reasons the non-IT folk went that route in the first place, and virtually guaranteeing that they won’t come to the IT …

I’ve been doing a little reporting project, and I’ve been searching around for quite some time for a good graphing and charting solution for general-purpose use. I had come across ReportLab before, but it just looked so huge and convoluted to me, given the simplicity of what I wanted at the time, that I moved on. This time was different.

This time I needed a lot of the capabilities of ReportLab. I needed to generate PDFs (this is not a web-based project), I needed to generate charts, and I wanted the reports I was generating to contain various types of text objects in addition to the charts and such.

I took the cliff-dive into the depths of the ReportLab documentation. I discovered three things:

1. There is quite a lot of documentation
2. ReportLab is quite a capable library
3. The documentation actually defies the simplicity of the library.

It’s a decent bit easier than it …

So, I recently ran into one of those situations where a customer complains that his MySQL database is slow, and “it worked great until about two weeks ago”. The first thing I look at in these situations is not the queries or any code, but the indexes. Once I determined that the indexes were almost nonsensical, I had a look at the queries themselves, which proved that the indexes were, in fact, nonsensical. Running the queries as written in the code, from a mysql shell, with EXPLAIN, I was able to further confirm that the indexes (most of them, anyway) were never used.

Easy right? Just reindex the table!

NOTE: I’m going to skip the talk about all of the database design issues on this gig and just go straight to solving the immediate problem at hand. Just know that I had nothing to do with the design.

But, supposing this table has 15 million rows and is running on a machine with 2GB of RAM and only …