Here are some of the fake O'Reilly book covers I mentioned in a
prior post. These have been optimized for use as black
& white Kindle screensaver wallpaper images. If you
haven't done so already, you can install a Kindle screensaver hack with a couple of
downloads.
Update: I've embedded a slideshow from PicasaWeb, but it requires Flash. If you don't see it you can click on the links below to go directly to PicasaWeb.
I’m researching algorithms for automatic fault detection in MySQL (see my previous post for context). I need real-world data samples to test the algorithm. Can you help by sending me a bit of data from your production server?
The end goal is an open-source tool that will be a standard part of a typical MySQL installation. The problem I’m trying to solve for all MySQL users is this: something went wrong, what was it? Most of the time there’s no way to answer that; you have to set up a set of tools and hope you capture enough information to diagnose the problem next time. We need a tool that just runs all the time even when you don’t think anything is going to go wrong.
You can help build this tool. I need samples from a wide variety of healthy and sick servers, both heavily and lightly loaded. I need samples …
[Read more]I was curious what information MariaDB’s “phone home” user feedback plugin sends. (It works on more than just MariaDB, by the way.)
It’s easy enough to find out: just load the plugin, then select from the INFORMATION_SCHEMA.FEEDBACK table. This returns a lot of rows that are obviously the status counters and variables, as well as the plugins loaded in the server. A quick exclusion join will eliminate those, and the result on my laptop is this:
select f.* from feedback as f
left outer join global_variables as v on f.variable_name = v.variable_name
left outer join global_status as s on f.variable_name = s.variable_name
left outer join plugins as p on f.variable_name = p.plugin_name
where s.variable_name is null and v.variable_name is null and p.plugin_name is null;
+--------------------+--------------------------------------+
| …Over the course of years, I have observed that the three most sensitive indicators of MySQL having a server lockup are the queries per second, number of connections, and number of queries running. Here is a chart of those three on a production system. Find the bad spot:
I am currently working on developing an automated system that detects abnormal behavior in these three metrics, but doesn’t require any a priori inputs or thresholds, e.g. you don’t have to tell it “more than X is bad.” (It could be that during a low period of the day, X is different than during the peak load.)
It turns out that this is hard to do reliably, without a lot of false positives and without false negatives (not triggering during an incident). If there is existing literature on the mathematical techniques to do this, I’d be …
[Read more]
Are you a MySQL DBA? Checkout the common_schema project by Oracle Ace
Shlomi
Noach.
The common_schema is an open source MySQL schema
that packs a number of utility views, functions and stored
procedures. You can use these utilities to simplify MySQL
database administration and development. Shlomi just released revision 178, and I'm happy and
proud to be working together with Shlomi on this project.
Among the many cool features created by Shlomi, such as
foreach, repeat_exec and
exec_file, there are a few
%_dependencies procedures I contributed:
At ideeli, we use a variety of techniques to eliminate single points of failure and contention. These involve load balancing multiple application servers, caching layers, and calls to asynchronous services. However, at the end of the day we have a single master MySQL database. It doesn’t matter how many dozens of application servers are handling requests if the database is stalled. Without the right tools, it can be difficult to determine exactly what the database is doing at any given time. In this two part post, I will discuss some techniques for finding those problematic queries.
FINDING THE SLOW STUFF
The MySQL slow query log should be the first stop when looking for queries to optimize. Even with tools like mysqldumpslow, the format of the …
[Read more]The pt-query-digest tool has a number of special cases that affect how it “fingerprints” queries when it groups similar queries together to produce an aggregated report over the group. One of these is a special rule for queries that appear to come from mysqldump, of the following form:
SELECT /*!40001 SQL_NO_CACHE */ * FROM `users`
All such queries will be fingerprinted together and presented in a single class of queries. I remember many instances where mysqldump queries crowded the report of the “most important” queries and just caused other queries to be excluded. Grouping them together made it obvious that mysqldump’s load on the server was a problem, but didn’t obliterate other interesting things we wanted to see in the report.
Further Reading:
This recent session at OSDC 2011 Canberra is based on part of an Open Query training day, and (due to time constraints) without much of the usual interactivity, exercises and further MySQL specific detail. People liked it anyway, which is nice! The info as presented is not MySQL specific, it provides general insight in how databases implement concurrency and what trade-offs they make.
See http://2011.osdc.com.au/SQLL for the talk abstract.
The third edition is nearly done. I’ve committed first drafts of all chapters, and all but one appendix. I need to do the last appendix and then rewrite the preface, which is a few days of work at my current pace. After that, it’s the usual tech review, copyediting, updates to figures, etc — and then it’s off to production.
I’m really pleased with this edition. I was planning on it just being a refresh of the second edition to reflect what’s new in MySQL-land, but it’s almost a complete rewrite. There’s a lot more focus on a logical approach throughout: what happens in the server, what are the limitations, why this matters, what are the practical consequences and applications, and therefore…. The “and therefore” is the real reason you buy a book such as this one.
Further Reading:
One of the things I think we did right with Maatkit (and now with Percona Toolkit) is making the documentation part of the code itself. So much redundancy and wrong documentation has been eliminated by making the tool actually read its own documentation when it starts up. As an example, the default value of the –shorten option is defined in the documentation (it’s Perldoc) like this:
=item --shorten
type: int; default: 1024
Not only is the documentation part of the code, but the tool’s –help output is generated from it too. The existence, type, defaults, and even the behavior of the command-line options is defined in the documentation. If I execute the tool with the –help option, you can see that default value:
[baron@ginger bin]$ ./pt-query-digest --help | grep -- --shorten
--shorten=i Shorten long statements in reports (default
--shorten 1024 …