This is mostly a reflection on a limitation in InnoDB that, in my
opinion, has persisted for too long. I founded while reviewing
the Amarok media
player. The player uses MySQL in the backend, embedded or
regular server, so it makes for a great source of real
life data.
The IssueBy default, Amarok uses MyISAM tables. This means that
if it crashes or stops unexpectedly (a logout while playing music
may cause this), the latest updates to the DB are all lost. So
I've been looking into using InnoDB instead to avoid loosing my
playlists or player statistics.
The ProblemThe limitation that bothers me is this one: "Index key prefixes can be up to 767 bytes"
which has been in place for several years.
Take this Amarok table for example:
CREATE TABLE urls (
…
While working on a deployment we came across a nasty surprise. In
hindsight it was avoidable, but it never crossed our minds it
could happen. I'll share the experience so when you face a
similar situation, you'll know what to expect.
ScenarioTo deploy the changes, we used a pair of servers
configured to replicate with each other (master-master
replication). There are many articles that describe how to
perform an ALTER TABLE with minimum or no downtime using MySQL
replication. The simple explanation is:
- Set up a passive master of the database you want to modify the schema.
- Run the schema updates on the passive master.
- Let replication to catch up once the schema modifications are done.
- Promote the passive master as the new active master.
The details to make this work will depend on each individual situation and are too extensive for the purpose of this article. A …
[Read more]Why should you convert myisam tables to innodb ?
For the perfectly simple reason that innodb tables do not get locked by concurrent selects & inserts. So if you find that your myisam table is suffering for too many locks - due to concurrent selects and inserts, it is time for you to covert the table to innodb.
The simple query which does the trick is
Alter table myisam_table_name engine =
You have a huge mysql table - maybe 100 GB. And you need to run alter on it - to either add an index, drop an index, add a column or drop a column. If you run the simple mysql "alter table" command, you will end up spending ages to bring the table back into production.
Here is a simple hack to get the thing done. The benefit of the hack is that the alter runs quite fast. But since this is a hack
I recently granted ALTER
access in MySQL so a user
could run the ALTER TABLE
command . However after I
granted the necessary privileges, the user was still not able to
perform the tasks needed. Reproducing the issue using a test
instance, I granted a test user the required privileges and MySQL
reported no errors or warnings when the ALTER TABLE
was run:
Welcome to the MySQL monitor. Commands end with ; or \g. Your MySQL connection id is 15 Server version: 5.1.41-log MySQL Community Server (GPL) Type 'help;' or '\h' for help. Type '\c' to clear the buffer. mysql> grant alter,create,insert on *.* to 'test'@localhost; Query OK, 0 rows affected (0.00 sec) mysql> show warnings; Empty set (0.00 sec) mysql> show errors; Empty set (0.00 sec) mysql>
The reason I granted the addition CREATE
and
INSERT
privileges is that according to the MySQL
documentation ( …
I like the 5.4 developments, overall. It has useful stuff and is being developed and released a reasonable pace. Good progress. While perusing the MySQL 5.4.4 changelog, one particular change drew my attention, since it’s been (re)appearing since 2006. It’s the removal of the TYPE= keyword which was obsoleted since MySQL 4.1 in favour of the ENGINE= syntax in CREATE/ALTER TABLE.
While on the surface it may seem ok to remove the obsolete keyword, there are quite a few apps out there that use it, and that cannot be changed. So these will now be unable to use MySQL 5.4 or beyond. I filed this as a bug in 2006, MySQL bug#17501. If you’re interested in the “history of reappearance”, take a peek at the comments and their timeline. I just put in a new comment to note the 5.4.4 change. …
[Read more]
The other day, a client mentioned they were getting strange
results when running ALTER TABLE
. The episode
involved modifying an existing primary key to add an
auto_increment
primary key
: it was
“shifting” values. Say what?!
As it turns out, it was a very special value getting changed:
zero
. Some fiddling revealed the underlying reason.
Care to join me?
To understand what’s going on, follow the example below as we start with an empty database, create a table and insert a few rows:
mysql> use test; Database changed mysql> show tables; Empty set (0.00 sec) mysql> create table test_table (id int not null primary key) engine=innodb; Query OK, 0 rows affected (0.01 sec) mysql> desc test_table; +-------+---------+------+-----+---------+-------+ | Field | Type | Null | Key | Default | Extra | +-------+---------+------+-----+---------+-------+ | id | …[Read more]
A feature which hasn't been so public about the
implementation
of partitioning is the support for atomicity of many ALTER
TABLE
statements using partitioned tables.
This atomicity exists for
ALTER TABLE ADD PARTITION ....
ALTER TABLE REORGANIZE PARTITION ...
ALTER TABLE DROP PARTITION ...
ALTER TABLE COALESCE PARTITION
Given that partitioning often works with very large tables
it
was desirable to have a higher level of security for ALTER
TABLE
of partitioned tables. To support this a DDL log was
implemented.
This DDL log will in future versions be used also for many
other
meta data statements. The DDL log will record all files
added,
renamed and dropped during an ALTER TABLE command as above.
The design is done in such a way that the ALTER TABLE will
either
fail and then all temporary files will be removed (even in …
In two words: online operations. In a paragraph: Forget partitioning, row-based replication and events. The big reasons most people are going to salivate over 5.1, and probably start plans to upgrade now, are the online operations:
- online ALTER TABLE for column rename, column default value change, and adding values to the end of an ENUM/SET
- Online, table-based logging. No more need to restart your server to enable or change the general or slow query logs. You can have the standard file-based output or choose a table format…which you can query.
Over the past few years of dealing with mysql in larger environments, one thing I’ve always felt that ALTER TABLE statements are flaky with replication, but could never really prove it. I never had a chance to dig into some of strange things I saw, and would tend to overlook and fix them
While working for a client, I encountered an issue that I could no longer ignore. In order to reload a small table from a master database to a number of slaves, I simply piped the output of mysqldump into the master and expected everything to flow into the slaves.
The bug is very specific, and probably not very common. If you send a statement like below, as mysqldump automatically adds, to your master:
/*!40000 ALTER TABLE table_name DISABLE KEYS */;
and have configured something like
replicate-rewrite-db=mydb->mydb_slave
on …
[Read more]