Just a quick post that I've moved The FederatedX Storage Engine for MySQL to Launchpad from my Mercurial repository. I'm trying like anything to get a spare moment to work on it-- writing a book takes every spare moment I have. I have made it group accessible and am working with Antony Curtis (who with Arjen encouraged me to get this on LP) on it. So, what I need to do is start a 5.1 compile (it has a problem with latest 5.1) and then give it a test run to see what the problem is. I want to keep this project moving!

The Launchpad page for it is:

https://launchpad.net/federatedx

In a previous post, I was trying to figure out the most optimal way to switch from two large innodb table space files to using innodb_file_per_table to take advantage of some of the benefits of this setting. I had one part of it solved, which was to stop MySQL, add innodb_file_per_table to the my.cnf, then restart, perform a "no-op" alter of "ALTER TABLE t1 ENGINE=InnoDB" which would cause the table to be re-created an it's own .ibd file. The remaining problem was how to be able to resize the huge table space files after converting all the tables to a smaller size (in my case from 10GB to 10MB).

Someone suggested a better way:

1. Alter all innodb tables to MyISAM
2. Stop the server
3. Add innodb_file_per_table to my.cnf
4. Change innodb_data_file_path to new settings (10MB tablespaces) in my.cnf
5. Move all innodb files (logs, data) to a backup directory
6. Restart MySQL
7. Alter …

I have recently decided I would like to take advantage of the benefits of InnoDB file-per-table. Namely, that OPTIMIZE TABLE benefits-- defragmentation (Grazr does a lot of writes, both INSERT and DELETE), .ibd files being able to shrink if optimized, etc.

So, the next question is how to convert a ton of data to this setup. What I have tried is this:

1. Add innodb_file_per_table to my.cnf
2. Restart MySQL
3. Alter all InnoDB tables engine=InnoDB

This works great... except, my two 10GB tablespace files remain. I would like to make them smaller, but there is no way of doing this.

Also, there is no trick where you back up the newly created .ibd tables after the alter, move the large table space files to a backup dir, restart and pray that the new smaller tablespace files magically work with your .ibd files. That does *not* work ;)

So, dump all data and restore …

Before today, Drizzle was known as a light form of rain found in Seattle (among other places). Not any more. If you have not read the news already today, Drizzle, Clouds, “What If?” is the new kid on the RDBMS bock.

Faster, leaner and designed with the original goals of ease-of-use, reliability and performance, Drizzle will make an impact in those organizations that are seeking a viable database storage solution for large scalable applications. The key to Drizzle is several fold. First, the crud has been removed. The first part of Drizzle development is to remove bloat or non functioning software from the MySQL tree. In fact if you monitor the commits, it reads like, this has been removed, these files have been deleted, this code has been refactored, this new library has been introduced. Design decisions that have limited MySQL’s development for years are being …

Well at least your MySQL database world.

As Paul eluded to, PrimeBase Technologies has a project to provide synchronous replication for MySQL in a High Availability environment. It is more then an idea, there is a plan.

Is it possible?
What are the use cases?
How can you use it?
Would you use it?

Some input to date. We need these questions and more, and we seeking more input for discussion.

Unfortunately the opportunity to hear any input during a presentation is left to the last day of the conference, so we have created a BoF session on Tuesday night for a round table discussion if necessary. We encourage people to bring specific cases and situations for feedback, the reasons why MySQL Replication, MySQL Cluster, DRBD/HeatBeat or any other solution does not satisfy your needs, and what …

I’ll be following closely the progression of Storage Engines available in the MySQL Database server, well soon to be available when 5.1 gets to GA (hopefully by end of Q2 which is what we have been told). Tick, Tick, time is running out.

PrimeBase XT (PBXT) and Blob Streaming is obviously my clear focus, actually now working for PrimeBase Technologies, the company which I want to note for people is an Open Source company, committed at providing an open source alternative to the other commercial players. You also have at the MySQL Conference talks on the the existing InnoDB from Innobase (a subsidiary of market RDBMS leader Oracle). There is a …

The initial release of the FederatedX Pluggable Storage Engine for MySQL is now
available. I developed the Federated Storage Engine when I worked at MySQL,
and really saw a lot of potential with it. However, there were many other
projects that I had to give attention to, and many features and bug fixes that
users wanted didn't come into fruition because of the busy schedule. I left
MySQL a year ago to pursue an opportunity with Grazr what has kept me equally
busy. However, I have made a resolution this year to give some projects that I
have wanted to improve the attention that they need. I still see a lot of
potential with the "federated" concept.

Federated as it is isn't what many people expect it to be. IBM for instance,
has a federation as a very integral part of DB2. The Federated Storage Engine
is a proof-of-concept storage engine -- not to say it doesn't …

Now that you can load an engine, we are going to look at reading data. For this we will need to implement three methods. We will also need the following schema:

CREATE TABLE `services` (
`a` varchar(125) NOT NULL DEFAULT '',
`b` text
) ENGINE=SKELETON DEFAULT CHARSET=latin1



Storage engines provide "handler" objects that are used to read/write/update tables. They inherit from the handler class defined in sql/handler.h.
The file ha_skeleton.cc holds the implementation of the handler for the Skeleton engine. Once a handler object is created it is cached and can be used for different tables that a particular storage engine controls. MySQL uses the open() and close() methods of handler object to tell the handler what table it should currently work with.

open() and close() are not called for each usage of a table, or each usage of a transaction.

For …

So I’ve been looking into the Archive Storage Engine. What I would really like to do with it is get data in realtime, because (of course) the higher-ups want reports on realtime data — that is, they are not satisfied with a report that is run regularly, they want all the data up until “now”.

It is inadvisable to replicate from one storage engine type to another. I have not yet played with it, but since an Archive table doesn’t allow updates and deletes, replicating from a MyISAM or InnoDB table to an Archive one is a bad idea.

Most folks probably run a batch job; but I wonder if it can be done in real-time. Or rather, ‘what is the best way to run it real-time?’ One way, off the top of my head, is to do this are to replicate to a blackhole table with a trigger, to insert into an archive table whenever an INSERT statement is called. The blackhole table should not give an error upon UPDATE or DELETE statements.

…