Dear Kettle friends,
Will Gorman and Mike D’Amour, Senior Developers at Pentaho, are presenting Pentaho’s Google integration work at the Google I/O Developer Conference. (at the Sandbox area to be specific) Yesterday, Pentaho announced that much.
Here are a few of the integration points:
Writing small volume of data (Bytes-MBs) with sync
(fsync()/fdatasync()/O_SYNC/O_DSYNC) is very common for RDBMS and
is needed to guarantee durability. For transactional log files,
sync happens per commit. For data files, sync happens at
checkpoint etc. Typically RDBMS does syncing data very
frequently. In this case, overwriting is much faster than
appending for most filesystems/storages. Overwriting does not
change file size, while appending does. Increasing file size
requires a lot of overheads such as allocating space within the
filesystem, updating & flushing metadata. This really matters
when you writes data with fsync() very frequently. The following
are simple benchmarking results on ext3 RHEL5.3.
1. creating an empty file, then writing 8KB 128*1024 times with
fdatasync()
fdatasync per second: 3085.94321
(Emulating current binlog (sync-binlog=1) behavior)
2. creating a 1GB data file, then …
Following the launch of the Open Database Alliance a number of interesting reports were published that examined its role in establishing MariaDB as an alternative development branch for MySQL and as a vendor-neutral open source database collective.
I had a few questions myself, which Monty Widenius and Peter Zaitsev, CEO of Percona, were good enough to answer for me via email. They also agreed for the exchange to be published here. This is what they had to say:
Q: Monty has stated that the intention is to open up the Alliance to include other open source database projects - any indication of how this would be done given the diverse …
[Read more]We’re proud to announce the availability of the 2nd alpha-version of MSQL Workbench 5.2. As you may already know, this is the youngest member of the workbench-family and also the version that adds another role - the one of the database-querying-tool - to the application. There have been major enhancements - not only to the querying part - since we released the first alpha version. Please keep in mind that this is still an alpha version and it’s not recommended for use in a production environment. While most parts of the final application are already in place there are still a few things left to implement or details subject to change. Please download the program, take it for a test-drive and tell us what you think.
Source code and binary packages of MySQL Workbench 5.2.1 OSS for Windows, Mac OS X Leopard and some Linux distributions are freely available for download at:
…
[Read more]
First there was LAMP. But are you using GLAMMP? You
have probably not heard of it because we just coined the term
while chatting at work. You know LAMP (Linux, Apache, MySQL
and PHP or Perl and sometimes Python). So, what are the extra
letters for?
The G is for Gearman - Gearman is a system
to farm out work to other machines, dispatching function calls to
machines that are better suited to do work, to do work in
parallel, to load balance lots of function calls, or to call
functions between languages.
The extra M is for Memcached - memcached is
a high-performance, distributed memory object caching system,
generic in nature, but intended for use in speeding up dynamic
web …
Following on the heels of our memcached performance tests on SunFire X2270 (
Sun's Nehalem-based server) running OpenSolaris, we ran the same
tests on the same server but this time on RHEL5. As mentioned in
the post presenting the first memcached results, a 10GBE Intel Oplin card was
used in order to achieve the high throughput rates possible with
these servers. It turned out that using this card on linux
involved a bit of work resulting in driver and kernel
re-builds.
Following the launch of the Open Database Alliance some people have assumed that it is only a matter of time before MariaDB becomes the de facto replacement for MySQL.
That assumes that Oracle will allow the development of MySQL to stagnate, either deliberately or through neglect - something that we have expressed our doubts about, but even if that were the case it appears that the GPL (or more to the point MySQL’s dual licensing strategy) may restrict the potential for MariaDB.
Curt Monash recently raised the question of whether closed-source storage engines can be used with MySQL (and, by …
[Read more]
Following on the heels of our memcached performance tests on SunFire X2270 (
Sun's Nehalem-based server) running OpenSolaris, we ran the same
tests on the same server but this time on RHEL5. As mentioned in
the post presenting the first memcached results, a 10GBE Intel Oplin card was
used in order to achieve the high throughput rates possible with
these servers. It turned out that using this card on linux
involved a bit of work resulting in driver and kernel
re-builds.
Following on the heels of our memcached performance tests on SunFire X2270 (
Sun's Nehalem-based server) running OpenSolaris, we ran the same
tests on the same server but this time on RHEL5. As mentioned in
the post presenting the first memcached results, a 10GBE Intel Oplin card
was used in order to achieve the high throughput rates possible
with these servers. It turned out that using this card on linux
involved a bit of work resulting in driver and kernel
re-builds.
So lets test some different configurations and try and build some best practices around Multiple SSD’s:
Which is better? Raid 5 or Raid 10?
As with regular disks, Raid 10 seems to performance better ( accept for pure reads ). I did get a lot of movement test to test like with the 67% read test -vs- the 75% or 80% tests. But all in all RAID 10 seemed to be the optimal config.
Should you enable the controller cache? One of the things I have found in my single drive tests is that “dumb” controllers tend to give better performance numbers then “smart” controllers. Really expensive controllers tend to have extra logic to compensate for the limitations of traditional disk. So I decided to play with some of the controller options. The obvious one is cache on the controller.
Some tests showed substantially better performance when the disk cache was disabled ( both read & write ).
If better …
[Read more]