From time to time we detect query patterns that are not well fitted to the BTree+ structures provided by InnoDB. One such situation is when you need to traverse a hierarchy (tree) or graph structure with many nodes. Specialist databases exist for this such as Neo4J. However there exists a simple solution in the form of  OQGraph which is distributed with MariaDB and is documented here.

The OQGRAPH engine is based on an original idea by Open Query founder Arjen Lentz, and was developed in-house with Antony Curtis at Open Query.

A …

In recent time, MariaDB 10 has been getting many new storage engines. We’ve seen TokuDB, CONNECT, SEQUENCE, SPIDER, CassandraSE for various use cases. For a long time, MariaDB shipped OQGRAPH, but it was disabled in MariaDB 5.5. It will make a come back as OQGRAPH v3 has been worked on actively by Andrew McDonnell. Keep track of this via MDEV-5319.

Another engine being worked on by Kentoku Shiba & team is the mroonga engine, which allows you to do full text search. It is optimised for CJK languages, and is supposedly very fast. To track this, follow …

Today, while at Oracle's MySQL Connect conference in San Francisco, I had the opportunity to talk to Gerardo Narvaja and that inspired me to try using OQGraphv3 and TokuDB. Of course, the first challenge is to compile it on my MacBook which has MacOS Lion 10.7. I installed the Apple's latest XCode Command Line tools and eventually compiled a MariaDB which had both TokuDB and OQGraph available

Just mentioning it here because hard work deserves to be recognised and mentioned: Andrew McDonnell is doing an awesomely good job of testing, understanding and fixing bugs in OQGraph... which is no mean feat because the core of the code is based on the Boost Graph Library which isn't a trivial C++ library.

I did a presentation on OQGraph for the +Southern California Linux Expo on Sunday and some of the audience seemed really interested. Demonstrated finding the shortest path in a graph of millions of edges, with a deliberately crippled config running in a VirtualBox environment.

(edit, updated URLs on slides)

You may have already seen the announcement MariaDB Foundation to Safeguard Leading Open Source Database. We at Open Query wholeheartedly support this (r)evolution of the MySQL ecosystem, which appears to be increasingly necessary as Oracle Corp is seriously dropping the ball with security updates and actually just general development and innovation. Oracle has actually done some very good work, I happily acknowledge that – but security issues are critical, having crashing bugs and incorrect query results in a .28 of a GA release is uncool, and not incorporating awesome development efforts by the community is just astonishing.

MariaDB is where the Sphinx fulltext …

Using the IMDB data, I have loaded the actor and actress database into a few simple InnoDB tables and have performed the following test:

MariaDB [imdb]> show status like "Innodb_rows_read"; +------------------+---------+ | Variable_name | Value | +------------------+---------+ | Innodb_rows_read | 1237459 | +------------------+---------+ 1 row in set (0.00 sec)

MariaDB [imdb]> select x.*

I had a good and lively audience at my presentation today. I was happy to give them a sneek preview of the Mk.III implementation of OQGRAPH.

Oh yeah! I guess I'm announcing the availability of the source code of the MkIII implementation.

Links are in my slides... http://goo.gl/UrybZ

Btw, building the storage engine requires Boost libraries and libJudy installed. It's currently in a MariaDB

Peter and Arjen will be at DrupalCon SF 2010. Peter specifically for the event, Arjen staying around the SF area after the MySQL Conference last week.

Specifically, we’ll be talking with people about using the OQGRAPH engine to help with social graphs and other similar problems, easily inside Drupal. You may recall that Peter already created the friendlist_graph extension for the friendlist Drupal module.

From the MySQL Conf and other earlier feedback, OQGRAPH is proving to be a real enabler. And since it’s free/GPLv2 and integrated in MariaDB 5.2, there’s generally no hindrance in starting to use it.

Walter and I are giving a tutorial on Monday morning, MySQL (and MariaDB) Dual Master Setups with MMM, I believe there are still some seats available – tutorials are a bit extra when you register for the conference, so you do need to sign up if you want to be there! It’s a hands-on tutorial/workshop, we’ll be setting up multiple clusters with dual master and the whole rest of the MMM fun, using VMs on your laptops and a separate wired network. Nothing beats messing with something live, breaking it, and seeing what happens!

Then on Tuesday afternoon (5:15pm, Ballroom F), Antony and I will do a session on the OQGRAPH engine: hierarchies/graphs inside the …