In the previous post of this series (which can be found here) I discussed how to migrate your data once you have decided how to shard your schema.

Once your data is sharded, it’s time to modify your application code. I will not dive into the many open source platforms that provide partial sharding support (Hibernate Shards, Gizzard, and the like), and will take Java (sorry, old habits are hard to overcome) as an example – however, the same holds true for any programming language.

Without Using ORM

If you wrote your code without an Object/Relational Mapping tool, kudos to you. Sharding will be easier, as you control the SQL statements.

Upgrading Connection Pool

Your first task is to write a connection pool that is “sharding” aware.  The class should look something like this:
…

In the previous post of this series (which can be found here) I discussed how to identify tables that can serve as good candidates for sharding.

Once you have decided which tables should be sharded (all the rest should be global tables), the choice of sharding keys is rather straightforward, as most will use the table primary key as the shard key. Of course, if multiple tables are sharded, and there is a foreign key relationship between these tables, then the foreign key will serve as the shard key for some tables.

Many people attempt to shard based on customer_id or a resource id, but I have seen how this usually fails in production environments. It is very hard to know in advance which customers belong together in the same database, and since customers can suddenly increase their traffic, this might create an unbalanced situation in which some …

This is the first in a three part series of blogs in which I’ll try to explain how to take an existing application and shard it.

Database Sharding has proven itself a very successful strategy for scaling relational databases. Almost every large web-site/SaaS solution uses sharding when writing to its relational database. The reason is pretty simple – relational database technology is showing its age and just can’t meet today’s requirements: a massive number of operations/second, a lot of open connections (since there are many application servers talking to the database), huge amounts of data, and a very high write ratio (anything over 10% is high when it comes to relational databases).

Many sites and blogs posts explain what sharding is, for example here and …

Backing up data is critical for production databases – and there are a lot of well-known solutions for backing up databases.

When the database is sharded, backing up data becomes problematic. If the backup is not synchronized across all shards, data inconsistency might occur. In this blog post I’ll try to detail the possible backup scenarios for sharded databases when using ScaleBase.

Backup Types

Let’s start by understanding the different backup types that are out there. You can read all about it here.

A physical backup involves copying all database files to a different location. Copying can take several hours for a decent database if it’s done to a disk or a tape. It might take only seconds if the database files reside on SAN/NAS storage hardware that supports snapshot technology.

A logical backup is a …

Despite my best intentions, I haven't posted on this blog for a while, which is a shame! I've become busy writing on so many other places since I moved into my new role in the Oracle Linux product management team in April. I've learned a lot and I am feeling quite at home here! The team is excellent and very nice to work with — I am slowly getting the "Big Picture".

But even though I've been neglecting this blog, there are a lot of things that are publicly visible and document some of my activites:

I've created two podcasts for the Oracle Linux podcasts:

• OCFS2: Features and Benefits
• …

Wow. Our launch was picked up by many news sites. Here’s just a partial list. Thanks guys.

• BusinessWire
• CWCS
• Diversity
• DowJones (Subscription required)
• DZone
• …

Boston, Mass., August 15, 2011 – ScaleBase, Inc. today announced the general availability of ScaleBase 1.0 for unlimited scalability of MySQL databases. ScaleBase 1.0 delivers MySQL performance and high availability, without the need to change a single line of application code. Users of MySQL can download and easily deploy the software by visiting http://www.scalebase.com/solution/download/.

ScaleBase utilizes two techniques for scaling: read-write splitting and transparent sharding (a technique for massively scaling-out relational database). The software enables MySQL to scale transparently, without forcing developers to change a …

I recently came across an interesting blog post on RedMonk (not surprising, as I read most of their posts). It’s called It’s Beginning to Look a Lot Like SQL and basically it talks about query language for NoSQL databases. It seems that as NoSQL becomes more popular, users want to do more with it – a good level of querying, for example, is needed.

Now of course, since NoSQL is a family of products that work in radically different ways, it’s not certain that this is possible (or even desirable – read Alex Popescu’s post on the subject).

But my question is – why do you even need a query language for NoSQL data stores? After all, running queries on distributed data might be complex to implement, and time consuming. The better …

It's been a while since my last post on this blog; I definitely need to get back into the habit! One of the reasons for my radio silence was that I switched roles here at Oracle. After having been with the MySQL team for 9 years, I felt it was time for a change. Fortunately I did not have to look far – I'm now a member of the Oracle Linux product management team and I am having a lot of fun there.

However, I realized that while I was an active Linux user on the desktop, quite a lot has happened on the enterprise and data center side of things. Linux has really come a long way and I am glad to be back in this field, drinking from the firehose and learning a lot about recent developments and technologies. For me, this is kind of going "back to my roots", as I have been deeply involved with Linux at SuSE before I joined MySQL in 2002.

Anyway, I'm still alive and you can expect …

The Open DB Camp in Sardinia 2011 has had a number of sessions on varying topics. Topics range from MySQL over MongoDB to replication and High Availability.

I decided to tap into the database expert resources present here at Sardegna Ricerche by discussing a non-database issue, where one can expert database experts to have insights beyond those of end users. And they did.

The topic was the particular case of information overload many of us suffer from on our hard disks: Too many files, too hard to find.

• How do we find the bank statement from April 2007 from the more-seldom-used account?
• What are the ten best work-related pictures from last year?
• Is this the most current …