There’s a strong trend to automation across the cloud. That’s a great thing for startups because it reduces operational headaches & lets them focus on building products. Join 31,000 others and follow Sean Hull on twitter @hullsean. But as that trend begins to touch the database tier, all sorts of complications emerge. Let’s take a … Continue reading When hosting data on Amazon turns bloodsport →

Heterogenous replication involves moving data from one database platform to another. This is a complicated endevour because datatypes, date & time formats, and a whole lot more tend to differ across platforms. In fact it’s so complex many enterprises simply employ a commercial solution to take away the drudgery. Join 31,000 others and follow Sean … Continue reading Replicate MySQL to Amazon Redshift with Tungsten: The good, the bad & the ugly →

I just dove into Tungsten replicator very recently as I need to replicate from Amazon RDS to Redshift. I’d heard a lot of great things about Tungsten, but had yet to really dig my heels in. Join 28,000 others and follow Sean Hull on twitter @hullsean. I fetched the binary and began to dig through … Continue reading Wrestling with bears or how I tamed Tungsten replicator →

Every day hundreds of millions of dollars are wasted by allowing improperly tuned or misconfigured systems, misunderstood infrastructure, and inefficient IT operations to live and thrive in data centers around the globe. There are both direct and indirect costs associated with allowing these unhealthy systems to continue to exist. Let’s look at some.

The setup:

Let us start by using a small example. We will start by looking at a small database setup. This setup will have a single master-slave, with a database size of lets say 500GB. Traffic is steady and let’s say this translates into 500 IOPS on the master. You have chosen to host this on Amazon’s AWS. A common way of ensuring backups occur in AWS is to setup ebs snapshots of the slave. In terms of usage, let us assume your CPU is about 50% used and you have about 20GB of hot data that needs to stay in the memory for the database.

If we look at what this …

Join 29,000 others and follow Sean Hull on twitter @hullsean. Can RDS worsen an outage ?? That’s another way to think about this question. In my experience, it very clearly increases outages, by tying one or both hands behind your back. Believe me when I say, that is terribly frustrating when you’re putting out fires! […]

Join 29,000 others and follow Sean Hull on twitter @hullsean. Oh RDS, you offer such promise, but damn it if the devil isn’t always buried in the details. Diving into a recent project, I’ve been looking at upgrading RDS MySQL. Major MySQL upgrades can be a bit messy. Since the entire engine is rebuilt, queries […]

If you’ve used relational databases for more than ten minutes, I hope you’ve heard of slow queries. Those are those pesky little gremlins that are slowing down your startup, and preventing scalability you so desperately need. Luckily there’s a solution. What I’ve found is if I send a report to developers every week, it keeps […]

Since 2011 e-commerce has grown from $856,000,000.00 to closing out 2013 at more than $1,248,000,000,000.00 in global revenues, a nearly 41% growth rate over a two year span. The forecast for 2014  is targeted at $1,500,000,000,000.00, another 20% increase is gross revenues (see chart below). With trillions of dollars in play year on year now, and astounding revenue growth rates still ahead, it is critical that any e-commerce site be diligently monitoring not just their website uptime, but more importantly their Web applications. Obviously it is critical that your customers be able to get to your website, but if it doesn’t build fast and complete client requests and transactions even faster then you will be losing clients and revenue at an alarming rate. The latest figures show that if your site doesn’t do a full page load in 3 seconds or less, …

Troubleshooting MySQL on Amazon can be a real test of patience. There are quite a few different things to watch out for in terms of connectivity & networking. Sometimes a checklist can help. Join 16,000 others and follow Sean Hull on twitter @hullsean. Here’s my exhaustive list of things that can block you. 1. Be […]

I like the image of this city of Mesa Verde. It’s fascinating to see how ancient cities were built, especially as an inhabitant of one of the worlds largest cities today, New York. I’m a long time relational database guy. I worked at scores of dot-coms in the 90′s as an old-guard Oracle DBA, and […]