EC2 is nifty, but it doesn’t appear suitable for all needs, and that’s what this post is about.

For instance, a machine can just “disappear”. You can set things up to automatically start a new instance to replace it, but if you just committed a transaction it’s likely to be lost: MySQL replication is asynchronous, EBS which is slower if you commit your transactions on it, or EBS snapshots which are only periodic (you’d have to add foo on the application end). This adds complexity, and thus the question arises whether EC2 is the best solution for systems where this is a concern.

When pondering this, there are two important factors to consider: a database server needs cores, RAM and reasonably low-latency disk access, and application servers should be near their database server. This means you shouldn’t split app and db servers to different hosting/cloud providers.

We’d like to hear your thoughts on EC2 …

Yahoo opens up Hadoop distribution. Microsoft and Novell claim customer wins. And more.

Follow 451 CAOS Links live @caostheory

The elephant in the room
Plenty of news emerged form the Hadoop Summit this week, including Cloudera announced support for Amazon Elastic Block Storage (EBS) and introduced Sqoop, open source tool for importing databases into Hadoop, while Yahoo! Released! The! Yahoo! Distribution! Of! Hadoop! opening up its Hadoop developments to the wider community. As Savio Rodrigues noted, there has been a surge in the number of contributors for the Hadoop project in the last year.

Best of the rest
…

Vyatta raises series C funding. Greenplum launches data cloud initiative. Fedora 11. And more.

Follow 451 CAOS Links live @caostheory

# Vyatta raised $10m in series C round, led by Citrix.

# Carlo Daffara published Horses, carriages and cars an assessment of the shifting OSS business models, and a proposal of what is the optimal model.

# Greenplum delivered version 3.3 of its analytical database, launched its Enterprise Data Cloud initiative.

# Daniel Abadi asked whether betting on the MySQL mass market for data warehousing a good idea.

# Roberto Galoppini …

Hi all,

Here is another installment on working in the cloud, the AWS cloud that is. Today's topic: creating a custom AMI. This may sound like as easy task. And it would have been, had AWS documentation been up to scratch. I spent lots of time messing around with this, and I finally got it to work. Here's how:

1. Launch an exiting image (I chose a Fedora 8 image). Modify the /var/www/html/index.html file to something else (thus making it "your unique AMI). I went one step further: I installed JDK 6 and MySQL on mi AMI for use at a later date.
2. Now you have to upload your private key and x509 certificate to the AMI. Here's how to do this in the terminal:

       scp -i ~/.ssh/<yourkeypair.pem> ~/.ec2/<pk-whatever.pem> ~/.ec2/<cert-whatever.pem> root@your-public-DNS:/mnt.

   The private key and x-509 certificate should be uploaded to the mnt directory to prevent them from being …

Hi all,

Here is another installment on working in the cloud, the AWS cloud that is. Today's topic: creating a custom AMI. This may sound like as easy task. And it would have been, had AWS documentation been up to scratch. I spent lots of time messing around with this, and I finally got it to work. Here's how:

1. Launch an exiting image (I chose a Fedora 8 image). Modify the /var/www/html/index.html file to something else (thus making it "your unique AMI). I went one step further: I installed JDK 6 and MySQL on mi AMI for use at a later date.
2. Now you have to upload your private key and x509 certificate to the AMI. Here's how to do this in the terminal:

       scp -i ~/.ssh/<yourkeypair.pem> ~/.ec2/<pk-whatever.pem> ~/.ec2/<cert-whatever.pem> root@your-public-DNS:/mnt.

   The private key and x-509 certificate should be uploaded to the mnt directory to prevent them from being …

Cloudera lands funding. SourceForge acquires Ohloh. Novell reports Linux growth. And more.

Follow 451 CAOS Links live @caostheory

Cloudera shows signs of progress
GigaOM reported that Cloudera raised $6m Series B funding from Accel and Greylock and is now looking beyond web applications to wider enterprise adoption of Hadoop. Cloudera also announced its first certification program for Hadoop.

Open source goes mainstream in the UK
There have been signs of change recently with regards to open source adoption in the UK, which has traditionally lagged behind the rest of Europe and the US. CBR Magazine provided an analysis of …

Hi all,

Hmmmm...

I switched on my machine this morning thinking a new day will bring new results. Nope. Same old results. As I mentioned in an earlier blog entry, I created a server instance using Fedora with LAMP, complete with MySQL. Hmmm.. Sound good? Well, it comes with MySQL 4.1. Normally an update would be a straightforward process. Since Fedora 8 doesn't come with an

apt-get

command, I chose to go the

yum

route. I tried

 yum update mysql-server

. No dice. problems. I spent another 30 minutes or so, trying to correct the problem, but to no avail. So I decided to create a whole new server instance with Fedora 8 and no MySQL. I manually installed MySQL on the machine, by doing the following:

• I download the MySQL sever, cient, and headers and libraries from the MySQL.com.

      wget …

Hi all,

Hmmmm...

I switched on my machine this morning thinking a new day will bring new results. Nope. Same old results. As I mentioned in an earlier blog entry, I created a server instance using Fedora with LAMP, complete with MySQL. Hmmm.. Sound good? Well, it comes with MySQL 4.1. Normally an update would be a straightforward process. Since Fedora 8 doesn't come with an

apt-get

command, I chose to go the

yum

route. I tried

 yum update mysql-server

. No dice. problems. I spent another 30 minutes or so, trying to correct the problem, but to no avail. So I decided to create a whole new server instance with Fedora 8 and no MySQL. I manually installed MySQL on the machine, by doing the following:

• I download the MySQL sever, cient, and headers and libraries from the MySQL.com.

      wget …

Hi all,

Today I'll be write more about my experiences with Amazon Web Services. Before I continue, I'd like to provide some background information.

I signed up for Amazon Web Services. I used the Fedora LAMP AMI to create a server instance. The main reason for choosing this AMI was its inclusion of MySQL and Apache, a nice little package indeed. I ran into a few problems at first. Because I am based in Europe, I am allowed to use only the EU-West region, and the default in the US-East region. The drop down list is rather small, and i took a couple of unsuccessful instances to figure out that this was the problem. When I say "unsuccessful instances," I mean that the nothing appeared in the browser when I copied the public DNS to the browser's address bar. I attached an EBS volume to the server instance and it was OK.

Now for more on the command line.

I spent a lot of time messing around with the commands in the …

Hi all,

Today I'll be write more about my experiences with Amazon Web Services. Before I continue, I'd like to provide some background information.

I signed up for Amazon Web Services. I used the Fedora LAMP AMI to create a server instance. The main reason for choosing this AMI was its inclusion of MySQL and Apache, a nice little package indeed. I ran into a few problems at first. Because I am based in Europe, I am allowed to use only the EU-West region, and the default in the US-East region. The drop down list is rather small, and i took a couple of unsuccessful instances to figure out that this was the problem. When I say "unsuccessful instances," I mean that the nothing appeared in the browser when I copied the public DNS to the browser's address bar. I attached an EBS volume to the server instance and it was OK.

Now for more on the command line.

I spent a lot of time messing around with the commands in the …