Well, the servers we bought (Dell R200) were in fact very cheap. We were using 2 of them and they could not stack up to 4-5 year old servers (Dell 1750 and 1850) which have SCSI hard disks. 
I added a 4 year old Dell 1850 with SCSI disks next to the 2 R200's which were all connected to a separate server running pound load balancer. The total number of maximum concurrent connections tripled.  The Dell R200's cannot actually take SCSI/SAS drives (unless you go buy a proper controller). By default, they can only take Nearline-SAS which I heard is not something you would want for a web server.
I would have to say that I am putting the blame on the SATA drives. I would have thought that by now, SATA drives would be as fast as old SCSI drives, but I guess not.Apart from that, I am still surprised that the new R200's with EIGHT Gbs of ram could not keep up (not even close) …

Follow up post to this one
At my company, we have some really old web servers that host our website. They are from late 2004 and early 2005. Now, at the time, I am sure they were quite expensive.

Recently, we wanted to move to newer web servers and we bought the basic Dell R200 rack servers. Now, arguably they are quite cheap, but thinking about it they will definitely be faster then 4 year old servers, right? Well, that’s what my predecessor thought when he bought those servers and I myself didn’t think about it till we wanted to start using them.
We noticed that the new servers are at least twice as slow as old servers and that just completely baffled us. No one saw it …