A special extended edition of Tech Messages for 2011-01-07 through 2011-01-08:

• The Services Used By Y Combinator Startups [Infographic] – ReadWriteCloud
  "The results show that the majority of Y Combinator startups depend on cloud services instead of self-hosted servers, traditional web hosting or co-location services."
• Tom Limoncelli on Time Management | O'Reilly Time Management for System Administrators | Sysadmins
  Video companions to to Time Management for System Administrators by Thomas A. Limoncelli.
• Guidelines …

Warning, this may be kernel version specific, albeit this kernel is used by many database systems

Lately I’ve been working on getting more memory used by InnoDB buffer pool – besides obvious things like InnoDB memory tax there were seemingly external factors that were pushing out MySQL into swap (even with swappiness=0). We were working a lot on getting low hanging fruits like scripts that use too much memory, but they seem to be all somewhat gone, but MySQL has way too much memory pressure from outside.

I grabbed my uncache utility to assist with the investigation and started uncaching various bits on two systems, one that had larger buffer pool (60G), which was already being sent to swap, and a conservatively allocated (55G) machine, both 72G boxes. Initial finds were somewhat …

Mark your calendars for Nov 2 as Mark Callaghan and Facebook's MySQL team will be talking about how MySQL is used at Facebook.

Lately we have been especially enjoying the opportunities that Poor Man’s Profiler provides us – but also the technology has improved a lot too – there have been few really useful mutations.

One mutation (hyper-pmp) was Ryan Mack’s approach of having somewhat more efficient sampling – instead of firing gdb each time, he instructed gdb to get backtraces every time monitored process gets a signal (SIGUSR2 for example). This allows to maintain a persistent debugger attachment – and then signal periodically to get stacks analyzed.

Other mutation was auto-pmp – high frequency polling of process state (e.g. how many threads are running), and when a certain threshold is exceeded – obtaining stacks for further analysis (this combines …

We quite often say, that benchmark performance is usually different from real world performance – so performance engineering usually has to cover both – benchmarks allow to understand sustained performance bottlenecks, and real world analysis usually concentrates on something what would be considered ‘exceptional’ and not important in benchmarks – stalls of various kind. They are extremely important, as the state when our performance is lowest is the state of performance we provide to our platform users.

On a machine that is doing 5000qps, stalling for 100ms means that 500 queries were not served as fast as they could, or even hit application timeouts or exceptional MySQL conditions (like 1023 transaction limit). Of course, stalling for a second means 5000 queries were not served in time…

We have multiple methods to approach this – one is our …

Learned a few things during my trip to the Philippines this time. Another country that amazes me. For the first time in history, Philippines attracted more foreign direct investments (FDI) than Malaysia, in 2010 (see: Malaysia’s FDI plunge).

Mobiles

There are a lot of people here using prepaid phones as opposed to postpaid phones. This is because the requirements of getting a postpaid account is quite tough (you need bank documents, etc. before they give you an account).

Many people carry more than one phone (or have more than one SIM). Smartphones face an uphill battle – they cost too much and there is generally no operator subsidy because everyone prefers prepaid accounts. It makes economic sense to have more than one SIM, as you’ll end up saving money (operators like to offer free text, calls, …

Apparently, Digg performed a big migration from MySQL to Cassandra and a big migration to their new Digg v4 architecture and now their VP of Engineering has been shown the door:

Ever since Digg launched its new site design, it’s been plagued with all kinds of trouble, not least of which is that it keeps going down. The problems with the new architecture are so bad that VP of Engineering John Quinn is now gone, we’ve confirmed with sources close to Digg.

In a Diggnation video today, CEO Kevin Rose explained some of the technical issues the site is dealing with and why it can’t simply roll back to the previous architecture. The new version of Digg, v4, is based on a distributed database called Cassandra, which replaced the MySQL database the site ran on before. Cassandra is very advanced—it is supposed to be faster and scale …

In facebook stream you’ll see the time period at the bottom of the stream. For example: 4 minutes ago, 2 days ago, 3 weeks ago…. In our recent project we have to show similar time fashion for our application’s activity stream. So I write a function to retrieve the time duration. In our mysql database, [...]

…

Sometimes we may need to collect facebook user’s basic data for our website or application. Then we’ve to call graph api or legacy api or fql query to collect some specific data. This is a basic requirements for most of the fbconnect base website or facebook application. To simplify this task here I published some code so that you could easily use this code and can integrate in your site. Read Full Article

Sometimes you may need to collect facebook user’s basic data for your website or application. Then you’ve to call graph api or legacy api or fql query to collect some specific data. This is a basic requirements for most of the fbconnect base website or facebook application. For this reason, here I’m sharing the code, [...]

…