Join 8000 others and follow Sean Hull on twitter @hullsean. In the past two years we’ve written a ton of material on scalability. Here’s the greatest hits… Why Generalists Are Better at Scaling the Web The internet stack is a complex infrastructure of interlocking components. An scalability engineer must be adept at Linux, plus webservers, […]
The post Scalability Tips & Greatest Hits appeared first on Scalable Startups.
It is a truth universally acknowledged that the average is the index of central tendency. But what if the tendency isn’t central?
I’ve worked many performance issues where the latency or response time was multimodal, and higher-latency modes turned out to be the cause of the problem. Their existence isn’t shown by the average – the arithmetic mean; it could only be seen by examining the distribution as a histogram, density plot, heat map, or frequency trail. Once you know that more than one mode is present, it’s often straightforward to determine what causes the slower mode, by seeing what parameters of …
[Read more]This week we finish up talking about the Openark Kit for MySQL. Ear Candy is using both --master-data and --tab with mysqldump, and At the Movies features Robert Hodges of Continuent presenting Scalable MySQL Operation in the Cloud with Continuent Tungsten.
Openark Kit series:
Part 1
Part 2
Openark Kit
oak-purge-master-logs
PURGE BINARY LOGS at the MySQL manual page.
This week we continue talking about the Openark Kit. Ear candy is the Percona Configuration Wizard, and At the Movies is a keynote from the SkySQL and MariaDB Solutions Day about the SkySQL and MariaDB merger and the MariaDB Foundation.
Openark Kit series:
Part 1
Part 3
Events
DB Hangops -
every other Wednesay at noon Pacific time
Training
SkySQL Trainings
With our recent release of TokuMX 1.0, we’ve made some bold claims about how fast TokuMX can run MongoDB workloads. In this post, I want to dig into one of the big areas of improvement, write performance and reduced I/O.
One of the innovations of TokuMX is that it eliminates a long-held rule of databases: to get good write performance, the working set of your indexes should fit in memory. The standard reasoning goes along the lines of: if your indexes’ working set does not fit in memory, then your writes will induce I/O, you will become I/O bound, and performance will suffer. So, either make sure your indexes fit in …
[Read more]In computer performance, we’re especially concerned about latency outliers: very slow database queries, application requests, disk I/O, etc. The term “outlier” is subjective: there is no rigid mathematical definition. From [Grubbs 69]:
An outlying observation, or “outlier,” is one that appears to deviate markedly from other members of the sample in which it occurs.
Outliers are commonly detected by comparing the maximum value in a data set to a custom threshold, such as 50 or 100 ms for disk I/O. This requires the metric to be well understood beforehand, as is usually the case for application latency and other key metrics. However, we are also often faced with a large number of unfamiliar metrics, where we don’t know the thresholds in advance.
There are a number of …
[Read more]
FOREIGN KEYs in MySQL Cluster is a big step
forward. It is now possible to run enterprise software with NDB
Cluster as the storage backend. Over the years, the lack of
FOREIGN KEYs have been one of the most limiting pieces of
functionality. Who wants to fiddle with TRIGGERs or recode
applications to enforce data integrity?
But finally, it is here. It is implemented natively at the Data
Node level, where NDB stores its data. It is well known that
FOREIGN KEYs come with an overhead. E.g., when writing a record
into a child table, the existence must be checked in the parent
table. Since data is distributed across multiple Data Nodes, the
child record and parent record may be on different nodes or
shards (Node Groups). Hence there is extra work to be done in
terms of internal triggers and network communication, the latter
being the more costly. The performance impact must be taken into
account when doing …
Join 6100 others and follow Sean Hull on twitter @hullsean. Lots and lots of web applications need to page through information. From customer records, to the albums in your itunes collection. So as web developers and architects, it’s important that we do all this efficiently. Start by looking at how you’re fetching information from your [...]
Many application developers would know of New Relic. A SaaS performance and monitoring tool targeted towards your web application monitoring including PHP, Ruby, Java, .Net, Python and Node.
With the release today (June 19, 2013) of the New Relic Platform, custom monitoring of data stores including MySQL are now possible. Try it now free. This link will provide you a free standard account (no cost, no billing details necessary), that enables you to perform application monitoring, server monitoring, MySQL instance monitoring and monitoring of many other products via many plugins.
Over the next few posts I will be discussing some of the design decisions I made for this MySQL plugin. New Relic has certain …
[Read more]Join 6100 others and follow Sean Hull on twitter @hullsean. The Foundation of Speed All servers use disk to store files. Operating system libraries, webserver & application code, and most importantly databases all use disk constantly. So disk speed is crucial to server speed. [mytweetlinks] [quote] Disk speed is crucial for MySQL databases. It has [...]
