Welcome to Log Buffer. This week’s issue #187 was another group effort. Thanks to all our contributors – you rock!
Suggested by Pythian’s Bradd Piontek, is a post he really liked because he used to write pipelined functions for Dynamic Search queries, – Tom Kyte’s something new I learned about estimated cardinalities. He’s also highlighted something new Tom learned about sqlplus. And the fact that Richard Foote announced the …
[Read more]We then finally came to the topic that comes naturally to anyone familiar with the MySQL architecture. Could Hadoop, or Hive, or whichever, be plugged into MySQL as a storage engine? And why would you want to do that? And can Timour's work to push down JOIN conditions be of any help? (The last question was interesting since Ted and his team were inherently against talking about JOINs at all :-)
We ended up concluding that 2 things could be interesting and considered "low hanging fruit":
SQL to PIG compiler.
(Btw, it is not immediately obvious if this task needs MySQL at all.)
(This is the first in a series of blogs written while I was trying to get home from Santa Clara. Posting them now as I'm back home and online.)
These MySQL conferences certainly get more interesting every year. Last year we got acquired and I went home thinking I need to start looking for a new job. This year Eyjafjallajõkull erupted and I had trouble getting home at all!
Pre-conference there was also the suspense to see whether the conference will take place at all, but O'Reilly and Colin pulled it together perfectly! It seems most if not all Europeans will eventually find their ways home, so all in all, all is well that ends well.
To recount everything we've learned, I will step backwards in time through the highlights as I saw them.
While there is a dizzying array of technologies that have the “NoSQL” label applied to them, I’m looking for one to replace a MySQL cluster. This particular cluster has roughly a billion records in it, uses a few TB of disk space, and is growing all the time. It is currently a MySQL master with several slaves and handles a fairly light query volume. The reasons we’d like to change it are:
(and before you ask, yes “rotating blades” comes from “become a fan”)
I’m forming the ideas here first and then we can go and implement it. Feedback is much appreciated.
Two tables.
Table one looks like this:
CREATE TABLE fan_of (
user_id BIGINT,
item_id BIGINT,
PRIMARY KEY (user_id, item_id),
INDEX (item_id)
);
That is, two columns, both 64bit integers. The primary key covers both columns (a user cannot be a fan of something more than once) and can be used to look up all things the user is a fan of. There is also an index over item_id so that you can find out which users are a fan of an item.
The second table looks like this:
CREATE TABLE fan_count (
item_id BIGINT PRIMARY KEY,
fans BIGINT
);
Both tables start empty.
You will have 1000, 2000,4000 and 8000 concurrent clients attempting to run the …
[Read more]Talend raises $8m. Cisco leaves a TIP. Exit strategies. And more.
Follow 451 CAOS Links live @caostheory on Twitter and
Identi.ca
“Tracking the open source news wires, so you don’t have
to.”
# Talend raised $8m Series D from existing investors Balderton Capital, AGF Private Equity and Galileo Partners.
# Cisco promised to open source Telepresence Interoperability Protocol.
# Alfresco Community 3.3 included CMIS 1.0, Google Docs and IBM Lotus integration.
# Exit strategies – secrets of success for open source companies, from the Open Source Think Tank.
# …
[Read more]I have noticed a definite increase in NoSQL buzz over the last few months. This is partly confirmed by Google Trends, this service shows data relating to how search topics rank:
The last couple of months has seen a dramatic rise in both the number of searches and also the number of news items relating to NoSQL.
But the traditionalists need not yet fret, interest in NoSQL is yet but a blip on the data management radar, as demonstrated by this compairson between NoSQL and MySQL search rankings:
I will be interesting to see how the dynamics of this change
throughout 2010 though.
Related articles by Zemanta …
Other day we had a small discussion about data stores and hardware; and which one drives the other when it comes to data storage solution, rather it is a hard discussion as both on its own are bigger entities; and one can not easily conclude as it depends on use cases and actually speaking data [...]
Brian Aker parted ways with the mainstream MySQL release, and with Sun Microsystems, when Sun was acquired by Oracle. These days, Aker is working on Drizzle, one of several MySQL offshoot projects. In time for next week's MySQL Conference & Expo, Aker discussed a number of topics with us, including Oracle's motivations for buying Sun and the rise of NoSQL.
The key to the Sun acquisition? Hardware:
Brian Aker: I have my opinions, and they're based on what I see happening in the market. IBM has been moving their P Series …
[Read more]Very nice and interesting post from Michael Stonebraker explaining how errors dictate CAP Theorem (Consistency, Availability and Partition-tolerance); as only one objective from the CAP can be achieved during normal error conditions as NoSQL system seems to relax the consistency model as CAP theorem anyway proves that one can’t get all 3 at the same [...]