In the last several months at Grazr, we've been wrestling with a
large database (running on MySQL) of feeds and feed items. The
schema is essentially a feeds table with child tables items,
items_text (text), and enclosures. We have this database to
provide the means for users to be able to merge (a Stream) feeds
so that you have an aggregate feed with items for whatever feeds
you want in the list of feeds for your merge. It works great, the
only problem being the volume of data, which more data means the
query to produce that merge becomes slower. We want this merge to
be able to be run on the fly, and if it's too slow, the user
experience is unacceptable.
So, now I'm in the process of implementing a "Hot Cache" of feeds
with an LRU (Least Recently Used) policy. The idea being, that
this cache provides a smaller data set for performing the merge
query against. We need to be able to handle storing much more
data than we currently do …
Showing entries 1 to 1
Jun
17
2008
Showing entries 1 to 1