Sean Gallagher of eWeek talks about the bandwidth demands of RSS feeds on popular sites.
What I see is that RSS feeds are providing a service that many people have wanted–the most up-to-date information around.
I’ve thought for a long time that what the “search” world is missing is a search company flipped upside down–a search infrastructure geared towards the most recent changes and news. I want a service that stays current of all changes across the world and gives me immediate access to them. As information ages it falls off the search results. Old news wouldn’t be the focus of this kind of search service. There’s great demand for searches like this as the RSS feeds are indicating. I’m not sure if “new” news is a bigger market than searching in general, but I know for me I rely on access methods to new items on the net at least as often as I jump to the Google search field to research an item I expect Google has archived access to.
Of course, to track all new information across the world would be a ridiculous challenge today.
When Google launched their news.google.com I thought, “Cool, they’ve solved the 80% problem of accessing new information.” News.google.com brings together the top x number of news sites—sites I’m likely to want to monitor for changes. It doesn’t have to look for deltas everywhere.
RSS feeds, aggregator sites, and services such as Technorati and Feedster provide similar services for the blogosphere. In my mind RSS feeds are succeeding because they get people to the recent changes (in this case new posts) on a site. (In fact, I think people are migrating towards blogs because they emphasize new information–which is yet another way to get around the search issue.)
The downside is that for popular RSS feeds, the feeds are…well…popular and that costs money.
As this crossover is reached–as it does in all successful tech areas–it puts demands on the existing infrastructure that was built with old assumptions thereby encouraging innovation. I think what we’re missing and have been missing all along is a better infrastructure for searching and accessing new information on the Net or better yet, the changes anywhere on any page.
The market may also force bandwidth suppliers to adjust how they charge for bandwidth. Access to static/catalog-like information is one thing. Access to the most recent information around–which info consumers might continuously monitor–is another. The market may be on the cusp of demanding another magnitude drop in bandwidth prices.
Yes, searching for recent changes and news is a tremendous challenge to perform brute force by robots surfing the Net looking for deltas. I wonder, though, if there’s not a better way to mine for deltas through the pipes of the Net itself–something highly distributed. Another approach might be for all sites to publish their deltas.
Feedster and the like build lists of blog and news sites from which they mine so they can limit the search space. But what about the tracking problems of a customer looking for recent sale items on an online store? Right now you usually have to register for an RSS feed or email newsletter at each ecommerce site. Yuk. I just want to go somewhere to find it. I don’t want to register for content or clog up my machine with more emails or links–RSS or not–than I need to.
It’s an interesting problem.