Crowdsourcing - Alone in the crowd
Crowdsourcing sites make me feel lonely. Photo by Feuillu

I don’t dig Digg. I used to savor the taste of del.icio.us/popular, but it’s gone sour. Netscape.com leaves me cold. Ditto for Reddit, Newsvine and so on.

All of these sites are close—_so close!_—to being truly insightful news tools, but they’re missing a crucial ingredient, one that Amazon has offered for years. More on that in a sec.

These sites are the darlings of the new wave of news “crowdsourcing” properties. If you haven’t heard of it, the idea is to do an end run around traditional editors and allow website visitors to pick the day’s top stories. You vote for (or bookmark, in the case of del.icio.us/popular) the web pages that interest you from anywhere on the web, and the most popular articles float to the front page.

These sites are remarkable in their way, and do provide a useful service by quickly surfacing what’s new and popular among their audiences. But new and popular isn’t necessarily what I want from a news site—unless, that is, my idea of “interesting” is completely in sync with the group that’s doing the voting. Show me articles that are interesting to my crowd, and everything’s copacetic.

In the early days of del.icio.us, I happily found that this was often the case, and the del.icio.us/popular page was a great source of new and useful technology info. As the site grew, though, the links on that page become less interesting to me. More middlebrow, less unique, less of a coherent personality. The del.icio.us flavor grew bland.

I still crave an editorial voice. And as the crowd of voters grows larger, the editorial voice evaporates.

It turns out that even when I’ve consciously chosen to dispense with traditional editors, I still crave an editorial voice. And as the crowd of voters grows larger, the editorial voice evaporates. It’s the same problem with top–40 radio stations, bestseller book racks, summer blockbuster movies and network newscasts. These are designed to safely fit everyone’s tastes but wind up fitting no one’s.

Joshua Schacter, the creator of del.icio.us, acknowledged this problem in a talk he gave last winter in London. Josh noted that content aggregation works well for small populations with an obvious bias or point of view, but when the population grows larger, the point of view gets weaker. He added that he no longer found the del.icio.us/popular page particularly interesting.

The usefulness of these sites, or at least of their front pages, has become victim of their popularity.

Fortunately, this popularity does bring with it a mountain of data, which offers up an exciting possibility: Intelligent content recommendations.

This is where Amazon and its personal recommendations come in. Instead of emphasizing best sellers on its front page, Amazon gives you best-guess recommendations based on your past purchases. You know the drill: “You bought this, so we think you’ll like this” and “Customers who bought this also liked…”

Last.fm may be an even better example. It listens in on your musical tastes to create a custom radio station that mixes both new and familiar music. The result is dead-on, totally customized to my tastes. At this writing, Last.fm knows the last 29,151 songs I’ve listened to. By combining that information with the millions (billions?) of listens from other people, the site’s algorithms make startling good recommendations about music that I might like.

If Last.fm just went the “most popular” route, I’d wind up with Justin Timberlake, Christina Aguilera and Beyonce. Instead, it gives me music that’s popular among people with musical tastes similar to mine. From the crowd, it crafts an editorial voice that rhymes with my own but that still surfaces new music.

The crowdsourcing sites have the data, now they have to crunch it.

Here’s the thing. To generate their recommendations, Last.fm and Amazon are working with the same basic data as the crowdsourcing news sites: votes. Amazon calls them purchases, and Last.fm calls them played tracks.

The crowdsourcing sites have the data, now they have to crunch it. If they do, they can use my everyday behavior on the site (clicked articles, bookmarks, votes) to find articles enjoyed by “neighbors” who share my demonstrated interests.

I don’t mean to trivialize the technical challenge of doing this, and to be fair, Digg, del.icio.us et al already provide some tools to let me find my neighbors on my own and manually create custom newsfeeds based on their picks. But ideally, the sites would do that work for me behind the scenes. More social-networking sites should provide their social benefits without requiring me to be actively social and manually seek out network friends and neighbors on my own. They should surface those connections for me.

These sites already know lots about my reading tastes. I’d love to see them put that knowledge to more use.

Read more about...