Facebook creating their own search engine

Interesting post by Aditya Agarwal about how facebook created their own search engine. Facebook is a closed system and not spidered by google with 30 millions users wanting search.

What they did was spider their own site and created a reverse-index.

Heres some staggering facts:

- Over 600 million searches per month.
- Approximately 1 terabyte of in-memory data.
- Average search query time of less than 100ms.

The results are more relevant because its based on your social graph distance. ie People you know are more relevant.

Nicely done.

Fast-Flux Service Networks

Here's a link to a very interesting explanation of fast-flux services networks. This is the next level beyond spam bots.

They use revolving dns entries and nameservers (to avoid detection) and hit proxy zombie machine for content. These zombies in turn pull their content from some other machine!

Ted vids

Although I was familiar with TED, I didn't realize they post their videos.

Go check them out. Lots of good stuff.