Fav5

1. Thanks to Mike Baur for sending me a link to this video of some really cool image visualization software being developed at Microsoft... I especially liked the aggregation feature which pulled images from Flickr together to form a complete image. If you thought the video was cool, you may also be interested in these other videos from the TED conference.
   


2. Vote for your favorite 1980’s arcade game. Although I genuinely loved most of the games on the list, I had to go with Tron.
   


3. There’s a good article from the New York Times about Google’s continued focus on search: Google Keeps Tweaking Its Search Engine. Matt Cutts also put in his 2 cents on about Google’s search focus.
   


4. Researchers are finding that bullying in the virtual world is becoming quite a problem, especially for newbies.


5. Ask.com has received a makeover. Ask3D is their new approach at integrating various vertical search results together in a single results page. Nice.

First Fav5

I'm starting a new weekly installment on my blog entitled "Fav5" where I list 5 items of interest for the week. I'm not putting any restrictions on the list, so anything may appear here. You may notice this week's list is a little heavy on the search engine side.

1. I recently found out that Ask, MSN, and Yahoo are now supporting Google's Sitemap Protocol. The announcement was made back in Nov. 2006 although Ask apparently joined a little later, and MSN still hasn't implemented it yet. The "official" website for sitemaps is now http://www.sitemaps.org/.
   
   Just last month an autodiscovery method was announced which makes it easy to notify all the search engines of your sitemap by naming it in your robots.txt file. Alternatively you can use the search engine's interface or ping a search engine via an HTTP request to let them know the sitemap file location.
   


2. If you didn't already know, Google posts their Tech Talks on Google Video. There are some top-notch presentations out there if you have an hour to burn per talk.
   
   The most recent talk is The Internet of Things: What is a Spime and why is it useful?
   by Science Fiction writer and futurist Bruce Sterling. Early in the talk, Bruce rightly lauds Vannevar Bush's prediction of a memex device as one of the most "brilliant acts of technological forecasting, ever."
   


3. Google announced their new Web History feature a few weeks ago:
   

   Today, we're pleased to announce the launch of Web History, a new feature for Google Account users that makes it easy to view and search across the pages you've visited. If you remember seeing something online, you'll be able to find it faster and from any computer with Web History. Web History lets you look back in time, revisit the sites you've browsed, and search over the full text of pages you've seen. It's your slice of the web, at your fingertips.
   

   Matt Cutts shows some use cases for Web History and also ties it in to Bush's memex device.
   


4. Joel points out a very interesting usability problem posed by the elevators installed in the new World Trade Center highrise.
   


5. And on a personal note, I'm going to be presenting a poster Search Engines and their Public Interfaces: Which APIs are the Most Synchronized? at the World Wide Web conference in Banff, Alberta next week. You can see a PDF of my poster here. Please come by my poster and say hello if you're at the event.
   
   I just found out Johan (ex-ODU professor) is going to be at WWW too. It's too bad his poster is kinda lame.
   

Server encoding caching experiment

To determine if my server-side component encodings could be inserted into indexable/cacheable HTML files, I ran a little experiment. I created 3 HTML files that contained encoded chunks in HTML comments at the base of each file:

html_encoded1.html - 2 KB
html_encoded2.html - 45 KB
html_encoded3.html - 99 KB

If you view the source of the pages, you’ll see something like this at the end:

<!-- BEGIN_FILERECOVERY
chunks = 4
filename = xor.o
recover = 2
orig_size = 1105
block_size = 554
block_num = 3

fY/xaGQn0V5MOOpLnM1WIsIUMirrVBQ2XNhidvc5yjL9tEyKTmNjNPjcrJzcPWvs INxxHl1Gt5lKQAYoNi1DXOhFI5ExBm15Nxx1T/hFCwVvsyaHsQQdd3lcqWJl+WTw BTlkiI8yWcPPoy38dqgTVnc4aSNd+0YQWW0bDl67/6XTnych3rSXn5YEYhVMU2eS LCR/0N4pAhKgeMb7SXtdJNQ6WykqDXYJAjtTOIrT2CLaPNRdKbU/ydsvUSDenSt+

Etc…
END_FILERECOVERY -->

I placed these files in my public_html folder on April 19, and linked to them from my index.html page. Today I checked Google, MSN, Yahoo, and Ask to see if any of them were cached. Here’s the results:

Google – cached all three
MSN – cached 1 and 2
Yahoo – indexed 2 only (not available in their cache)
Ask – nada

To see if Google can handle any more, I have created 4 new files of 150, 200, 250, and 300 KB. Looks like 99 KB is too large for MSN. Yahoo’s cache is really inconsistent- maybe 2 is in there, maybe it’s not. Why didn’t they grab 1?

I’ll check back in a couple of weeks and see if anything else has been cached.

Update: 6/20/06

Google and MSN have cached all files that range up to 300 KB. Yahoo has only indexed the first 3 (none are cached), and Ask has nothing.

Now I'm going to create a 400 KB, 500 KB, and 1 MB file and see what happens.

Update: 2/21/07

The cache limits for the search engines appear to be the following: Google - 977 KB, Yahoo - 214 KB, and MSN - 1 MB. I still cannot tell for sure what Ask's limit is, but I ran an experiment where I found 984 KB cached for a document that was 1.6 MB. Google's limit has been confirmed by others.