Bing is "copying" Google's search results?

Earlier this week, Google revealed the results of a sting operation that appeared to catch Bing red-handed. Microsoft has apparently been using Internet Explorer to determine which queries and search results their users were clicking on when using Google, and they incorporated that information into Bing's search results. Although Google says this is unfair copying of search results, Microsoft is claiming that the information is just a small part of their overall formula for ranking search results.

The debate has received a lot of attention. Even Colbert had something to say about it: "Evidently, hiybbprqag is a word meaning 'You got serverd!'"

This debate has been good fodder for my Web Science course, and it was even relevant to my Seminar class which discussed ethics and intellectual property this morning. Search engine results are intellectual property, so is Microsoft's use of clickstream data fair use, or did they cross the line?

Update:

This post generated some interest on my Facebook account. One of the comments included this link to Danny Sullivan's article that gives more analysis of the situation.

Apple vs. Google vs. Microsoft War Chart

This infographic by Shane Snow beautifully illustrates the epic battle that Apple, Google, and Microsoft are engaged in. From ebooks and phones to office and email, all three giants are battling for more users.

Although many of these services are free to the end users, there are billions of dollars in revenue at stake. Click on the map to zoom-in.

Fav5

We just completed our first week of 16 for the fall semester. Only 15 more to go! ;-) And with that, my pick of the week's top 5 items of interest:

1. Microsoft has recently released Photosynth to the public. This technology was developed jointly by Microsoft Live Labs and the University of Washington a few years ago. It takes photos from various view points and synthesizes them into a 3D object when can be rotated. Warning: leaving your browser pointing to a Photosynth page will use up 50% of your processor (at least it did for me), slowing everything down.
   
   
   
   
2. The Internet Archive, Library of Congress, and a few other partners are going to archive many of the government's websites at the end of the Bush administration. In the past this has been done by the National Archives and Records Administration (NARA), but they decided it wasn't worth the effort this time around. I'm glad someone else thinks it is.
   
   
3. After less than one year, Yahoo Mash is no more. Wonder how many people are losing a year's worth of social interaction? Someday the same thing is going to happen to MySpace or Facebook, and there are going to be revolts in the streets.
   
   
4. Some interesting research done by facesaerch reveals that individuals usually google Windows and Linux on weekdays and Apple on weekends. Does this mean we "work and suffer with MS and Linux" during the week and "relax with Apple on the weekends"? Or are people just more interested in vegetation on weekends? ;-)
   
   
5. Internet Explorer 8 Beta is available. Like many others, I've made the switch to Firefox, mainly because I liked the add-on features. Apple's sly Safari install didn't convince me to switch. But I'm very tempted to give IE 8 a try. Has anyone tried it out yet?
   

Fav5

My pick of the week's top 5 items of interest:

1. Microsoft is helping researchers by providing new plugins to Microsoft Office. If you've ever used Word to write a thesis or journal article, you know what a mess it is. I hope these tools will help a little.
   
   
2. Three French journalists who work for Global Security Magazine were expelled from the Black Hat conference for allegedly sniffing the press room computer network. A little ironic that attendees can be kicked out for hacking at a conference that shares hacking info.
   
   
3. This morning I was given the chance to view the new Facebook interface. I give it two thumbs up for simplicity. I wonder though how they picked my Broncos fan box to appear on the first page and relegated my Cowboys fan box to the 4th tab?
   
   
   


4. Harding University is ranked number 1972 in the top 4000 universities in the world according to the latest Webometrics Ranking. The ranking is supposedly based on "electronic access to scientific publications" and "other academic materials".
   


5. Check out this timeline of Internet memes. (Thanks, Jonathan.)
   
   
   

Fav5

My pick of the week's top 5 items of interest:

1. With the growing popularity of the iPhone, website designers are paying much more attention to how their sites look on devices with very small real estate.
   


2. Will Microsoft bite at $33 a share for Yahoo?
   


3. We're getting just a little closer to realize quantum computing.
   


4. This will be of interest to many of my students: How much do game programmers make?


5. Last fall, Harding Univ made the move to Google Mail, freeing us from the burden of maintaining our own email system. Unfortunately, the IT guys left all of our old emails on the old system and didn't forward them on to our new GMail accounts. I just received an email saying in the next few weeks the IT guys will purge all of our emails from the old system. Anyone want to bet some very important email is about to be erased forever? (I did move my email over, but it took a few hours... I would hate to have lost emails from when Becky and I were dating.)
   

Fav5

Happy Independence Day! My pick of the week's top 5 items of interest:

1. The Deep Web is getting a little shallower: Google has recently improved their ability to crawl Flash content. It used to be that a website with a Flash interface was practically invisible to search engine crawlers, but now Google can find the links to other pages within the Flash program and even indexes the program's textual content.


2. If you were late signing up for a free Yahoo email address, you might have ended up with frank1837abc@yahoo.com since all the good names were already taken. But now Yahoo has opened up two new domains under "ymail" and "rocketmail". Although Yahoo is still the email market leader with 266 million worldwide users, they want to stay ahead of Microsoft who is a close second with 264 million.
   


3. Are Google, Yahoo, and Microsoft censoring themselves more than they must in China? According to a report by Citizen Lab, some sites are blocked by some search engines and others let them through. In a test to see how many questionable sites were blocked, Google had censored 15.2% of the sites tested, Microsoft censored 15.7%, Yahoo 20.8%, and Baidu (the most popular Chinese search engine) 26.4%.


4. A Harvard marketing professor takes on Chris Anderson's "Long Tail" theory by analyzing real data about online video rentals and song purchases. Her conclusion is that maybe we aren't as individualistic in our taste as Anderson suggested.
   


5. In the spirit of July 4, read about how LANL scientists are making fireworks a little "greener".

Fav5

My 5 items of interest for the week:

1. This is very cool: Google News now shows quotes taken from news sources. What has John McCain been saying recently?


2. Udi Manber, VP of search quality at Google, answers 20 questions about web search. (I made my search engine class read this.)


3. Researchers at University of California, Santa Cruz are working on long-term archiving using disks rather than tape. Their system, Pergamum, "is a distributed network of intelligent, disk-based, storage appliances that stores data reliably and energy-efficiently."


4. Video games are becoming a big business ($9.5 billion last year in the US), and universities are listening, creating degree programs for game developers.


5. Microsoft worked so hard to get their Office Open XML (OOXML) standardize accepted by ISO. And now the bad news: Office 2007 doesn't produce documents that adheres to their standard. My guess is the next service pack changes the file format so it does conform.
   

Fav5

My pick of the week's top 5 items of interest:

1. This is why I'm not an early adopter: HD DVD will be joining Betamax as another technology that lost a format war. Sorry, Microsoft, but this time Sony is on the winning side. Anyone want a $50 HD DVD player? Cheer up... maybe the Micro-hoo thing will still work out...
   


2. Building and deploying a 5000 pound spy satellite: $100 million. Destroying the same satellite because it never worked: $60 million. Watching all that tax payer money explode in space: priceless.
   


3. An interesting study has been done by Peter Gloor at MIT. He mines Google for the web graph associated with the name of a political candidate or a company and uses it to predict election results and stock prices. I wonder about the effectiveness of this technique though since it can easily be corrupted by individuals creating frivolous links and because Google does not report all the backlinks it is aware of.


4. Microsoft has just made a brilliant move: they are now giving away development tools like Visual Studio and SQL Server to college students. This is an excellent way to get universities to adopt MS tools since the financial considerations are now a mute point.


5. It seems that my students aren't the only ones tempted to copy-and-paste from the Web. The Wall Street Journal reports that plagiarism on social networking sites is on the rise:
   

   In a recent survey of more than 400 online daters commissioned by Engage.com, 9% of respondents said they copied from another person's profile; 15% suspect their own words were stolen.

   
   

Fav5

My pick of the week's top 5 items of interest:

1. Google announced a few weeks ago that it has decided to take-on Wikipedia and at the same time give authors a share of the advertising revenue. They propose authors create knols, articles of expertise on a variety of subjects. This article on Wikipedia does a good job of summarizing the potential problems faced by Google's proposition.
   


2. Researchers in Israel have created the nano-Bible, engraving 300,000 words in Hebrew on a chip the size of a grain of sugar.


3. In the world of digital preservation: "Mike Wash, chief information officer at the Government Printing Office, expects GPO to have more than a petabyte of content available in five or 10 years." That's a lot of data to ensure long-term access to.


4. A story on CNET says that Microsoft's latest Office 2003 service pack blocks older file formats from being loaded. Microsoft did this supposedly for security reasons, but they didn't tell anyone until just recently. Hope you didn't have any old Word 6.0 and Word 97 files sitting around that you might need access to someday...
   


5. Life imitating art: Did you see The Office episode where Michael drove his car into a lake because the GPS told him to? Just a few days ago, a NY man drove his car onto a train track on the advice of his GPS. Thankfully he was able to escape from the trapped car before an on-coming train smashed into his car going 60 MPH.

Fav5

My pick of the week's top 5 items of interest:

1. The academic community has shot themselves in the foot once again: Two researchers from UT Austin have written a paper showing how to de-anonymize the anonymous 100 million+ movie ratings Netflix made available to the public in 2006. While this is an impressive feat, it will likely make companies that have access to rich data sets much less likely to release them to researchers in the future. While this may be a positive thing for privacy advocates, it's certainly not for researchers. What these UT guys have done in effect is demonstrate why academia should not be trusted with data they desperately need to do their research.
   
   Update: A friend of mine emailed me about this issue and made some good points: Maybe Netflix should have offered some prize money to figure out how to properly release anonymized data. Also, researchers have an obligation to show how others might misuse "anonymous" data... companies making such data available to the public should show much more care in the future.
   


2. We're all carrying ticking time bombs in our pockets... UPDATE: Or maybe not.
   


3. More bad news for Microsoft: a recent benchmark test revealed XP was twice as fast as Vista. And in court, MS's marketing director was just as confused about what "Vista capable" means as the rest of us.
   


4. Philipp Lenssen has written an interesting article about Abstract vs Realistic Icons, making some really good points about how making icons more photo-realistic doesn't necessarily improve usability.
   


5. Colin N., a former CS student, emailed me about a new games contest sponsored by Microsoft for students. I'd like to change the was the CS software development class works by targeting this competition next year.

Fav5

My pick of the week's top 5 items of interest:

1. A humorous take on What If Gmail Had Been Designed by Microsoft? (Thanks to Mike Baur for sendimg me that one.)
   


2. While we're on the subject, Gmail now supports IMAP. That means you can now use email clients like Outlook and Firefox to access your Gmail without popping the emails off of Google's servers. Had Google supported IMAP several months ago when the Harding faculty was forced to switch to Gmail, it would have made many lives much easier. I was abile to get it to work with my harding.edu email address following the instructions and using user@harding.edu for the username.
   


3. If you like Google, Java, and mobile devices, you may want to give Android a shot. Reto Meier has written a little how-to on developing a person-mapping application with Android. Non-developers may not be quite so fascinated with Android.
   


4. Playing catch-up: an interesting article pointing to 12 research projects aimed at reducing spam.
   
   
5. Congrats to the Dallas Cowboys on a best-ever record of 10-1 after defeating the Jets 34-3 on Thanksgiving. Next victim: the 10-1 Green Bay Packers. Also congrats to the Razorbacks on an amazing win in triple overtime over the #1 ranked LSU Tigers. The Cowboys really need to draft McFadden next year...
   

Fav5

My pick of the week's top 5 items of interest:

1. Evidence that studying really does matter: A recent study finds that college freshman who have a roommate with a video game console spent an average of 40 minutes less time studying per day than those freshman whose roommate did not have a game machine. They also scored 0.241 points lower (on a 4.0 scale) at the end of their first semester. It's not that the games were causing the students' minds to waste away... they were simply eating into the time the students would normally have spent studying.
   


2. The digital smiley face :-) turned 25 a few days ago. Carnegie Mellon University professor Scott E. Fahlman claims to have been the first to use it in a computer message, and no one else has refuted that claim. Of course, proving you were the one to have invented the happy face is a difficult task; it would be rare for anyone to have access to a personal digital message from over 25 years ago.


3. Kudos to the New York Times who recognized the potential to greatly increase revenue by unlocking their subscription-based content to the public.
   


4. A Microsoft employee prematurely posted upcoming changes to Search Live on a blog. LiveSide caught it before it was pulled by MS.


5. And finally, Microsoft lost big in Europe: $700 million and some change.

Fav5

My pick of the week's top 5 items of interest:

1. This morning one of our senior computer science students did their senior seminar on Silverlight, a new web technology developed by Microsoft to enable .NET client-side programming on the Web. Like the Flash player, Silverlight first requires the engine to be downloaded and installed. I got it to work just fine with Firefox (although I hate restarting my browser, especially when I have 20 different pages open). This technology looks promising, especially if Microsoft really pushes hard for adoption. I wonder why they don't include it along with Flash in Vista? Maybe they're afraid of another bundling lawsuit.


2. One of the examples used in the Silverlight presentation was Microsoft's new search engine, Tafiti. Tafiti is powered by Live Search, but it uses Silverlight for it's interface. Tafiti is a perfect example of how a pretty interface does not necessarily make your application any more usable (see the tree of search results below for "Harding University").
   
   
   P.S. I noticed today (3 days after this post) that the Tafiti tree view screen was eating 25% of my CPU cycles! I opened another tree view window to view different results, and 50% of my CPU cycles were being used! So not only is the screen useless, it will also cause your computer to waste time and energy. Booo.
   


3. Google has recently improved their date range functionality on the advanced search. These improvements allow you to do things like, find the pages Google crawled from my blog within the past week.
   


4. And Google is offering $30 million to the first team able to send a robotic rover to the moon that is capable of roaming the surface and sending readings back to Earth. Of course accomplishing such a feat is going to cost much more than $30 million, but it's a nice gesture.
   
   
5. And just for fun: Sidesplitting tech comics.

Fav5

My pick of the week's top 5 items of interest:

1. Facebook has recently decided to allow search engines like Google to crawl their member pages to make them accessible to a much larger audience. For those that don't want their profile exposed to the world, Facebook provides an opt-out. This is a good move by Facebook that will likely draw in more members, and it will certainly be useful to social networking researchers.
   


2. While on the subject of Facebook, have you seen the Friend Wheel? It creates a graph of your friends and connects your friends who are friends with each other. A really dense graph means your friends are very cliquish. A sparse graph would mean you have friends from a very broad range of people. Looks like I'm somewhere in the middle:
   
   


3. The first Semantic Robot Vision Challenge was held in late July. The competition required robots to locate objects in a room by first utilizing the Internet to search for pictures of the objects. For example, if the robot had to find a pencil, it would first search Google Images for pencil and then, based on the results, attempt to locate something similar in the room. I've tried teaching Ethan what "sleep" is using a similar method, but he hasn't yet been able to locate it anywhere in his room.
   


4. An interesting paper popped up in arXiv this week: Open Access does not increase citations for research articles from The Astrophysical Journal. The authors conclude:
   

   There are a number of excellent arguments in favor of changing the scientific publication system to an open access model. The open access citation advantage is not one of them.

   Why doesn't making a paper OA increase its citations? The authors admit in the introduction:
   

   ...in a well funded field like astrophysics essentially everyone who is in a position to write research articles has full access to the literature.

   So, if everyone has paid access to everyone else's papers, there is no OA advantage. I don't think anyone would have thought otherwise. The problem is, most disciplines are not well-funded enough to provide everyone paid access to the literature, and hence the OA movement. So the conclusions reached by the authors seem trivial to me. Am I missing something?
   


5. Is Apple the next Microsoft (monopolist)? Dan Frakes doesn't think so.
   
   

Fav5

1. Thanks to Mike Baur for sending me a link to this video of some really cool image visualization software being developed at Microsoft... I especially liked the aggregation feature which pulled images from Flickr together to form a complete image. If you thought the video was cool, you may also be interested in these other videos from the TED conference.
   


2. Vote for your favorite 1980’s arcade game. Although I genuinely loved most of the games on the list, I had to go with Tron.
   


3. There’s a good article from the New York Times about Google’s continued focus on search: Google Keeps Tweaking Its Search Engine. Matt Cutts also put in his 2 cents on about Google’s search focus.
   


4. Researchers are finding that bullying in the virtual world is becoming quite a problem, especially for newbies.


5. Ask.com has received a makeover. Ask3D is their new approach at integrating various vertical search results together in a single results page. Nice.

Web Curator Tool, standardizing PDF, and orphaned works

Some notable events in the world of digital preservation:

• The National Library of New Zealand and the British Library have collaborated to produce the Web Curator Tool (WCT), a tool that allows non-technical users to archive websites in a simplified manner. It’s essentially a wrapper around the Heritrix web crawler with numerous management functions added on. In a recent article, Philip Beresford from the British Library discusses the history of WCT and shows how it can be used to crawl and archive a website.
  
  
• In an effort to convince the world that the PDF format is ideal for long-term storage, Adobe is submitting it to ISO for standardization. Microsoft has also submitted their Ecma-approved Office Open XML for standardization to ISO, a radical departure from the "secret-sauce" mentality Microsoft has held for years. Governments and other organizations are slowly becoming aware of the problems created by storing their data on closed formats that change over time, and Microsoft and Adobe don’t want to be dropped from their largest customers. By standardizing these formats, interoperability should be much less of an issue in the future.
  
  
• Brewster Kahle, co-founder of the Internet Archive, recently lost a U.S. appeals decision in Kahle v. Gonzales. Kahle, along with several notable companies like Google, MSN, and Yahoo, are trying to get orphaned works (copyrighted work whose owner cannot be reached) into the public domain in order to remove legal barriers that prohibit the scanning and digital distribution of those works. Kahle rightly blames Disney for the mess:
  

  What happened is that some overzealous copyright laws got passed with heavy lobbying from folks like Disney and these are screwing things up... Instead of keeping just Mickey Mouse or just the profitable works under copyright for longer, they fundamentally changed the structure of copyright.

  
  
  

Wikipedia: nofollow and noMSedit

Some new news from the world of Wikipedia:

• Minor: All external links from Wikipedia are now using the NOFOLLOW attribute. This attribute tells web crawlers like Google that the link has not been vetted, so it will not be used in their algorithms to artificially bolster the ranking of some pages. Wikipedia’s action will seriously reduce the amount of link spam that currently plagues many entries.
  
  
• Major: Microsoft has attempted to hire Rick Jelliffe, chief technology officer of XML tools company Topologi Pty. Ltd., to “correct” Wikipedia entries on ODF (OpenDocument format) and OOXML (Microsoft Office Open XML). You can see Rick's original post about the offer here. Apparently Wikipedia is keeping Microsoft employees from making the edits themselves, so Microsoft thought a third party could update the entries that apparently shed a negative light on Microsoft’s format. This astroturfing blunder has created quite a few waves.