Clash of the Titans

Yesterday I attended the fourth annual Clash of the Titans event at Regent University. Bill O’Reilly moderated between former Israeli Prime Minister Ehud Barak and Palestinian Spokesperson Dr. Hanan Ashrawi as they debated the question: "Can Military Force Bring Lasting Peace to the Middle East?" All three participants did a really good job of conveying their point of view without being unnecessarily rude. In fact, Barak claimed that if Ashrawi was in charge of the Palestinians, there would be peace in the region.


The crowd was quite enamored with O’Reilly when he first came out on stage; I thought a few of the audience members looked as if they’d died and gone to heaven. And both Barak and Ashrawi received equal applause. There wasn’t any booing this year unlike the last debate, but I suppose that’s because last year’s subject, the war in Iraq, hit so close to home, and there was a large military presence in the audience last year.

I think Ashrawi did a really good job of conveying to the audience the Palestinians viewpoint. Both Barak and Ashrawi admitted the only solution to the current situation was the two-state solution, but neither thought this solution would come anytime soon, especially with Hamas ruling in Palestine (Ashrawi was very careful to avoid criticizing Hamas, stating they were in power due to the protest vote).

Barak stated that he wished Israel could be located somewhere else in the world. America has it pretty good since they aren’t situated in the middle of a hostile group of neighbors. Barak told an old joke about Moses that went something like this:

Moses stuttered because he was not an eloquent man. God asked him where he wanted to settle and Moses said, 'Ca-ca-ca-ca,' and God understood he meant Canaan. But Moses was trying to say 'Canada.'

Reconstructed www.survivorsunited.com

Gina Jones at the Library of Congress tipped me off to another missing website: www.survivorsunited.com. The LOC had crawled this site last August as part of the Darfur Collection. According to the Yahoo directory, this site is about an

Organization formed to help women who were victims of rape or sexual assault in the Darfur conflict. Site provides news, timeline, collection of related documents, and calls to action.

I have reconstructed the website and made it available here.

Google Code Search

A few weeks ago Google launched Google Code Search which allows you to search through source code that Google finds while crawling the Web. Like Google Scholar, this promises to be a really useful tool for searching specifically for resources that you want while ignoring all the other garbage that’s out there. It's also another small step into pulling the deep web up to the surface.

Of course this also opens the door to embarrassing numerous programmers and organizations and exposing many security holes by allowing you to search for code vulnerabilities, usernames/passwords, backdoor passwords, etc. Google has acknowledged these potential problems but maintains that all tools can be used for good or bad purposes and that the good far outweighs the bad in this case. Kudos for Google not caving into the whiners.

My own vanity search revealed that Google has only indexed a very small amount of code that I’ve made available on the Web. (Of course not all the code I’ve made available has my name on it.) I am really surprised Warrick hasn’t been indexed yet considering the code has been available for quite some time. It’s possible that switching the URL where the source code is located when the version changes it is causing Google to be a little skittish.

Back from the Bahamas

Becky and I returned Saturday morning from our cruise to the Bahamas. Our room on board was spacious, and we loved getting breakfast in bed and twice-daily room service. We had a great time snorkeling in Nassau and the Atlantis hotel, and the beach in Freeport was incredible. I celebrated my birthday onboard and got to blow out candles on a cake of Baked Alaska.


Besides getting to spend a lot of time with Becky, I also had plenty of time to read. I brought along a book called The Language of God: A Scientist Presents Evidence for Belief by Francis S. Collins, the head of the Human Genome Project. Collins does a fantastic job at presenting how science and faith can be and should be integrated. Collins is an excellent communicator, and he especially does a good job of summarizing the current state of scientific knowledge. I highly recommend this book to the atheist, agnostic, and believer. I found the Language of God and Finding God at Harvard (which I just completed last month) to be very encouraging to my faith.

Now it’s back to the grindstone...

Storm before the calm

Becky and I are going on our first cruise next week. We depart Norfolk tomorrow for Nassau and then to Freeport. This was supposed to be the "celebrate finishing the dissertation" cruise that we were going to take next summer, but the baby changed all that. Becky just finished her first trimester, so she should be feelin’ good for the cruise.

I’m glad to be getting out of town... yesterday a nor’easter came crawling into town. Combined with the high tide and full moon, my neighborhood was flooded just as bad as it was when Ernesto came through. I took these photos this morning.

Colonial Place mermaid	Apartment building next door
House next door	Our parking lot

Mark Foley websites recovered

Mark Foley, a Congressman from Florida, resigned on Sept 29, 2006 over allegations of inappropriate emails to minors who worked as Congressional pages. It was brought to my attention on Tuesday (thank you Martha!) that his websites

http://www.house.gov/foley/

and

http://www.markfoley.com/

were both shut off after the resignation. I have reconstructed both sites using Warrick and made them available here.

Become.com's web crawler

Today a member of the Heritrix list serve pointed everyone to an article on Sun’s website that discusses Become.com’s web crawler. The article dates back to August of 2005, so it’s a little dated. I couldn’t find any updated information on the crawler, but apparently it is proprietary, and the source code will likely never see the light of day.

Become.com actually developed 2 crawlers in 2004- one written entirely in Java and the other mostly Java with some C++. The article states that the crawlers "may be the most sophisticated, massively scaled Java technology application in existence."

The article doesn’t mention anything about Heritrix, a crawler which is also completely written in Java. Although Heritrix doesn’t currently have a distributed architecture, it could still be deployed in such an environment. It would be really interesting to see the two crawlers compete at the National Java Crawling Championships.

Software Engineering: Best Job in America

Money Magazine has listed software engineering as the number one Best Job in America. Even with the threat of off-shoring jobs, a computer science degree isn’t looking so bad after all. In fact, of the top 10 jobs listed, 4 of them could be obtained by someone with a CS degree. Having been both a software engineer and now a college professor (number two on the list), I can attest that both jobs are very rewarding.

Club a la 700

This morning I sat in the audience during the taping of the 700 Club, the daily news show hosted by Pat Robertson, Gordon Robertson, and Terry Meeuwsen. The 700 Club is filmed at the CBN headquarters which is located on the Regent University campus in Virginia Beach. The show is broadcasted daily on several cable stations.

When Becky and I first moved to Virginia, our tourist book mentioned that the 700 Club was filmed in Virginia Beach, and I really wanted to see it being filmed. Becky thought that I was crazy. I’m not a frequent viewer of the 700 Club, but I’ve always wanted to see some TV show being filmed live, and this is the first opportunity I’ve ever had. Living in Searcy, Arkansas doesn’t present many opportunities. Since I’ve got tons of spare time these days, I figured I could take off one morning to see the taping.

I was only one of five audience members (apparently the audience size fluctuates from day to day). Pat came out before the show began and said a quick prayer for us and the show and then joined Terry who was already positioned at the news counter.

The show began with comments on the recent Pope vs. radical Islamic spat. There was a piece about Iranian president Mahmoud Ahmadinejad and then talk about Iraq. The show was mainly scripted, but Pat made several remarks off the cuff. I believe it’s these types of remarks which have gotten him in trouble in the past. After the piece on Ahmadinejad which focused on his desire to bring about the end of times, Pat exclaimed: "Well, that is just weird..."

Dr. Kevin Leman, a prominent Christian psychologist and author, was the single guest. My mom would have liked to meet Dr. Leman since she’s a big fan of his Birth Order Book. He was interviewed by Terry and talked about his new book on single parenting, but as soon as the interview was over he bolted. I guess I’ll get that autograph some other time.

After the taping Pat thanked us for coming, and Terry hung around and talked to each of us. We then were given a tour of the CBN building. It was like touring a museum since there were numerous pieces of original art including the largest painting of the Lord’s Supper produced in the 20^th century. Our tour concluded with a very earnest prayer (which I really appreciated). After that I met up with Becky for lunch.

I really enjoyed getting a behind-the-scenes look at how a TV show is ran. The guys at CBN are real pros. Now it's back to the grind...

Our bean has a heart!

This morning Becky had her second visit to the obstetrician. Since we were going to hear the baby’s heart beat for the first time, I got to tag along. At just 11 weeks, our little bean (ok, he’s probably the size of a walnut now) has got a strong heart rate!
wucca-wucca-wucca-wucca-wucca... Perfect for a future tennis star...

Thinking about next fall

The fall semester has just begun, and I again find myself wishing I was back in front of a classroom. As I contemplate returning to teaching next fall (God willing), there are several things I’d like to do differently. Here are three nuggets I have come across recently that have gotten me thinking about next fall:

1) Pair Programming

Pair programming is a relatively new way of teaching students how to program in CS1 classes. I first learned of pair programming in a recent CACM article entitled "Pair Programming Improves Student Retention, Confidence and Program Quality" by McDowell et al (2006). McDowell has written on the topic since 2002 and even provides a video on how to teach pair programming.

In McDowell's experiments, they found that more students completed their course, were more satisfied with their work, and stuck with the CS major than students that had to program independently. They also found that the paired students performed similarly on the final exams which means the paired students were learning to program just as well as the non-paired students. The article also discusses how this could be useful for retaining women in CS.

Like every other university, Harding has seen a downward trend in the number of CS majors over the last several years. I think paired programming may be one of the tools we could use to assist us in building back up enrollment. I’m excited to try it in my introduction to programming classes soon.

2) Programming with Alice

Becky recently met an instructor who was teaching middle school and high school teachers about Alice, an alternative programming language which teaches programming by manipulating objects in 3D space. When the instructor leaned that Becky’s husband taught programming courses, he handed her an intro to Alice textbook which she passed on to me.

Alice was developed at Carnegie Mellon, but it's soon to be overhauled by Electronic Arts which will give it a really crisp look that the XBox generation has learned to expect. Alice allows students to quickly get an animation running without the boring details of variable declarations and semicolons.

I’d really like to teach Alice to a group of students who have no programming background and see how they do. If I can get them excited about Alice, I may be able to get them excited about computer science.

3) Limit wireless Internet access during class

Dennis Adams wrote a fantastic Viewpoint piece entitled "Wireless Laptops in the Classroom (and the Sesame Street Syndrome)" in the September issue of the CACM (2006). Adams, a professor at the University of Houston, opened the article with two examples of “laptops gone wild” in the classroom: a 2002 school brochure which inadvertently contained a photo of a student playing Solitaire while his unaware professor lectured, and 2) a Wall Street Reporter who witnessed several students using chat rooms, IM, and surfing the Web while Adams lectured.

Adams rightly points out that professors need an off button- a way to turn off wireless access or at least limit access during class. Professors cannot be expected to compete with Google, IM, Solitaire, FaceBook, et al. during their lectures or provide constant infotainment. No professor, no matter how good, can compete with the infinite number of distractions that the Internet places just inches in front of a student. Adams is not alone in his assessment.

I have personally used embarrassment tactics as a means to ensure only proper use of laptops while I lectured, and I’ve had some modest success. I called out one of my students who was using MS Paint to create a picture while I lectured, and every day after that he sat very alertly. I’ve been fortunately enough to teach to small classrooms where it’s easy for me to roam and see what my students are doing, but such a strategy certainly doesn’t work in large classroom settings.

Someone is going to create some software that allows only limited Internet connectivity during class, and that person is going to make a killing. And of course the guy who invents the software to circumvent the policing software is going to be the next Shawn Fanning.

Remembering 9/11

Today is the five year anniversary of the 9/11 terrorist attacks. I didn't know anyone personally who was lost in the attack, but like everyone else, it affected me deeply.

My friends and I were talking over lunch yesterday about how we first found out about the attack. I was listening to Fox and Friends while shaving and getting ready for a day of teaching (it was my fifth year teaching at Harding Univ). I remember one of the hosts saying that they thought a plane had struck one the Twin Towers. I ran over to the TV and couldn’t believe the site... what kind of horrible pilot would make such a huge mistake? I watched for a little while and then witnessed the second tower being hit; this was no accident. There wasn't anyone around to talk to about it (my roommate had already left), so I raced over to the office and found Dana Steil (a colleague of mine in the CS department) who hadn't heard about it. Dana turned on the radio, and I just sat at my desk wondering what was going on. A few minutes later we went to chapel, and I think I remember President Burks announce what was happening, and we prayed for quite a while about the situation. I don’t really remember what happened the rest of the day... it’s a fog.

Although I normally avoid the many 9/11 shows and movies that come out near the anniversary, I decided to see the World Trade Center (starring Nicholas Cage) this weekend after several friends had recommended it. Becky and I were genuinely moved by the courage of the men and women who ran into a huge disaster just to complete strangers. I especially liked the Dave Karnes character who, feeling that God was calling him to action, raced to Ground Zero from his home in Connecticut to search for survivors after everyone else had called it a day.

It’s tough to watch parts of this movie, but you will definitely leave the theatre knowing that courage and goodness are not in short supply in America.

May God bless and comfort those who are still deeply scared by these attacks.

Google's cached date = last request date

Vanessa Fox, a member of the crawl team at Google, announced on Tuesday that Google would start posting the last request date on their cached pages. Google used to only indicate the date that the page was last retrieved, so if Google made an If-Modified-Since request and the web server responded with a 304 (not modified) response, the cached date would be left unchanged. Now the cached date will indicate the date of the 200 or 304 response. Matt Cutts also discussed this change and even made a little video for those that needed a visual explanation.

Frankly, I was very surprised to learn that Google’s cache date worked this way. In effect, it’s was much like Yahoo’s Last-Modified date… it was really just an indication as to when they noticed the page changed. I have crawl data from 2005 that indicates Google would periodically issue regular HTTP GET requests, possibly just to verify that the content had indeed not been changing.

I’m not totally sure what MSN’s cache date is indicating. From my 2005 crawl data, MSN apparently never issued an If-Modified-Since request. If they are still operating with the same crawl policy, then they are storing the time they last contacted the web server, so their cache date would indicate the same thing as Google’s.

What this means for Warrick: Google will more frequently now have the most recent version of a page. Therefore Google’s overall percentage of contributed resources will likely increase in the reconstructions I’ve been performing the last few weeks.

On a side note, someone asked Matt Cutts why Google does not post the cached date of PDFs, and Matt said he was going to ask the crawl team about it.

Rainy Friday

The Hampton Roads area is currently being doused by tropical storm Ernesto. Regent, ODU, and practically every school in the area has shut-down due to the flooding. I attempted to drive over to ODU just for fun, but I was stopped at every turn as the photos belowillustrate. It's just the beginning of a beautiful Labor Day weekend. :)


. . .

View it while you can: Microsoft UK commissioned Ricky Gervais and Stephen Merchant to produce a couple of spoof training videos with Gervais playing David Brent, the boss from the UK’s The Office series. For some reason, Microsoft apparently didn’t want these videos to ever be shown publicly and have started an investigation as to how the videos were leaked. The videos still remain on Google Video, so watch them while you can… they are hilarious!

. . .
Agassi- you are a stud. How far can he go on his final tournament stop, the US Open? More importantly, can I get Becky to go along with naming our future son (if it’s a boy) Agassi McCown?

. . .
Thanks, Elrod, for taking some photos of the new chapel at Harding. It looks great!

. . .
Congratulations to Becky on being awarded Regent University’s Employee of the Month. She received a choice parking spot for September, a little spending money, and a letter of recommendation signed by Pat Robertson (you can’t have too many of these sitting around the house). Becky also was awarded the School of Education’s Staff Member of the Year last May.

An employee will go home and ask his neighbor, "Hey, did you get an award?" "No man. I mean I slave all day and no one notices." Next thing you know, he smells something funny from his neighbor's house. Neighbor hanged himself due to lack of recognition. - Michael Scott (The Office)

Bayside League: 2006-2007 season

Saturday night was the Bayside League’s third annual fantasy football draft. Mark Velez was kind enough to host it at his place. There were 14 teams- just 2 shy of having 2 leagues. Chris Deny, last year’s champion, drafted by phone, and I drafted my dad a team.

I got to draft second, and since Payton Manning was taken first, I was able to nab who I hope will be the best running back this year in the NFL: Larry Johnson. I was also able to draft a few Broncos and Cowboys- I just can’t help but play favorites.

Last year I came in second place. Let’s see what I can do this year…

1. RB – Larry Johnson
2. RB – Willis McGahee
3. QB – Jake Plummer
4. WR – Donald Driver
5. TE – Chris Cooley
6. WR – Andre Johnson
7. DEF – Cowboys
8. K – Josh Brown
9. RB – Mike Bell
10. QB – Mark Brunnell
11. WR – Nate Burleson
12. K – Lawrence Tynes
13. DEF – Patriots
14. TE – Desmond Clark

Funny Commercials

One of my hobbies is “collecting” commercials. I know that sounds odd. What I mean is I am a big fan of commercials, and I like to download or keep “pointers” to commercials that I think are really hilarious, entertaining, or just plain fantastic. Google Video and YouTube are two of the best sources for commercials. In fact my favorite commercial this year is the Liberty Mutual “pay-it-forward” commercial which is now available on YouTube:



When we got home from church last night, we caught the “World’s Funniest Commercials” on TBS. In general I was disappointed by the crassness and message of many of the commercials, but there were a few gems:

LA Country Fair – "Duh, Ashley, all wool comes from a cow..."
1-800-Got-Junk – “Rat Advertising Trial (R.A.T.)”
Avis: Gansta Rap – “Gotta get that money made”
Solo Mobile: Housewarming – “You are a legend”

Honorable mention: Pacman Puppet Show

All the commercials can be seen in high quality at veryfunnyads.com.

The ACLU and nonsectarian prayer

Last week, a federal judge dismissed a Fredericksburg City Council member's lawsuit challenging the council's nonsectarian prayer policy. Ironically, the ACLU of Virginia is fighting in this case to limit the liberties of an individual rather than protect them. It’s not quite so surprising when you learn their opponent is a Christian who desires to use the name “Jesus Christ” in an opening prayer for the city council.

A lawyer friend of mine who is intimately familiar with this case offered the ideal ACLU prayer, guaranteed to be as nonsectarian as possible:

"At this point we would like to call on a genderless, nameless higher power than ourselves and invoke that being (or those beings) intervention (or non-intervention as the case may be) upon this governmental body."

If it ever does come down to that, most of us would rather have no prayer at all. That’s exactly what the ACLU is hoping as well.

Hypertext 2006

Michael is in Odense, Denmark this week presenting two papers at the Hypertext 2006 conference:

1. Evaluation of Crawling Policies for a Web-Repository Crawler by McCown and Nelson
2. Just-In-Time Recovery of Missing Web Pages by Harrison and Nelson

I should be there presenting the crawling policies paper, but Michael graciously went in my place so I could stay here and work. Our papers are 2 of the 12 that will be presented.

Lazy Paper accepted to WIDM'06

I got some really good news yesterday- my Lazy Preservation paper (“Lazy Preservation: Reconstructing Websites by Crawling the Crawlers”) was accepted for publication at the WIDM’06 workshop along with Joan’s mod_oai paper. Only 11 of the 51 submissions (21.5%) were accepted which means the workshop participants are going to be getting very familiar with the research aims at ODU. ;-)

WIDM is in Arlington, Virginia on November 10 and is held in conjunction with CIKM’06. This will be a good opportunity for me to visit my sister again in D.C. I was up there a few weeks ago giving the good news about Becky’s pregnancy to my parents (they were visiting from St. Louis). Random event: while I was in D.C., my mom and I ran into Karl Rove at a bookstore- we chatted for a while (Mom’s a fan), and he seemed like a nice guy, but I'm sure most politicians do. Anywho, I’m looking forward to the workshop.

Google Analytics

This morning I installed Google Analytics on my blog and ODU website. It’s a free tool that allows me to track how users enter, leave, and navigate my website. It involved simply posting some JavaScript (below) on the pages I wanted to track:

<script src="http://www.google-analytics.com/urchin.js" type="text/javascript">
</script>
<script type="text/javascript">
_uacct = "UA-######-#";
urchinTracker();
</script>

It’s going to take a few weeks before there’s any data collected, but I’m really curious to see what this will reveal about the popularity of Warrick since I don’t have access to the CS web server logs.

Update on 8/25/06

It's been a week, and I'm now able to see some analysis of my blog and my ODU cs website. The screen below shows a summary of my blog's traffic from Aug 18-24:


Visitors increased from 11 to 26 during this week, and pageviews ranged from 25-49. Three quarters of the traffic are new visitors (Google is using cookies to track this).

The Geo Map Overlay is fascinating. My blog tends to appeal more to Americans and Europeans: I got only 2 visits from Australia, 1 from South America, and 1 from Africa. There were 14 visits from Tampere, Finland and 9 from Nokia (also in Tampere). The Visits by Source graph shows the Finish hits to be from Timo's nothingforsale.com website where I now have a link pointing to my blog.

Most people find my blog through Google. So what are people searching Google for that lands them on my blog?

Apparently my blog entry about Yahoo's error 999 is by far the most popular. Searches 1, 4, 5, and 9 will all return this entry in Google's top 10 results. The "shiri maimon" entry doesn't show up in Google until page 3 (top 30 results).

What pages are referring visitors? Apparently Elrod's blog is the biggest referrer so far. This is due to a comment I left on Aug 18.

What really surprises me is that anyone is visiting frankmccown.com, a website with no content. I did a search for "frank mccown" in Google, and the site came up number one. Come on Google... you guys are supposed to be punishing content-less sites like this, not promoting them. I guess Google's PageRank is far from what they originally published in 1998; there's maybe 1 or 2 links to this page from anywhere on the Web.

My cs website is getting a little more traffic than my blog even though I only have the tracking on a few of the pages. What was most interesting to learn was that Wikipedia was by far the largest referrer, sending me around 60 referrals last week. Most of the referrals are coming from the Internet Archive entry where there's a pointer to Warrick.

The Warrick page received 125 visits and 175 pageviews last week (18 and 25 per day, respectively). Here are some search terms people are using to find Warrick:

• google api convert documents to HTML
• Warrick
• Warrick website download
• Warrick archive.org
• warrick perl
• cached archive website
• recover website from google cache
• google cache website recovery