Orion posted an interesting thread at SEW, citing this fallacies of relevance page. The SEW thread also has some good posts by other members & looks to be shaping up into a great thread.

Orion stated that he did not think current systems could yet grasp relevancy fallacies.

When you are trying to win an arguement, if you use any logical fallacies make sure you use these 38 sure fire techniques. <-- amazing resource!

Berkeley has been recording lectures from some of the best minds in search. So far some of the videos include Norvig, Battelle, & Brin. Gary posted a bit about Sergey here.

I am not sure what the problem was, but my connection kept breaking in the middle of the shows, which is annoying. They have a wide variety of Podcasts available here.

If you like patents try here, here, and here.

Google's search quality evaluation process site may have been around for years.

SearchBistro recently posted a 22 page PDF titled General Guidelines on Random-Query Evaluation that was last revised on December 31, 2003. In addition to posting the Random-Query Evaluation PDF, Henk van Ess has recently posted:

• examples of offensive (or low quality) sites

• some whitelisted sites:
  

  Here is a non-exhaustive "white list" of the sites whose pages are not to be rated as Offensive (nor as Erroneous):
  Kelkoo, Shopping.com, dealtime.com, bizrate.com, bizrate.lycos.com, dooyoo.com;

• tips for rating sites:
  

  If it's a machine-generated, no added value affiliates, it's Spam. If it provides some unique values, for example, customer feedback, local information, it should be rated on the merit scale even if it has some affiliates. Similarly, if the game site allows you to download a game, without being intrusive (i.e. install a spyware without notice), it should be rated on the merit scale, instead of Spam.

• How reviewers communicate to come up with solutions when review quality scores are far apart from one another

Search Classification Types:
The Google review guide classifies searches as being

• navigational (example: a search for United Airlines)

• informational (example: how do I..)
• transactional (example: buy 18K White Gold Omega Watch)
• any mixture of the above categories.

Resource Quality Rating:
Google then asks raters to classify sites listed in random queries using the following categories:

• Vital
  • most queries, especially generic type queries do not have a Vital result.

  • Vital result example: search for Ask Jeeves returns www.ask.com.
• Useful
  • these should have some of the following characteristics (although it likely will not exhibit all of them): comprehensive, quality, answers the search query with precission, timly, authoritative.

  • This is the highest rating attainable for most pages on most search queries.
  • Useful result example: search for USA Patriot Act returning the ACLU page covering the USA Patriot Act.
  • For some plural queries, such as Newspapers in Scotland, the best results may be lists of related sites. Reviewers must also check some links on the page to ensure the page is functional.
• Relevant
  • One step down from Useful. Relevant results may satisfy only one important facet of a query, whereas Useful results are expected to be more broad and thorough.

  • Results that would have been Vital if a more common interpretation did not overshadow it are considered relevant.
• Not Relevant
  • Not Relevant results are related to the topic but do not help users.

  • If a person searching for Real Estate finds a San Diego Real Estate website that would probably not be relevant since most people searching for that do not live in or want to move specifically to San Diego.
  • As the San Diego example is too narrow geographically other sites could also be too narrow in other non location based ways, such as being outdated or too specific to a subset idea of the query.
• Off Topic
  • Is not a useful page. Irrelevant.

  • Usually occurs when text matching algorithms do not account for some terms that can have multiple meanings.
• Offensive
  • Pages or sites that often do not hold merit on any query.

  • Example Offensive sites: spyware, unrequested porn, AdSense scraper and other keyword net type sites, etc.
• Erronious
• Didn't Load
• Foreign Language
• Unrated

Vital to Offensive are in order of quality. The higher the better. Erronious through Unrated are cast as non votes. When in doubt between rating values raters are expected to rate at the lower of the two rating values.

Why this is Important:
By learning how and what they want evaluators to look for it makes it easier to understand how to deliver what the search engines want.

This post was a quick review of General Guidelines on Random-Query Evaluation. If you are heavily interested in SEO it is well worth your time to read the original document, which lists many more examples and is in far greater detail than this post.

Random Thoughts:
With how relatively low the wages are for these positions ($10 - $20 an hour) you have to wonder:

• why it took so long for this information to come out

• if some of these people are using the information they gained from participating in other ways
• if these people know anything about Google's business model, and how much THEY could be making on a per click basis if they created well cited content that fit Google's guidelines.
• and a far off tangent! what would happen if Google's business model made self employment too profitable to where they could not afford to pay workers

Portalized:
Google offers portalization of Google.com. Danny Sullivan has an in depth review. They have a number of features and intend to add many, such as RSS feed support.

Stemming:
Rand points out a post by Xan on stemming and a free online stemming tool

DMOZ:
kills the submission status review. Now its even easier to be corrupt ;)

New York Times:
Begins charging for some of their content. Most of their content remains free. They are also replacing the CEO of About.com.

When Not to Submit to Directories:
when a person creates about a half dozen general directories and promotes them all together. that is not building value, that is trying to cash out and milk the web.

Many directory owners have become exceedingly greedy recently. All the while search algorithms continue to advance and few of the directory owners are actually trying to build any legitimate value.

The Search:
You can pre order John Battelle's new book. He said if you use this link he may be able to autograph it for you, assuming he can work out the shipping details.

The Size of Google's Index:
might have been a bit frothy

Google Factory Tour:
video presentations (should be up soon), Philip Lessen has highlights

Mirago AdSense:
Apparently they have a product similar to AdSense, which might be useful for companies like HotNacho.