How to Fix the Broken Link Graph

When Teoma launched in 2002 GoogleGuy made the following comment:

"I just have one question. Are the Overture results on top an April Fool's Day joke, or is that for real? ;) "

Since then Google has put ads above their organic search results, done selective self-serving within their "organic" results, and built a business that is pulling in over $20 billion a year. It turns out aggressively carpet-bombing the web with ads is no joke. :D

But the net effect of that success is that many people know the value of links. SEO is a zero sum game, and so if you don't engage in heavy self promotion you will remain obscure on Google's linkless internet:

• no one links honestly any more.
• all links are suspect.
• no one links freely any more.
• those that do link freely are considered naive.
• page rank is specifically worth money.
• links are currency.
• articles that once contained great links - no longer link to story targets.

On the web ideas (and business models) spread quickly. So as companies learn that Google encourages and pays you to pollute the web with garbage that is what many people will do. Google goes so far as to explicitly state what not to do, and many people view that as a checklist of opportunities, as it wouldn't be on that list unless it worked. ;)

The lack of community and camaraderie within the SEO industry both remarkable & unsurprising give that the SEO industry is a bit of a canary in the coal mine in terms of adopting new best practices (or worst practices, in some cases).

• At the lower end of the market people are operating like robots, mechanically spamming in a way that sure feels like crapping on the virtual living room floors in established public forums.
• Just yesterday I read a blog post listing me amongst a list of resources where everything recommended had a link - except for our site. The lack of link was so bizarrely out of place that they literally had to explain why they didn't link to our site. Crazy stuff, especially from an SEO "professional" who claims to like and value your work!
• As attention becomes more scarce, many people are willing to do anything to get a bit of it.

Meanwhile Google has no issues funding that very "linkless" web pollution by paying the likes of Demand Media to syndicate their ads, and encourage the use of rel=nofollow on links while trying to build out the model for digital sharecropper overlords. Google has pushed so far on nofollow that content spam is the single most profitable SEO business model going today.

The end result of this business decision? Google becomes its own nemesis by funding spam:

This triple occurrence of Optimization by Proxy creates a self-reinforcing cycle where the made-for-adsense website owners are rewarded with cold hard cash for their efforts. What's worse, this cash flow has been effectively subtracted from the potential gains of legitimate content producers. One can say that the existence of Google search/adsense/adwords makes all this commerce possible in the first place, but this does not make the downward spiral of inefficiency disappear.

What is even more strange about the above quoted article is that a Google search-quality engineer submitted it to Hacker News using "Sufficiently advanced spam is indistinguishable from content" as the title & wrote the following:

All of the fascinating things about signals are confidential for all of the reasons listed in the article, and Google has been sued so many times by sites that think they should rank better than they do that I can't really give examples.

I think it's safe to say though that there are a lot of people worried about and thinking hard about what the web is turning into and how to rank it appropriately.

Most of the content is no longer written by devoted hobbyists, people no longer link as often to things they like, and much of the content on the front pages of reddit, digg (and sometimes even hackernews) was put there by people trying to make your search results worse.

Given that Google pays for the creation of content and is the most profitable distribution channel for many webmasters, few businesses have anywhere near Google's influence on "what the web is turning into."

The smaller the corpus of voters there are the fewer people you need to influence to manipulate the search results. And so stuff like this becomes popular:

But if you have a live and flourishing link graph then efforts of spammy delight won't be able to compete anywhere near as well against the best sites. The problem is the best sites often remain in obscurity & even when they spread through social networks most of those links use nofollow.

The powerful element of links is that they give search an informational bias. Most other forms of user feedback (even awareness) are to some degree driven by ad budget, which would give the results a commercial bias that would cut Google's AdWords revenues.

Years ago on porn search Matt Cutts stated "One thing to consider is that our rankings (because of PageRank and the link structure of the web) often lean more toward information sites. You usually have to look a little more deliberately to find porn on Google." And last year a Google search engineer on Reddit stated: "Incredibly, we take an active role in ignoring porn search. As in, we neither care nor not care about whether our changes affect the search for porn. I'm quite impressed myself at how good it is given this, and sometimes I wonder how much better it could be..."

The point being that Google can choose to be a passive reflection of the web, but they choose *not* to and have impacted the web by perverting the link structure. Even Google engineers admit "people no longer link as often to things they like" - this is not due to "scalable algorithms" - but due to a long history of calls for spam reports, calls for paid links reports, threatening to remove PageRank, pushing rel=nofollow out for a niche purpose and then claiming it to be a panacea, associating (non-Google!) link selling with bogus cancer cures, creating the AdSense API, excessive emphasis on domain authority, claims like "we are packrats at Google. We never seem to throw out information about the link structure of the web. :)", etc.

Yes any new webmaster can quickly rank any trashy content in Google - not on their own site - but on one owned and controlled by a Google arbitrage partner like Demand Media.

Google engineers focused on making their own jobs easier (by making links harder to get) rather than encouraging and promoting the advancement of the corpus of content & links/votes they rely on. The fundamental question for search is if the ecosystem is healthier with many micro-parasites or fewer macro-parasites. I always promote diversity as a good thing, but capitalism typically promotes homogenization to increase yields. That will work for Google right up until...

• some of the large arbitrage players form their own ad network, or sign an exclusive deal with Bing
• quality publishers pull out of search and people prefer to ask their own communities what is good rather than searching to find bland 3rd grade answers wrapped in AdSense

...at which point Google will start promoting other sites & building a new model. Or they will become irrelevant.

But Google is not required to go through that pain.

They could undo the years of FUD that destroyed the link graph by stating the importance of outbound links, and then putting a bit of weight on it. It is something that was hinted at in the past, but the focus on making links harder to get has undermined the utility of using links as *the* measuring stick for quality because "people no longer link as often to things they like."