Much noise was made recently about Google taking a whack at so-called content farms -- sites which apply industrial production techniques to the creation of content targeting the long-tail of the query distribution. This is a subject of huge interest to many Internet businesses, either because they advertise on the AdWords Content Network (and, by extension, on content farms), because they compete with content farms on particular searches, or merely because they hate seeing content farms in their search results. As luck has it, I am three for three. It pains me to say it, but content farming is here to stay. It is an economic inevitability.

The Attention Economy

Much of the Internet currently operates in an attention economy, a level or two removed from direct monetization. Facebook is worth in excess of 50 billion not just because they're making money hand over fist -- though they are -- but because they have achieved a dominant position in the attention economy, and they command such huge rivers of attention that they can trade trickles of it to people for actual money.

Google is the dominant player in the attention economy -- they harvest vast amounts of attention via controlling navigation on the Internet (via a commanding lead in search), they sell attention in the form of AdWords ads, and they provide a marketplace for attention with their AdSense product.

Individual publishers -- from the New York Times down to the smallest hobbyist site on the Internet -- are also largely in the attention economy. For a mega-brand like the New York Times, attention can be generated -- they can literally make news. Disney has a repeatable industrial process which takes as input one female teenager and produces as output a cultural phenomenon with hundreds of thousands of rabid fans.

Smaller players -- Google back in the dorm room days or hobbyist sites today -- largely cannot create attention on these scales, they can only harvest attention which already exists. Attention exists in the world for things independent of their own existence. People play golf. People bake cookies. People read Dan Brown novels. People receive massages. For all these things and more, people demand content: they want to improve their golf swing, they want new cookie recipes, they want new Dan Brown novels, they want massage how-to videos. And they are willing to pay with attention, a scarce commodity which can be converted into cash.

The Economics of Content Creation

Consider a hypothetical Internet with no efficient way of converting attention into money. This is not difficult to imagine: it was essentially the Internet of the dot-com bubble, where everyone wanted "eyeballs" but "eyeballs" plus banner advertising resulted in economically non-viable businesses. In this hypothetical Internet, content is mostly produced by people who have intrinsic reasons for creating it: hobbyists who want to share their passion, law professors who want to increase their professional reputation, governments who need to employ somebody and might as well employ a webmaster, and the like. This is widely viewed as a Garden of Eden scenario: the Internet, without the corrupting influence of money.

We had this Internet, and the average user experience was miserable.

Ability to publish content on the Internet was once dominated by presence of arcane technical skills (being a "webmaster", a title which thankfully has fallen out of fashion). Webmasters were, by and large, very geeky people. They largely scratched their own itches, which (predictably) resulted in an Internet chock-full of Dungeons and Dragons character sheets, trivia about Matter-Eater Lad, and fansubbed anime episodes.

Less well-represented on the Garden of Eden Internet was content appealing to demographics which don't intersect with geeks that often. Women, the very young, the elderly, non-English speakers, etc etc, were across a very real digital divide from the D&D players. You could still find advice on how to make an apple pie online, but if you did, it was because you got lucky and had a CS professor with quirky interests (for a CS professor, at any rate).

This started to change with the widespread adoption of content management systems, which took the level of computer skill for content creation down from "close to programming" to "close to using a word processor." The first very popular CMSes were blogs, and there was much triumphantalist backslapping among bloggers that blogging was democratizing the Internet. You could be blogging in your pajamas and still take on the New York Times, or so the argument went.

Ability to use a word processor is more widely spread among the population than webmastering skills, but it is still a far cry from universal. Blogging caught on primarily with professional communicators: professors, journalists, and other folks who had long been using skill with the printed word and perceived authority with pre-existing audiences. Concurrent with this, there was an explosion of content creation aimed at the concerns of well-educated, middle-class American white urban professionals. Politics, financial advice, education, religion, international news: covered, covered, covered, both by established media and publishing interests moving online and by the new media (rather like the old media, except with orders of magnitude lower capital requirements). Content was now a democracy, in the same sense that America after the Revolution was a democracy: white property owners could be reasonably assured of having their interests represented.

There still existed massive demand -- unharvested attention -- for content outside the early adopters of the Internet. Larger scale online publishers began to go after the head of the demand distribution, and hobbyist sites continued to publish things like apple pie recipes, often with a quantum leap in presentational quality from just a few years previously. Google AdWords was one of the primary lubricants for making this happen -- a hobbyist site dominating a niche like e.g. apple pies could suddenly generate non-trivial amounts of money for the site owner, largely by taking transaction costs about negotiating advertising sales out of the equation. This also allowed Google to monetize its own attention surplus better, because sending a searcher to a site with AdSense on it gives them a second chance at getting paid for a click. AdSense has generated roughly a third of Google's revenue for the last several years.

The Industrialization Of Content Production

With technology continuing to bring down barriers to creating content and business model optimization like AdWords improving the opportunity to monetize attention, it was virtually inevitable that eventually the supply and demand curves would cross. They long since had for high-value verticals like e.g. mortgages, where huge transaction volumes, high margins, gigantic advertising spends, and liquid affiliate/lead gen markets have long subsidized huge volumes of content creation. Many quite savvy Internet users were simply unaware this had happened, since one does not search for mortgages or poker every day. The Internet is a virtually uncountable multitude of attention markets, and in many of them it was more expensive to create content than the harvestable attention could justify. Those niches continued to be underserved, in the capitalist sense of the word: people would have consumed more content for them, but that content did not exist.

Then disruptive innovation happened: basically, a number of firms figured out that the combination of algorithmically predicting attention plus outsourcing content creation could let them exploit relatively small amounts of attention, in parallel, at massive scales. This innovation caused the supply and demand curves to cross for a huge number of attention markets which had not crossed before. The result: content farming at massive, massive scale.

Consider bingo cards for elementary schoolteachers, a very niche subject that happens to pay my rent. Attention exists for it: bingo has long been used in American classrooms to review vocabulary across a variety of subjects. As teachers and parents gradually started using the Internet and using Google, their attention about bingo -- a tiny, tiny sliver of the massive river of attention Google controls -- became up for grabs. Some flowed to hobbyist sites like my own, some flowed to larger publishers like the NYT's About.com unit, and some was simply poorly served. Teachers typed queries into Google and got garbage results which were not responsive.

I have advertised on Google's AdWords Content Network for years, and for the last four years I've been essentially willing to buy as much traffic as Google cares to sell me for a range of quality below a given price. This makes my AdWords stats a proxy for who is getting traffic for bingo-related searches. (Google controls navigation on the Internet, so the presence of traffic for near-term desires like bingo cards strongly suggests that it was searched for. Check your Analytics if you don't believe me.)

My market has massive seasonal changes in attention, so let's look at consistent month-long slices of it, compared year-to-year. Here's a tale of four Februaries.

• In 2008, my AdWords spend was dominated by legacy Internet publishers like About.com, niche publishers in education, and hobbyist sites. Total spend was about $370, of which About captured almost $70 (~19%).
• In 2009, hobbyist sites and niche publishers decline with the ascendancy of a new publisher called Kaboose, an early iteration of a content farm, focused on topics of interest to women (including, e.g., bingo). Total spend was about $560, of which Kaboose captured almost $160 (a whopping 29%), more than quintupling their performance from 2008. Or, to put it another way, more than half of increase in the size of this small attention market can be attributed to one publisher. 2009 also sees a new site in my top 10: a minor player called eHow run by an obscure firm Demand Media.
• In 2010, spend again increases (to $640), and the top positions are dominated by content farms and ezinearticles, a legacy crowdsourced content farm. Kaboose loses share to new content farm entrants, and eHow has comparatively modest 50% year over year growth. Content farms now control over a third of this attention market.
• In 2011, spend again increases (to $920 -- nearly 50% year over year growth), and content farms dominate the attention market. eHow has improve its execution again, to the point where they singlehandedly capture $150 in ads, quintupling performance from a year before. (Yep, their revenue is now ten times what it was in 2009.)

The Microeconomics Of Content Farming

Why did content farming capture so much of the attention economy so quickly? Basically, once the process for creating content very responsive to a single search term was repeatable, it could be replicated down the long-tail very, very quickly, in response to market signals such as e.g. successful pages in related searches. My business has long had a page about Valentine's Day bingo cards because I know, being a publisher in the niche, that they're very valuable -- there exists a substantial amount of attention which will be paid to Valentine's Day bingo every February. Do you think Valentine's Day bingo cards is a tiny niche, on Internet scales? eHow has over thirty pages targeting variants on this top -- thirty slices of a fraction of a tiny niche which were worth individualized effort to target. Some representative titles:

• Church Valentine's Party Games
• Make Valentine Bingo Cards
• Classroom Valentine's Day Party Games
• Valentine's Math Games
• Christian Valentine's Games
• Christian Adult Valentine's Games
• Valentine's Party Games For Older Kids
• etc, etc, etc, etc

Zooming in on the performance of just one of these pages, about Valentine's bingo for churches, I paid $9 for ads on it in February 2011. If we make the unreasonably pessimistic assumption that it never makes money except in February, and that the remnant image advertising is basically a wash (not true, given the amount that Groupon and online games throw around at monetizing it), this suggests that the four text ads on the page probably generated on the order of $30 in revenue. Google's 68% revenue share means that Demand Media got about $20 in revenue from this page... in 2011 alone.

Content farms are targeting evergreen content, though: Valentine's Day is going to happen in 2012, and there will still exist churches who want to play bingo on it. Will revenue from this page go to zero? That is highly unlikely, because this page wasn't written in 2011 -- it was written in 2010, when I paid $1 for ads in it (implying about $2 in revenue). Due to changes in the search environment and Demand Media's increasing sophistication with leveraging internal traffic, it got nine times more valuable at no marginal cost in the course of a single year.

Content farms operate on a portfolio strategy: the pieces of content which succeed, like that page, subsidize the pieces of content which don't. As long as the average revenue portfolio-wide exceeds cost of content production, one should expect the content farms to pour capital into content production and scale it to the moon. The portfolio strategy appears to be winning, judging by eHow's meteoric rise in revenue and the demonstrated ability for content farms to choke out non-farming content sources. eHow alone showed my ads on five times as many pages in 2011 as in 2010.

And why wouldn't they? The unit economics of content farming are stunningly attractive. Demand Media pays on the order of $10 to have the 312 words on that page written and edited. If Wall Street could design an equity which cost $10 and paid $2 per share in 2010, $20 per share in 2011, and an unknown but positive amount thereafter, all other investment classes would be virtually obsolete. The only problem is systemic risks.

The only thing that can reverse this is content getting more expensive to create or attention getting scarcer (or harder to monetize) for these markets.

There is more attention to monetize: It is possible that Internet use will decline in the future, but I will offer excellent odds to anyone who wishes to bet that: Kansas schoolmarms in the elementary bingo market have quite a ways to go before they catch up to the average reader of this blog in online consumption, which predicts a large aggregate increase in attention harvestable on the Internet and even larger proportional increases to the attention markets they care about.

Google and advertisers increasing cost of attention: Ignoring huge sources of attention of dubious worth, like ads displayed next to Facebook games, an AdSense ad displayed to someone 2 seconds after they type in a query into Google is, essentially, a search ad.

Read that again, because it is important.

This means that content farms are essentially in the search ad monetization business -- i.e. the most profitable business in the history of the Internet. Search ads monetize extraordinarily well because in addition to capturing user attention they come with user intent. This makes them orders of magnitude more valuable than the old banner display networks (which users quickly become blind to), sidebar ads next to Farmville or pictures of that cute girl from chemistry class, and the like. Content farms preserve search intent because the laser targetting combination of their one-topic pages and AdSense means that the AdSense ads are guaranteed to be responsive content to the search and, give that everything else on the page was written by a content farm, the ads are the best content on the page.

Sure, farms cede a large portion of the reach of search to actual search engines, since they can't rank for head queries, but even 5% of Google's market cap would be nothing to sneeze at. Google has incentives to help them rather than competing with them. Meanwhile, any market with competitors will tend to drive the cost of ads up until they have expended all of their margin on the sale. For a high-margin category like software, if my competitor is willing to pay 51% of his sale price to generate one marginal sale (when you back it out to cost-per-click prices), I'm willing to bid 51%. The equilibrium outcome is that my advertising costs increase over time while my ROI decreases, but it remains profitable and I'd be a fool not to do it. Google and their publishing partners win and win big.

This Is Old News. Google Fixed Content Farming... Right?

Back in late February 2011, Google rolled out the Panda update, which was widely perceived to be aimed at content farms. What actually happened was that it separated Content Farming 1.0 from Content Farming 2.0 -- earlier entrants like ezinearticles and Mahalo (and a raft of sites you've never heard about) lost out to better executing farms, including eHow.

For example, instead of comparing Februaries like we did earlier, let's see the progression of Marches in the bingo niche. Largely due to the absence of Valentine's Day, March consistently has less attention available than February: aside from an anomalous 2008 (long story with short moral: don't bork your AdWords code), spends fell 28% in 2009 and 17% in 2010. The decline was much more pronounced in 2011, possibly attributable to Panda reshaping the attention economy landscape: it jumped to 37%.

However, the performance of individual publishers was mixed:

1. eHow (Demand Media) declined only 18%. This looks virtually in line with historical seasonal trends (growth in 2010 was so fast they were actually flat over the interval, i.e. growing much faster than market). Their performance in March 2011 (historically a "bad" month for bingo attention) crushed their performance in February 2009 (historically a "great" month for bingo attention). One could be excused for believing eHow was not net-affected by Panda
2. LoveToKnow got annihilated -- spend decreased 71%. (The comparable decrease in 2010 was only 25%.)
3. ezinearticles got annihilated -- spend decreased 68%. (The comparable decrease in 2010 was only ~10%.)
4. About.com was severely affected -- spend decreased about 46%. (The comparable decrease in 2010 was, again, lower -- only 21%.)

Summed over all the content farmers, Panda appears to have picked a winner with regards to this slice of the attention economy: eHow.

I had been wistfully hoping that when the content farms got crushed that my site, which competes with them for many queries, would pick up some of the redistributed attention. If this happened, it has been too minor to notice in my Analytics stats -- my organic searches from Google look roughly in line with where I would expect them to be absent Panda. The big winner and the big losers appear to be concentrated among farmers, with fairly minor spillover to the rest of this sliver of the attention economy. This makes sense to me, in a way -- I simply don't have a page which is more responsive to the need for church Valentine's bingo activities than eHow's does. I believe my pages are far and away better than eHow's -- my pages about making bingo cards will actually let you make bingo cards -- but reasonable people could disagree on whether that is more important than capturing all parts of the user's intent, including the "specifically for churches" bit of it. Outside of my narrow slice of the online experience, a Big Publisher advocacy group estimates that the Panda update redistributed $1 billion in advertising revenue, which is nothing to sneeze at. However, with the Content Network generating over $20 billion in annual ad sales, $1 billion looks less like a fundamental shift and more like repartitioning scraps left to the losers.

Did Panda Kill Farming?

Only the economics can kill farming, and it does not appear that Panda meaningfully changes the microeconomics of content farms. If you can sell $40 of ads against a $10 page prior to Panda, and after Panda you can only sell $20 of ads, well, farm on. The model scales to the moon as long as the portfolio is even marginally profitable. The losing farms will also be incentivized to go back to the drawing board and reevaluate where they place their content bets: perhaps it is no longer lucrative enough for them to go after certain micro-markets, like elementary school bingo cards (or like the bottom half of elementary school bingo cards), but their more valuable markets are probably still stupidly profitable. Those will get more competitive as they redeploy their content creation resources going forward, assuming they're capable at executing on that.

Demand Media, on the other hand, is grinning like the cat that just caught the canary. Not only is their core business proposition virtually unaffected, in spite of the worst nightmare of their business model (Google coming down on it like the fist of an angry god) coming true, their unit economics down the tail just got radically better. Going forward, they can expect less competition in the less lucrative markets, allowing them to capture larger fragments of the attention available in those markets, and proportionally higher revenues.

The Future: Outfarming The Farmers?

A frequent theme of dystopian science fiction is that man-machine hybrids outcompete the human race. Algorithmic/freelancer hybrids, like content farms, are pretty much there, for a large and increasing portion of the content tail. This is going to get exacerbated by changes in content production and consumption, such as the rise of video (which has orders of magnitude higher production costs than text) and decline of hobbyist content creation. In 2006, my business had significant competition for keywords from individual teachers' sites, where Mrs. Smith decided (back in 1996) to put up a web page to try out this new Internet thing on her computer. In 2021, there will be many less sites created by Mrs. Smiths, because Mrs. Smith in 2011 now has an iPad to watch her Khan Academy videos on, and the iPad is virtually useless for creating websites. I'm already seeing anecdotal behavioral changes in my customers ("Say, how do I hook a printer up to an iPad so I can make my cards from there? I hate turning on the computer -- I think it has a virus or something, and it is slow."), and ordinarily they're quite behind the curve.

Additionally, while Mrs. Smith had sufficient dedication to the niche to target the most common activities, she never made more than 5 or so pages about bingo. I have about a thousand bingo activities, created with focused application of custom software by freelancers. The content farms are making me look practically lazy with their scale of publication.

This suggests an obvious route for improvement for me: if it is stupidly profitable for me to pay $200 to Google so that it can pay $130 to eHow so that it can pay $50 to freelance writers to create 5 pages, why don't I just take the $200 and pay freelance writers to write those same 5 pages... and then 15 more? The only thing which has stopped me from doing it so far is concern about polluting the Internet. But the economic attraction of doing it is undeniable. If the choice is a user getting their bingo content from an anonymous freelancer at eHow working through their queue of 400 articles for the week or getting it from someone who is only employed to write bingo articles, shouldn't they get it from me?

This dilemma, repeated a thousand times across a thousand markets, is going to create the Internet of 2020. Break out your straw hats, folks: we are all going to be farming or, at best, a step removed from farming by paying intermediaries (Google and the farms) to do our farming for us. The main distinction is going to be between successful execution of farming strategies (like eHow) and poor execution of farming strategies (like their competitors who recently got whacked).

Demand Media is already shopping out their business model as a service: since newspapers and other legacy publishers are a) dying but b) scare Google (because they can cause Google to have bad PR, which might result in government regulation, which is Google's sole competitive risk), Demand Media would love newspapers to be the front man for their farmed content. That, or parallel arrangements, is going to be almost irresistible to anyone with sufficient signals of trust to rank for arbitrary longtail content in their niche. I mean, "Create a repeatable process to create content of a known level of quality, throw money at the process to scale it, then sell ads against the result" is the entire newspaper business model! Content farming just takes out the sucky bits like "own a multi-billion-dollar distribution network for dead trees" and "write articles which are relevant to a few hundred people at an amortized cost of over $1,000 per article." (It is an open secret that the most lucrative ads in a newspaper aren't around the news, but are in sections like Style and Travel. It does not take Pulitzer Prize-winning journalism to write articles on this season's hottest shade of fuchsia or compelling reasons to go to Cancun. "Real" news has always been a loss leader to sell advertising against their other content. If they can create ten times as much fluff at a tenth of the cost, why not? And if they can... do they need the "real" news in the first place?)

Can Google just tighten the screws with another Son of Panda update? That is unlikely to work unless they repeal the laws of economics: farming happens on every topic for which the supply and demand curve crosses. Slashing content farm's ability to rank across the board by 40% just makes a fraction of the content space monetarily unattractive to them, but the content space is virtually infinite and the ability to monetize attention is increasing all the time. If Groupon will pay for remnant inventory on a page about How To Pick Your Nose, who will compete for that attention except a content farm?

Is that the Internet I want? Probably not. But then again, I'm privileged -- as a geek, my interests in content will always be well represented on the Internet, even without monetary incentives to create it. People will go to StackOverflow to answer my questions before I've even asked them, they'll create Starcraft XII walk-through videos, they'll even write software, all without seeing a penny for it. The experience for less privileged folks, though, demonstrably sucked at the dawn of the Internet, and it is not obvious to me that removing most of the growth in content responsive to their needs is a net win for them. We might see an Internet where the content-rich win and everybody else gets farming.

Like I said... dystopian sci-fi.

Patrick McKenzie runs a small software business. When not blogging or taking over the worldwide printable bingo cards market, he is working on his new venture, Appointment Reminder.

Economics Drive Literally Everything

Media is all about profit margins. eHow was originally founded in 1998 & had $36 million in venture capital behind it. But the original cost structure was flawed due to high content costs. The site failed so badly that it was sold in 2004 for $100,000. The original site owners had GoogleBot blocked. Simply by unblocking GoogleBot and doing basic SEO to existing content the site had a revenue run rate of $4 million Dollars within 2 years, which allowed the site to be flipped for a 400-fold profit.

Demand Media bought it in 2006 and has pushed to scale it while making cheaper and lower quality content. Demand Media has since gone public with a $1.5+ billion valuation based largely on eHow. Just prior to the DMD IPO Google's Matt Cutts wrote a warning about content mills on the official Google blog.

Seeing Value Where Others do Not

People are arguing if Demand Media is overvalued at its current valuation, and at one point the NYT was debating buying Demand Media by rolling About.com into Demand & own 49% of the combined company. But the salient point to me is that we are talking about something that was bought for only $100,000 7 years ago. Sure the opportunities today may be smaller scale and different, but if you see value where others do not then recycling something that has been discarded can be quite valuable.

In this 6-page article about the eHow way on page 6 there was a recycling tip any SEO can do with the help of some OCR software:

The key, he said, was to keep costs low. If possible, don't pay for the intellectual content. Look for material, he urged, on which the copyright has expired. Any book published in the U.S. before 1923 was available.
He said he was in the process of turning vanloads of old books into websites. With a few hours of labor, for example, you could take a turn-of-the-century Creole cookbook and transform it into the definitive site for vintage Creole recipes. Google's AdSense program would then load the thing up with ads for shrimp and cooking pots and spices and direct people looking for shrimp recipes to your website.

A spin on the above, LoveToKnow has published a 1911 encyclopedia online. And ArticlesBase (an article farm which built up its popularity by linking to contributior sites) now slaps nofollow on all outbound links & is pulling in a cool $500k per month!

How did ArticlesBase grow to that size? It and Ezinearticles were a couple of the "selected few" which lasted through the last major burn down of article directories about 3 or 4 years ago. But it seems their model has peaked after this last Google update.

Search is Political

Content farms are proving to be a political issue in search. They are beginning to replace "the evil SEO" in the eye of the press as "what creates spam." Rich Skrenta created a spam clock which stated that a million spam pages are created every hour. He then followed up by banning 20 content farms from the Blekko search results & burning spam man. ;)

Microsoft also got in on the bashing, with Harry Shum highlighting that Google was funding web pollution. When Blekko's model is based on claiming Google polluted the web with crap, Microsoft says the same thing, and there is a rash of end user complaints, there are few independent experts the media can call upon to talk about Google - unless they decide to talk to SEOs, who tend to be quite happy to highlight Google's embarrassing hypocrisy. Freelance writers may claim that marketing is what screwed up the web, but ultimately Google has nobody credible and well known to defend them at this point. The only people who can defend Google's approach are those who have a taste in the revenue stream. Hence why Google had to act against content farms.

Always Be Selling

Demand Media's CEO is the consummate sales professional, when Google first warned about content farms Mr. Rosenblatt he used the above to disclaim that Google means "duplicate content" when they write about content farms. Then Demand quickly scrambled after they were caught publishing plagiarized content the following day. :)

Google stepping up their public relations smear campaigns against Bing and others is leaving Google looking either hypocritical or ignorant in many instances, like when a Google engineer lambasted an ad network without realizing Google was serving the scam ad.

Social Answers?

While on the content farm topic, it is worth mentioning that Answers.com was bought for $127 million & there is also a bunch of news about Ask's strategy in the Ask section near the bottom of this newsletter. On the social end of the answer farm model, Facebook was rumored to be looking into the space & Twitter bought a social answer service called Fluther. Even Groupon seems to be looking at the space. Quora is well hyped on TechCrunch, but will have a hard time expanding beyond the tech core it has developed.

High Quality Answer Communities?

At first glance StackExchange's growth looks exciting, but it has basically gone nowhere outside of the programming niche. In my opinion they are going to need to find subject matter experts to lead some of their niche sites & either pay those experts or give them equity in the sites if they want to lead in other markets. Worse yet, few people are as well educated about online schemes as programmers, so the other sites will not only lack leadership, but will also be much harder to police. Just look at the junk on Yahoo! Answers! There are Wordpress themes and open source CMS tools for QnA sites, but I would pick a tight niche early if I was going to build one as the broader sites seem to be full of spam and the niche sites struggle to cross the chasm. As of writing this, fewer than 50 Mahalo answers pages currently indexed in Google have over 100 views. It flat out failed, even with financial bribes and a PR spin man behind it.

A Warning

A Google engineer nicknamed moultano stated the following on Hacker News:

At the organizational level, Google is essentially chaos. In search quality in particular, once you've demonstrated that you can do useful stuff on your own, you're pretty much free to work on whatever you think is important. I don't think there's even a mechanism for shifting priorities like that.
We've been working on this issue for a long time, and made some progress. These efforts started long before the recent spat of news articles. I've personally been working on it for over a year. The central issue is that it's very difficult to make changes that sacrifice "on-topic-ness" for "good-ness" that don't make the results in general worse. You can expect some big changes here very shortly though.

A good example of the importance of padding out results with junk on-topic content to aid perceived relevancy can be seen by looking at the last screenshot of a search result here. Blekko banned the farms, but without them there is not much relevant content that is precisely on-topic. (In other words, content farms may be junk, but it is hard to have the same level of perceived relevancy without them).

New Signals

Google created a Chrome plugin to solicit end user feedback on content mills, but that will likely only reach tech savvy folks & the feedback is private. Google can claim to use any justification for removing sites they do not like though, just like they do with select link buying engagements. Look the other way where it is beneficial, and remove those which you personally dislike.

In a recent WSJ article Amit Singhal was quoted as saying new signals have been added to Google's relevancy algorithms:

Singhal did say that the company added numerous “signals,” or factors it would incorporate into its algorithm for ranking sites. Among those signals are “how users interact with” a site. Google has said previously that, among other things, it often measures whether users click the “back” button quickly after visiting a search result, which might indicate a lack of satisfaction with the site.

In addition, Google got feedback from the hundreds of people outside the company that it hires to regularly evaluate changes. These “human raters” are asked to look at results for certain search queries and questions such as, “Would you give your credit card number to this site?” and “Would you take medical advice for your children from those sites,” Singhal said.

Evolving the Model

One interesting way to evolve the content farm model is through the use of tight editorial focus, a core handful of strong editors, and wiki software. WikiHow was launched by a former eHow owner, and when you consider how limited their relative resources are, their traffic level and perceived editorial quality are quite high. Jack Herrick has struck how-to gold twice!

Going Political?

AOL purchased The Huffington Post for $315 million. Here are some reviews of that purchase. The following analysis is a bit rough, but I still think it is spot on - contrary to popular belief, most of Huffington Post's pageviews are still driven by their professionally sourced content.

Editors who have a distaste for pageview journalism are already quitting AOL. But if you are interested in the content farm business model, AOL's business plan was leaked publicly. Oooops. :D

Conflating Scraper Sites vs Content Farms

In addition to general content farms, Google is fighting a war against scraper websites. One such algorithmic update has already been done against sites repurposing content, and the content farm algorithm just recently went live & whacked a bunch of content farms. Check out the top losers from Sistrix's data.

Notice any content farms missing from the above list? Maybe the biggest one? Here is a list of some of eHow's closest competing sites (based on keywords, from SEM Rush). The ones in red got pummeled, the ones in yellow dropped as well & were fellow Demand Media sites, and the ones in green gained traffic.

Getting Hammered

Jason "will do anything for ink" Calacanis recently gave an about face speech claiming people need to step away from the content farm business model, and in doing so admitted that roughly everything he said about Mahalo over the past couple years was a complete lie. Surprise, surprise. The interesting bit is that the start up community - which used to fawn over his huckster PR driven ploys - no longer buys them. Jason claimed to have "pivoted" his business model again, but once again we see more garbage content. His credibility has been spent. And so have his rankings! Sistrix shows that not only is he ranking for fewer keywords, but that the graph has skewed downward to worse average positions.

After the Crash, What is Next?

The biggest content farms like Ask & eHow will still do well in the short run. Over the long run I see Google bringing the results of content farms to the attention of book publishers & then working to slowly rotate out from farmed content to published book content. Most readers do not know that most book writers are lucky if they earn $10,000 writing a 300 or 400 page tome. Publishers tell book authors that with the additional exposure they can often sell lots more other things, but unless the content is highly targeted that might not back out well for the author. But that cheap content is far better structured and far more vetted than the mill stuff is.

Over the past week I have been seeing more ebooks in the search results, though I am not entirely sure if that is just because I am searching for more rare technical stuff that simply might not be online.

The Question Nobody is Asking

I highlighted Google's hypocritical position in judging intent with links while claiming they need an algorithmic approach to content farms. But nobody is thinking beyond the obvious question. Everyone wants to know who Google punished the most, but nobody is asking who gained the most from this update.

Demand Media put out a statement that their traffic profile did not change materially.

But what they didn't mention is that eHow's rankings are actually up! In fact, their new distribution chart looks just like their old one, only skewed a bit to the left with higher rankings. eHow's profile is 15% better than it was before the update & the only site which gained more traffic from this update than eHow did was Youtube.

How Did We Get Here?

People may have been sorta aware of garbage content & saw it ranking, but were apathetic about it. Most people are far more passive consumers of search than they believe themselves to be - when the default orders switch people still tend to click the top ranked result. It was only when eHow started branding itself as a cheap and disposable answer factory that people started to become outraged with their business model.

Demand Media further benefited from flagrant spammy guideline violations, like 301 redirecting expired domains into deep eHow pages. People I know who have done similar have seen their sites torched in Google. But eHow is different!

If you listen to Richard's interviews, you would never know him to be the type to redirect expired domains:

We really want to let Google speak for themselves. Whatever Matt Cutts and Google want to (say) about quality we totally support that because again that’s their corporate interest. What we said and would have said is we applaud Google removing duplicate content ... removing shallow, low quality content because it clogs the search results. Both we and Google are 100 percent focused on making the consumer happy. It’s the right thing to do and it’s good for our business.

If you syndicate Google's spin you can get away with things that a normal person can't. Which is why eHow renounced the content farm label even faster than they created it.

Article directories & topical hub sites have been online since before eHow was created. But eHow's marketing campaign was so caustic & so toxic that it literally destroyed the game for most of their competitors.

And now that Google has "fought content farms" (while managing to somehow miss eHow with TheAlgorithm) most of Demand Media's competitors are dead & Richard Rosenblatt gets to ride off into the sunset with another hundred million Dollars, as eHow is the chosen one. :D

Long live the content farm!

An "Algorithmic" Approach

When it comes to buying links, Google not only fights it with algorithms, but also ran a 5-year long FUD campaign, introduced nofollow as a proprietary filter, encouraged webmasters to rat on each other, and has engineers hunting for paid links. On top of that, Google's link penalties range from subtle to overt.

Google claims that they do not want to police low quality content by trying to judge intent, that doing so would not be scalable enough to solve the problem, & that they need to do it algorithmically. At the same time, Google is willing to manually torch some sites and basically destroy the associated businesses. Talk to enough SEOs and you will find stories of carnage - complete decimation.

Economics Drive Everything

Content farms are driven by economics. Make them unprofitable (rather than funding them) and the problem solves itself - just like Google AdWords does with quality scores. Sure you can show up on AdWords where you don't belong and/or with a crappy scam offer, but you are priced out of the market so losses are guaranteed. Hello $100 clicks!

How many content farms would Google need to manually torch to deter investment in the category? 5? Maybe 10? 20 tops? Does that really require a new algorithmic approach on a web with 10's of millions of websites?

When Google nuked a ton of article banks a few years back the damage was fairly complete and lasted a long time. When Google nuked a ton of web directories a few years back the damage was fairly complete and lasted a long time. These were done in sweeps where on day you would see 50 sites lose their toolbar PageRank & see a swan dive in traffic. Yet content farms are a sacred cow that need an innovated "algorithmic" approach.

One Bad Page? TORCHED

If they feel an outright ban would be too much, then they could even dial the sites down over time if they desired to deter them without immediately killing them. Some bloggers who didn't know any better got torched based on a single blog post:

The Forrester report discusses a recent “sponsored conversation” from Kmart, but I doubt whether mentions that even in that small test, Google found multiple bloggers that violated our quality guidelines and we took corresponding action. Those blogs are not trusted in Google’s algorithms any more.

One post and the ENTIRE SITE got torched.

An Endless Sea of Garbage

How many garbage posts have you seen on content farms?

When you look at garbage content there are hundreds of words on the page screaming "I AM EXPLOITATIVE TRASH." Yet when you look at links they are often embedded inline and there is little context to tell if the link is paid or not, and determine if the link was an organic reference or something that is paid for.

Why is it that Google is comfortable implying intent with links, but must look the other way when it comes to content?

Purchasing Distribution

Media is a game of numbers, and so content companies have various layers of quality they mix in to make it harder for Google to find signal from noise. Yahoo! has fairly solid content in their sports category, but then fluff it out with top 10 lists and such from Associated Content. Now Yahoo! is hoping they can offset lower quality with a higher level of personalization:

The Yahoo platform aims to draw from a user’s declared preferences, search items, social media and other sources to find and highlight the most relevant content, according to the people familiar with the matter. It will be available on Yahoo’s Web site, but is optimized to work as an app on tablets and smartphones, and especially on Google Android and Apple devices, they said.

AOL made a big splash when they bought TechCrunch for $25 million. When AOL's editorial strategy was recently leaked it highlighted how they promoted cross linking their channels to drive SEO strategy. And, since acquisition, TechCrunch has only scaled up on the volume of content they produce. In the last 2 days I have seen 2 advertorials on TechCrunch where the conflicting relationship was only mentioned *after* you read the post. One was a Google employee suggesting Wikipedia needs ads, and the other was some social commerce platform guy promoting the social commerce revolution occurring on Facebook.

Being at the heart of technology is a great source of link equity to funnel around their websites. TechCrunch.com already has over 25% as many unique linking domains as AOL.com does. One of the few areas that is more connected on the social graph than technology is politics. AOL just bought Huffington Post for $315 million. The fusion of political bias, political connections, celebrity contributors, and pushing a guy who promoted (an ultimately empty) promise of hope and change quickly gave the Huffington Post even more link equity than TechCrunch has.

Thus they have the weight to do all the things that good online journalism is known for, like ads so deeply embedded in content you can't tell them apart, off-topic paginated syndicated duplicate content and writing meaningless posts devoid of content based on Google Trends data. As other politically charged mainstream media outlets have shown, you don't need to be factually correct (or even attempt honesty) so long as your bias is consistent.

Ultimately this is where Google's head in the sand approach to content farms backfired. When content farms were isolated websites full of trash Google could have nuked them without much risk. But now that their is a blended approach and content farms are part of public companies backed by politically powerful individuals, Google can't do anything about them. Their hands are tied.

Trends in Journalism

Much like the middle class has been gutted in the United States, Ireland (and pretty much everywhere that is not Iceland) by economic policies that gut the average person to promote banking criminals, we are seeing the same thing happen online to the value of any type of online journalism. As we continue to ask people to do more for less we suffer through a lower quality user experience with more half-content that leaves out the essential bits.

How to build a brick wall:

• step 1: get some bricks
• step 2: stack them in your workplace
• step 3: build the brick wall

The other thing destroying journalism is not only lean farms competing against thick and inefficient organizations for distribution, but also Google pushing to control more distribution via their various data grabs: Youtube video & music, graphical CPA ads in the search results, lead generation ads in the search results, graphic AdSense ads on publisher sites that drive searches into those lead generation funnels, grouping like data from publishers above the organic search results, offering branded navigational aids above the organic search results, acquiring manufacturer data, scraping 3rd party reviews, buying sentiment analysis tools, promoting Google maps everywhere, Google product pages & local review pages, extended ad units, etc. If most growth in journalism is based on SEO & Google is systematically eating the search results, then at some point that bubble will get pricked and there will be plenty of pain to go around.

My guess is that in 3 to 4 years the search results become so full of junk that Google pushes hard to rank chunks of ebooks wrapped in Google ads directly in the search results. Books are already heavily commoditized (it's amazing how much knowledge you can get for $10 or $20), and given that Google already hard-codes their ebooks in the search results, it is not a big jump for them to work on ad deals that pull publishers in. It follows the trend elsewhere "Free Music Can Pay as Well as Paid Music, YouTube Says."

It's Not All Bad

The silver lining there is that if you are the employer your margins may grow, but if you are an employee & are just scraping by on $10 an hour then it increases the importance of doing something on the side to lower your perceived risk & increase your influence. A few years back Marshall Kirkpatrick started out on AOL's content farms. The tips he shared to stand out would be a competitive advantage in almost any vertical outside of technology & politics:

one day Michael Arrington called and hired me at TechCrunch. "You keep beating us to stories," he told me. I was able to do that because I was getting RSS feeds from key vendors in our market delivered by IM and SMS. That's standard practice among tech bloggers now, but at the time no one else was doing it, so I was able to cover lots of news first.

Three big tips from the "becoming a well known writer front" for new writers are...

• if short form junk content is the standard then it is easier to stand out by creating long form well edited content
• it is easier to be a big fish in a small pond than to try to get well known in a saturated area, so it is sometimes better to start working for niche publishers that have a strong spot in a smallish niche
• if you want to target the bigger communities the most important thing to them (and the thing they are most likely to talk about) are themselves

Another benefit to publishers is that as the web becomes more polluted people will become far more likely to pay to access better content and smaller + tighter communities.

Prioritizing User Feedback?

On a Google blog post about web spam they state the following:

Spam reports are prioritized by looking at how much visibility a potentially spammy site has in our search results, in order to help us focus on high-impact sites in a timely manner. For instance, we’re likely to prioritize the investigation of a site that regularly ranks on the first or second page over that of a site that only gets a few search impressions per month.

Given the widely echoed complaints on content farms, it seems Google has a different approach on content farms, especially considering that the top farms are seen by millions of searchers every month.

Implying Intent

If end users can determine when links are paid (with limited context) then why not trust their input on judging the quality of the content as well? The Google Toolbar has a PageRank meter for assessing link authority. Why not add a meter for publisher reputation & content quality? I can hear people saying "people will use it to harm competitors" but I have also seen websites torched in Google because a competitor went on a link buying spree on behalf of their fellow webmaster. At least if someone gives you a bad rating for great content then the content still has a chance to defend its own quality.

With link stuff there is a final opinion and that is it. Not only are particular techniques of varying levels of risk, but THE prescribed analysis of intent depends on who is doing it!

A Google engineer saw an SEO blog about our affiliate program passing link juice and our affiliate links stopped passing weight. (I am an SEO so the appropriate intent is spam). Then something weird happened. A few months later a Google engineer *publicly* stated that affiliate links should count. A few years later Google invested in a start up which turns direct links into affiliate links while hiding the paid compensation in the background. (Since Google is doing it the intent is NOT spam).

Implying Ignorance

Some of the content mills benefit from the benefit-of-doubt. Jason Calacanis lied repeatedly about "experimental pages" and other such nonsense. But when his schemes were highlighted he was offered the benefit of the doubt. eHow also enjoys that benefit of the doubt. It doesn't matter that Demand Media's CEO was the chairman of an SEO consulting company which sold for hundreds of millions of Dollars. What matters is the benefit of the doubt (even if his company flagrantly violates quality guidelines by doing bulk 301 redirects of EXPIRED domains into eHow ... something where a lesser act can put you up for vote on a Google engineer's blog for public lynching).

The algorithm. They say. It has opinions.

What Other Search Engines Are Doing

A Bing engineer accused Google of funding web pollution. Blekko invites end users to report spam in their index, and the first thing end-users wanted booted out was the content mills.

But Google need to be "algorithmic" when the problems are obvious and smack them in the face. And they need to "imply intent" where the problems are less problematic & nowhere near as overt.

Makes sense, almost!

Search can be used as a wedge in a variety of ways. Most are perhaps poorly understood by the media and market regulators.

Woot! Check Out Our Bundling Discounts

When Google Checkout rolled out, it was free. Not only was it free, but it came with a badge that appears near AdWords ads to make the ads stand out. That boosts ad clickthrough rates, which feeds into ad quality score & acts as a discount for advertisers who used Google Checkout. If you did not use Google's bundled services you were stuck paying above market rates to compete with those who accepted Google's bundling discounts.

And there is no conspiracy theory to the above bundling. Here is a video on how quality score works & this Google Checkout page states it plain as day

And all the while Google was doing the above bundling (as they still are to this day) they were also lobbying in Australia about Paypal's dominant market position. eBay (which owns Paypal) is one of Google's 5 largest advertisers.

This Brand is Your Brand, This Brand is My Brand

Companies spend billions of Dollars every year building their trademarked brands. But if they don't pay Google for existing brand equity then Google sells access to that stream of branded traffic to competitors, even though internal Google studies have shown it causes confusion in the marketplace.

The Right to Copy

Copyright protects the value of content. To increase the cost of maintaining that value, DoubleClick and AdSense fund a lot of copy and paste publishing, even of the automated variety. Sure you can hide your content behind a paywall, but if Google is paying people to steal it and wrap it in ads how do you have legal recourse if those people live in a country which doesn't respect copyright?

You can see how LOOSE Google's AdSense standards are when it comes to things like copyright and trademarks by searching for something like "bulk PageRank checker" and seeing how many sites that violate Google's TOS multiple ways are built on cybersquatted domain names that contain the word "PageRank" in them. There are also sites dedicated to turning Youtube videos into MP3's which are monetized via AdSense.

Universal Youtube Search

Google bought Youtube and then swiftly rolled out universal search, which dramatically increased the exposure of Youtube. Only recent heat & regulatory review has caused Google to add more prominent links to competing services, nearly a half-decade later.

A Unit of Obscurity

Knol was pushed as a way to revolutionize how people share information online. But it went nowhere. Why? Google got caught with their hand in the cookie jar, so they couldn't force the market to eat it.

Scrape You Very Much

Philosophically Google believes in (and delivers regular sermons about) an open web where companies should compete on the merit of their products. And yet when Google enters a new vertical they *require* you to let them use your content against you. If you want to opt out of competing against yourself Google say that is fine, but the only way they will allow you to opt out is if you block them from indexing your content & kill your search traffic.

“Google has also advised that if we want to stop content from appearing on Google Places we would have to reduce/stop Google’s ability to scan the TripAdvisor site,” said Kaufer “Needless to say, this would have a significant impact on TripAdvisor’s ranking on natural search queries through Google and, as such, we are not blocking Google from scanning our site.”

From a public relations standpoint & a legal perspective I don't think it is a good idea for Google to deliver all-or-nothing ultimatums. Ultimately that could cause people in positions of power to view their acts as a collection which have to be justified on the whole, rather than on an individual basis.

Lucky for publishers, technology does allow them to skirt Google's suggestions. If I ran an industry-leading review site and wanted to opt out of Google's all-or-nothing scrape job scam, my approach would be to selectively post certain types of content. Some of it would be behind a registration wall, some of it would be publicly accessible in iframes, and maybe just a sliver of it is fully accessible to Google. That way Google indexes your site (and you still rank for the core industry keywords), but they can't scrape the parts you don't want them to. Of course that means losing out on some longtail search traffic (as the hidden content is invisible to search engines), but it is better than the alternatives of killing all search traffic or giving away the farm.

Yes, deliberate mistake :)

It grates when people write poorly, huh. When writers write well, the words almost become invisible. The focus shifts away from technical details, and onto the message.

Is there an easy way to write better blog posts? E-mails? Web copy?
Let's take a look at three guidelines for web writing.

1. If You Can Say It, You Can Write It

The Dilbert Mission Statement Generator - sadly now offline - comes up with convoluted gems this:

"Our challenge is to assertively network economically sound methods of empowerment so that we may continually negotiate performance based infrastructures"

Satire, one would hope.

However, the US Air Force uses the following mission statement:

"The mission of the United States Air Force is to deliver sovereign options for the defense of the United States of America and its global interests - to fly and fight in Air, Space, and Cyberspace"

"Deliver sovereign options"?

Who talks like this? Well, apart from the US military.

Nobody.

Good web writing is the same as good spoken language. Use short sentences, short words, simple structures and a natural, predictable flow of ideas. Avoid waffle, hyperbole and words that hide meaning. Whenever you finish a piece of writing, read it aloud. Cut or rephrase phrases that sound clunky, because they'll read clunky, too.

Your writing will sound warm and human.

The human voice is especially important online. Communicating at a distance, particularly two-way communication, is relatively new to humans. To help people connect with one another more easily, it pays to write in a warm, conversational style that mimics personal conversation when conducted in close, physical proximity.

When you think about how you would say something, especially to a specific person, you choose words, expressions and structures based on that personal context. Try to imagine that person in front of you as your write.

This approach works well for all applications - from formal legal sites, to personal sites.

2. Planning

Planning what you're going to say helps you to complete any writing task more quickly and easily.

• 1. Identify and list your goals. What is the message? What is the desired action you want your reader to take? What is the key thought you want your reader to take away?
  
  
  For example, a goal list might look like this:
  
  
  
  *inform people the last project went well, even though there were problems
  *highlight the good aspects about the project
  *highlight the problems
  *present ideas on how these problems can be overcome in the next project
  *get everyone revved up and excited about the next project
  
• 2. Think about the audience. Who is your audience? What do you know about the person or group?
• 3. Determine the right tone and format based on answers 1& 2
• 4. Write quickly. Don't edit, even if your writing is a mess. Separate out your writing and editing functions.
• 5. Draw a solid conclusion. Calls to action work well.
• 6. Read aloud what you've written. Cut, fix and tighten. Writing comes alive in the rewrite.

Solid blog posts sound spontaneous, but they're not. They're often structured, worked and reworked.

3. Hyperbole Doesn't Work On The Web

Hyperbole means extreme exaggeration. i.e. "All the perfumes of Arabia could not sweeten this little hand". Web readers tend to gloss over the flowery and the convoluted.

On the web, people scan, so the shape of your writing - how it appears on the page - can be just as important as what you say. So think about the shape and form of your writing. Can you use bullets, headings and images to break up large blocks of text? Sometimes, the best thing to do is not write at all. Can an image convey your message? If so, use it.

Also consider context. When visitors arrive on a page, a page deep within your site, do they know what your site is about from glancing at that one page? If not, consider using chunks of content to provide context. These chunks of information can be repeated on every page of your site, and should be self explanatory. Think directory entry. Your repeat visitors will become blind to it, but your first time readers will appreciate it.

We could go on all day about web writing. However, we'd like to hear your tips. How do you approach writing on your site? Do you plan? Do you wing it? What style of writing gets the best results?