On Feb. 25, 2011, Google released Panda to wreak havoc on the web. While it may have been designed to take out content farms, it also took out scores of quality e-commerce sites. What do content farms and e-commerce sites have in common? Lots of pages. Many with zero or very few links. And on e-commerce sites with hundreds or thousands of products, the product pages may have a low quantity of content, making them appear as duplicate, low quality, or shallow to the Panda, thus a target for massive devaluation.

My e-commerce site was hit by Panda, causing a 60% drop in traffic overnight. But I was able to escape after many months of testing content and design changes. In this post, I'll explain how we beat the Panda, and what you can do to get your site out if you've been hit.

The key to freeing your e-commerce site from Panda lies at the bottom of a post Google provided as guidance to Pandalized sites:

One other specific piece of guidance we've offered is that low-quality content on some parts of a website can impact the whole site’s rankings, and thus removing low quality pages, merging or improving the content of individual shallow pages into more useful pages, or moving low quality pages to a different domain could eventually help the rankings of your higher-quality content.

Panda doesn't like what it thinks are "low quality" pages, and that includes "shallow pages". Many larger e-commerce sites, and likely all of those that were hit by Panda, have a high number of product pages with either duplicate bits of descriptions or short descriptions, leading to the shallow pages label. In order to escape from the Panda devaluation, you'll need to do something about that. Here are a few possible solutions:

Adding Content To Product Pages

If your site has a relatively small number of products, or if each product is unique enough to support entirely different descriptions and information, you may be able to thicken up the pages with unique, useful information. Product reviews can also serve the same purpose, but if your site is already hit by Panda you may not have the customers to leave enough reviews to make a difference. Additionally, some product types are such that customers are unlikely to leave reviews.

If you can add unique and useful information to each of your product pages, you should do so both to satisfy the Panda and your customers. It's a win-win.

Using Variations To Decrease Product Pages

Some e-commerce sites have large numbers of products with slight variations. For example, if you're selling t-shirts you may have one design in 5 different sizes and 10 different colors. If you've got 20 designs, you've got 1,000 unique products. However, it would be impossible to write 1,000 unique descriptions. At best, you'll be able to write one for each design, or a total of 20. If your e-commerce site is set up so that each of the product variations has a single page, Panda isn't going to like that. You've either got near 1,000 pages that look like duplicates, or you've got near 1,000 pages that look VERY shallow.

Many shopping carts allow for products to have variations, such that in the above situation you can have 20 product pages where a user can select size and color variations for each design. Switching to such a structure will probably cause the Panda to leave you alone and make shopping easier for your customers.

Removing Poor Performing Products

If your products aren't sufficiently unique to add substantial content to each one, and they also don't lend themselves to consolidation through selectable variations, you might consider deleting any that haven't sold well historically. Panda doesn't like too many pages. So if you've got pages that have never produced income, it's time to remove them from your site.

Getting Rid of All Product Pages

This is a bold step, but the one we were forced to take in order to recover. A great many of our products are very similar. They're variations of each other. But due to the limitations of our shopping cart combined with shipping issues, where each variation had different shipping costs that couldn't be programed into the variations, it was the only viable choice we were left with.

In this option, you redesign your site so that products displayed on category pages are no longer clickable, removing links to all product pages. The information that was displayed on product pages gets moved to your category pages. Not only does this eliminate your product pages, which make up the vast majority of your site, but it also adds content to your category pages. Rather than having an "add to cart" or "buy now" button on the product page, it's integrated into the category page right next to the product.

Making this move reduced our page count by nearly 90%. Our category pages became thicker, and we no longer had any shallow pages. A side benefit of this method is that customers have to make fewer clicks to purchase a product. And if your customers tend to purchase multiple products with each order, they avoid having to go from category page to product page, back to the category page, and into another product page. They can simply purchase a number of products with single clicks.

Noindexing Product Pages

If you do get rid of all links to your product pages but your cart is still generating them, you'll want to add a "noindex, follow" tag to each of them. This can also be a solution for e-commerce sites where all traffic enters on category level pages rather than product pages. If you know your customers are searching for phrases that you target on your category pages, and not specifically searching for the products you sell, you can simply noindex all of your product pages with no loss in traffic.

If all of your products are in a specific folder, I'd recommend also disallowing that folder from Googlebot in your robots.txt file, and filing a removal request in Google Webmaster Tools, in order to make sure the pages are taken out of the index.

Other Considerations: Pagination & Search Results Pages

In addition to issues with singular product pages, your e-commerce site may have duplicate content issues or a very large number of similar pages in the index due to your on-site search and sorting features. Googlebot will fill in your search form and index your search results pages, potentially leading to thousands of similar pages in the index. Make sure your search results pages have a rel="noindex, follow" tag or a rel="canonical" tag to take care of this. Similarly, if your product pages have a variety of sorting options (price, best selling, etc.), you should make sure the rel="canonical" tag points to the default page as the canonical version. Otherwise, each product page may exist in Google's index in each variation.

Maxmoritz, a long time member of our SEO Community, has been working in SEO full time since 2005. He runs a variety of sites, including Hungry Piranha, where he blogs regularly.

My good friend Bill at SEOByTheSea has unearthed a Google patent that will likely raise eyebrows, whilst others will have their suspicions confirmed.

The patent is called Ranking Documents. When webmasters alter a page, or links to a page, the system may not respond immediately to those changes. Rather, the system may change rankings in unexpected ways.

A system determines a first rank associated with a document and determines a second rank associated with the document, where the second rank is different from the first rank. The system also changes, during a transition period that occurs during a transition from the first rank to the second rank, a transition rank associated with the document based on a rank transition function that varies the transition rank over time without any change in ranking factors associated with the document.

Further:

During the transition from the old rank to the target rank, the transition rank might cause:

• a time-based delay response,
• a negative response
• a random response, and/or
• an unexpected response

So, Google may shift the rankings of your site, in what appears to be a random manner, before Google settles on a target rank.

Let's say that you're building links to a site, and the site moves up in the rankings. You would assume that the link building has had a positive effect. Not so if the patent code is active, as your site may have already been flagged.

Google then toys with you for a while before sending your site plummeting to the target rank. This makes it harder to determine cause and effect.

Just because a patent exists doesn't mean Google is using it, of course. This may be just be another weapon in the war-of-FUD, but it sounds plausible and it’s something to keep in mind, especially if you're seeing this type of movement.

The Search Engine As Black Box

In ancient times (1990s), SEO thrived because search engines were stupid black boxes. If you added some keywords here, added a few links there, the black box would respond in a somewhat predictable, prescribed, fashion. Your rankings would rise if you guessed what the black box liked to "see", and you plummeted if you did too much of what the black box liked to see!

Ah, the good old days.

These days, the black box isn’t quite so stupid. It’s certainly a lot more cryptic. What hasn’t changed, however, is the battle line drawn between webmasters and search engines as they compete for search visitor attention.

If there are any webmasters still under the illusion that Google is the SEOs friend, that must be a very small club, indeed. Google used to maintain a - somewhat unconvincing - line that if you just followed their ambiguous guidelines (read: behaved yourself) then they would reward you. It was you and Google on the good side, and the evil spammers on the other.

Of late, Google appear to have gotten bored of maintaining any pretense, and the battle lines have been informally redrawn. If you’re a webmaster doing anything at all that might be considered an effort to improve rank, then you're a "spammer". Google would no doubt argue this has always been the case, even if you had to read between the lines to grasp it. And they’d be right.

Unconvinced?

Look at the language on the patent:

The systems and methods may also observe spammers’ reactions to rank changes caused by the rank transition function to identify documents that are actively being manipulated. This assists in the identification of rank-modifying spammers.

“Manipulated”? “Rank modifying spammers”? So, a spammer is someone who attempts to modify their rank?

I’ve yet to meet a webmaster who didn’t wish to modify their rank.

Google As A Competitor

Google’s business model relies on people clicking ads. In their initial IPO filing, Google identified rank manipulation as a business risk.

We are susceptible to index spammers who could harm the integrity of our web search results. There is an ongoing and increasing effort by “index spammers” to develop ways to manipulate our web search results

It’s a business risk partly because the result sets need to be relevant for people to return to Google. The largely unspoken point is Google wants webmasters to pay to run advertising, not get it for “free”, or hand their search advertising budget to an SEO shop.

Why would Google make life easy for competitors?

The counter argument has been that webmasters provide free content, which the search engines need in order to attract visitors in the first place. However, now relevant content is plentiful, that argument has been weakened. Essentially, if you don't want to be in Google, then block Google. They won't lose any sleep over it.

What has happened, however, is that the incentive to produce quality content, with search engines in mind, has been significantly reduced. If content can be scraped, ripped-off, demoted and merely used as a means to distract the search engine user enough to maybe click a few search engine ads, then where is the money going to come from to produce quality content? Google may be able to find relevant content, but "relevant" (on-topic) and "quality" (worth consuming) are seldom the same thing

One content model that works in such as environment is content that is cheap to produce. Cheap content can be quality content, but like all things in life, quality tends to come with a higher price tag. Another model that works is loss-leader content, but then the really good stuff is still hidden from view, and it's still hard to do this well, unless you've established considerable credibility - which is still expensive to do.

This is the same argument the newspaper publishers have been making. The advertising doesn’t pay enough to cover the cost of production and make a profit - so naturally the winner in this game cuts production cost until the numbers do add up. What tends to be sacrificed in this process - is quality.

NFSW Corp, a new startup by ex-TechCrunch and Guardian columnist writer Paul Carr has taken the next step. They have put everything behind a paywall. There is no free content. No loss-leaders. All you see is a login screen.

Is this the future for web publishing? If so, the most valuable content will not be in Google. And if more and more valuable content lies beyond Google's reach, then will fewer people bother going to Google in the first place?

The Happy Middle

Google argue that they focus on the user. They run experiments to determine search quality, quality as determined by users.

Here’s how it works. Our engineers come up with some insight or technique and implement a change to the search ranking algorithm . They hope this will improve search results, but at this point it’s just a hypothesis. So how do we know if it’s a good change? First we have a panel of real users spread around the world try out the change, comparing it side by side against our unchanged algorithm. This is a blind test — they don’t know which is which. They rate the results, and from that we get a rough sense of whether the change is better than the original. If it isn’t, we go back to the drawing board. But if it looks good, we might next take it into our usability lab — a physical room where we can invite people in to try it out in person and give us more detailed feedback. Or we might run it live for a small percentage of actual Google users, and see whether the change is improving things for them. If all those experiments have positive results, we eventually roll out the change for everyone"

Customer focus is, of course, admirable, but you’ve got to wonder about a metric that doesn’t involve the needs of publishers. If publishing on the web is not financially worthwhile, then, over time, the serps will surely degrade in terms of quality as a whole, and users will likely go elsewhere.

There is evidence this is already happening. Brett at Webmasterworld pointed out that there is a growing trend amongst consumers to skip Google altogether and just head for the Amazon, and other sites, directly. Amazon queries are up 73 percent in the last year.

There may well be a lot of very clever people at Google, but they do not appear to be clever enough to come up with a model that encourages webmasters to compete with each other in terms of information quality.

If Google doesn’t want the highest quality information increasingly locked up behind paywalls, then it needs to think of a way to nurture and incentivise the production of quality content, not just relevant content. Tell publishers exactly what content Google wants to see rank well and tell them how to achieve it. There should be enough money left on the table for publishers i.e. less competition from ads - so that everyone can win.

I’m not going to hold my breath for this publisher nirvana, however. I suspect Google's current model just needs content to be "good enough."

I've been in the online marketing space since 1998. Somewhere around the year 2000, I stumbled into affiliate marketing. I played in very competitive spaces via SEO from the get go - and ranked. In 2007, in order to scale my affiliate marketing efforts, I co-founded MFE Interactive, a website publishing company, and built out several affiliate brands.

I'd always done consulting here and there throughout the years, but never on a regular basis. To be honest, I liked (and still do like) the affiliate (on your own schedule and no one to answer to but your bank account) lifestyle.

In 2009 I made my first foray into running a consulting company. I quickly found out that there was a lot to making the jump from consultant to agency - a lot of which I hadn't thought about, hadn't properly positioned myself for and I decided to sell my shares less than two years after venturing into that realm.

After 18 months of what my friends like to call "semi-retirement" and living very comfortably off the passive income I'd spent nearly a decade developing, I decided once again to start an agency and officially launched PushFire in May of 2012 and we're already a team of 13 and growing.

But this time I knew a bit more about what I was getting into and as a result, had positioned myself better for the task. I figured I'd share with you what I've learned about making the jump from occasional consultant to agency over the last three years in hopes that maybe someone else can be better prepared for "the jump" if and when they decide to take it.

Deciding Whether to Go Solo or with a Partner

I personally know what I'm good at and what I'm not. I couldn't see launching an agency - at least one you want to experience quick growth - without having a partner (unless you want to invest the salary to have someone handle a "partner" position as a paid employee). There is simply too much for one person to do for them to do it all alone - and fast.

My personal weakness lies in operations. I don't like being chained to the office every day and travel too much promoting the agency to be the person managing operations. My personal strength lies in business development, growth and strategy. My weaknesses make me a horrible COO. My strengths make me a good CEO. I've found that the absolute key to creating a successful partnership lies in making sure that your partner has strengths where you DON'T. In PushFire, my partner Sean and I compliment each other very well. I'm a veteran with SEO and link building. He's great with PPC management (something completely foreign to me). I'm good at business development and client strategy. He actually enjoys (and is good at) managing the staff and clients. That makes him a fantastic COO.

Before you rush to launch an agency solo, ask yourself if you can really handle every aspect of running said agency (or are willing to hire to fill the gaps). If not, ask yourself who you know that would be a great COMPLIMENT to your skill set. And then make sure you vet that they actually have the skills you think they do and that you can work as a team without wanting to kill each other and being "locked" into a company from a public perspective. Work quietly together for a few months and see how it goes (that is what Sean and I did for about 3 months). And if all goes well, make it official.

Business Formation and Operating Agreement

Once you decided to make it official you need to form your business. I actually own part of an incorporation service (an accounting firm owns the other half) so I got my advice on what KIND of company to form from my partners in that site since I didn't know all the intricacies of owning a business in Texas (where I'd recently moved). If you're not sure of what kind of company will work best for you (Incorporation, Limited Liability Company, S Corporation, etc) be sure to consult with an accountant or attorney to figure it out. It's a pain to change after the fact.

If you are forming your business with a partner, it is absolutely imperative that you have an Operating Agreement from day one. IMPERATIVE. The Operating Agreement defines your ownership (if it's not a corporation which issues shares), your roles and your responsibilities. Be sure to include what happens should a partner want to leave, should you want a partner to leave, should a partner die, should a partner get divorced - EVERY POSSIBLE THING THAT COULD GO WRONG SHOULD HAVE AN AGREED UPON SOLUTION IN THE OPERATING AGREEMENT AS SOON AS YOU BECOME AN OFFICIAL COMPANY.

I don't care if your partner is your parent, sibling, friend - you need an Operating Agreement and you need one from day one. Sean and I are married and we STILL have an Operating Agreement that details out every possible scenario and what happens in the event of it. And if there is one area where you stretch the budget and pay a lawyer (and NOT use some free template online) it is for the Operating Agreement if you enter into a partnership.

The Business Paperwork

Next up you'll need to file all that awesome paperwork with the IRS, state and in some cases even your county. Getting an EIN, getting a sales tax number, and getting local operating licenses (or even finding out if they're required). And if you're hiring employees, then you'll need to file paperwork to pay unemployment taxes, etc. I personally hired a local accountant to ensure we didn't miss anything in regards to these. I'd rather pay a modest hourly fee now than much more expensive government penalties later.

Business Insurance

If you plan to have an office and employees (which is likely because you decided to no longer be a sole consultant), then you're going to want to get insurance. The basics would be General Liability and Property Insurance and Workers' Compensation Insurance.

Additionally, because what we do for a living as online marketers is not an exact science, you'll likely want to look into getting Errors and Omissions (E&O) insurance too.

The GLP and Workers' Comp insurance was fairly easy to find and get setup. E&O was another story. Most insurance agencies I spoke with didn't handle "digital marketing firms" and it took me a long time to find a recommendation to a company that not only provided E&O for people who do what we do, but also was willing to include copyright infringement into the policy (say for an employee using a picture or content they don't have permission to use, unbeknown to you). I finally was referred to TechInsurance and was able to get all three insurance policies via them.

For the record, I was not compensated in any way for mentioning them - I just had a REALLY hard time finding an E&O provider and they had all the coverage I needed and at what I thought was a fair price. Cost for the policies will vary on a multitude of factors, including your personal experience at what we do, if you've ever been sued, etc.

Also know that if you plan to take on Fortune companies as clients, this insurance - and proof of having it - will be a standard requirement and they usually request that your GLP and E&O insurance covers you in the 1-2 million dollar range.

The Legalities

As an agency, you're going to need contracts if you weren't using them as a consultant. And you're going to want them to be drawn up by (or at least overlooked and approved by) an ACTUAL LAWYER.

We had a Master Services Agreement (MSA) drawn up as well as a Statement of Work (SOW) for each service type we provide (which at the moment is SEO, Link Building and Promotion and PPC).

We also had a Mutual Non-Disclosure Agreement (NDA) created. Additionally, if you plan to use contractors or employees, you'll need contracts drawn up for each (one for employees and one for contractors). While some clients require you to use THEIR contracts (they like to keep any legal disputes in their home territory) many clients will not have these types of contracts and you will need to provide them.

For the record, I don't "hold" clients into contracts. If a client wants to stop using our services they can do so at any time. We have a 30 day notice "out" clause (for either side) in our standard MSA. But we still have contracts that clearly state what we've been hired to do and what our (and the client's) responsibilities are. Even if you don't have clients signing "one year deals" etc, you still should have contracts. Cover Your Ass - it's a statement to live by as a business owner.

Finding Office Space

Since you're opening an agency, that means you plan to have employees - otherwise you'd remain a consultant. Office space is a tricky issue for a brand new company. You often don't need a lot of space in the beginning, but if your company does well and finds itself quickly expanding, you could find yourself outgrowing your initial space soon.

I'd recommend you'd do your best to find office space (at least initially) with a short term lease (which for office space is usually 1-2 years). Also, since most of your business - at least in the beginning - will be over the Internet, you shouldn't kill your budget trying to rent a Madison Avenue worthy space in the beginning (whether you should do it at ANY time is up to you).

We took a 1 year lease in an industrial building with a killer Chicago loft like interior and a great kitchen. From the outside, it looks like a trucking company (because we're next to one). But it's got a great look and feel on the inside and 90% of our clients will never come to our office anyway. The one year lease was key because we were way too small for the space when we took it and will be way to big for the space when our 1 year lease is up.

Additionally, don't stress too much over location. I talk with a lot of folks that ask us when we'll be moving from the small town we're in (Katy, TX) to the "big city" ten minutes away (which is Houston, TX). Sean and I just discussed this the other day and I don't know that we'll ever make that "move" so to speak. It is much easier to get local press coverage and be a "top" employer in a smaller town. And again, most of our clients will never come to our office anyway. I take my cue from Marty Weintraub, who, rather than ditch Minnesota for the more "tech" scene states as a newly minted Inc 500 company, instead chose to become a pillar business there.

Contractors Vs. Employees

I was very adamant from the day we launched that we were going the in-house, local employee route. It is tempting to work with long distance contractors - especially when you know they're talented but would never move. As a consultant, I utilized long distance contractors for years (and all of our long term contractors are "grandfathered in" to this new decision).

But if you plan to grow an agency, it's my opinion that you need people in-house to effectively do it. Especially when it comes to management positions. It's simply much easier to call a company meeting with in-house staff then to arrange a Skype call between 13 people. It's much easier to develop positive relationships and company camaraderie as well.

What to Know about Hiring and Having Employees

Hiring is a task you pretty much only learn via experience. You have to make sure you find people who will fit in with the company culture you want to build - so you'll need to put some thought into what that is before you start posting "help wanted" ads.

To attract great employees we've learned two things. The first is not to skimp on paying for a job posting, especially if you live in a smaller town. We post our ads on Monster.com and haven't been disappointed in the quality of applicants we've received by doing so. Prior attempts using more local services were a waste of time, especially when looking for people with online marketing and/or technical skills - they're all looking online.

Secondly, you need to offer benefits if you want to be competitive with the other companies around you. A great company culture and an exciting position usually don't make up for a lack of health benefits in the United States. We looked to our banking service to get us recommendations for small business health insurance.

They introduced us to Digital Insurance who got us a great quote for health insurance for our growing team (again, I'm only mentioning them because we found them to be awesome and extremely helpful). When looking at health insurance keep in mind that - at least here in Texas - you need 75% participation in your plan and need to pay a minimum of 50% towards the health insurance premiums of all your covered employees.

To give us an edge over competing employers, we decided to get a higher quality plan with a low co-pay and lower deductible and contribute more than the required 50%. Make sure that you consider your health insurance costs for your employees when deciding on salaries. Even if you don't have health insurance for the first few hires, when you DO implement, you'll have to contribute for everyone, regardless of if their salary took it into account.

Additionally, once you're past having one or two employees, you're likely going to need an HR manual. Our accountant was able to get us a standard HR Manual we were able to edit to fit our specific needs - and once again, have a lawyer look over the final product.

We also work hard to keep the team motivated. From team building events with prizes (our latest was laser tag, our next outing is bowling) to performance incentives (this month, we're giving away the New iPad as a performance bonus), you'll need to ensure your team is emotionally satisfied in addition to being financially satisfied to keep retention high. We appreciate our team and their dedication to the company and do our best to show them that.

Creating Internal Processes and Tools

Hiring employees means you're going to need to train them. In the beginning, you'll likely do this by doing it verbally, but when you get to the point that you're hiring frequently, you'll want internal training documents for your new team members to read first and for you to answer questions about later. And different jobs will likely require different processes and training documentation. We've been creating this as we go. But we've found that a lack of written documentation can cause confusion and frustration. So the sooner you can get everything on pen and paper, the better.

Additionally, at some point you're going to find yourself doing a lot of repeated basic processes that could be better handled by legitimate automation. We just hired our first developer for PushFire to begin building internal tools to not only make our team's job easier, but to make it less mundane as well. Be sure to keep an eye when hiring on if you're hiring because of a genuine need or because your current team is wasting time doing things they don't need to do. When the latter occurs, I'd look into hiring a developer to fill those productivity gaps with automation. We're already seeing the benefits of making our team's life easier.

In the meantime - don't try and reinvent the wheel. There are tons of great tools out there that can make your team more productive. From Raven Tools to Ontolo to HighRise to tons of other services - there is likely someone, somewhere with a tool to make your specific team's job easier. Seek them out. Then hire someone to build what ISN'T available as you grow.

The Books

My opinion? Never do the books yourself. EVER. Because saving yourself a few hundred to a few thousand in accounting fees could cost you BIG BUCKS down the road. We use FreshBooks for our invoicing and have an accountant who then downloads the FreshBooks data into Quickbooks.

I find FreshBooks easier (though I used Quickbooks until recently) and she prefers Quickbooks. Our accountant provides us with a Profit and Loss (P&L) statement every month and keeps track of our invoicing and cash flow. This allows us to ask her any information we need to know financially while keeping our eyes focused on the business growth and day to day operations. For now, we outsource accounting, but we know that at some point we will need to take it in house as we grow.

We also utilize the payroll services at our bank (Wells Fargo) because it's easy and they pay all the required taxes for us. Our accountant still runs our payroll, but through their service. Additionally, most payroll companies offer you a guarantee that if you're ever hit for them improperly accounting for taxes, that's on them and not you. To me, it's worth the (what I consider) small fee to utilize the payroll services.

I'm Still Learning

I'm by no means the definitive source on making the move from an individual consultant to an agency. I can't offer advice on financing or funding because we've bootstrapped ourselves the whole way. But I hope the above let's you know some of what to prepare for and what you'll need, at the very minimum, to launch a successful consulting agency.

Lastly, DREAM BIG. DO GOOD WORK. BE A GOOD PERSON, PARTNER, BOSS, CLIENT AND SERVICE PROVIDER.

P.S. If you live in, near or are willing to relocate to Houston? We're hiring. ;-)

Google timed a nice Friday evening release to update of their policy toward copyright infringement.

Starting next week, we will begin taking into account a new signal in our rankings: the number of valid copyright removal notices we receive for any given site. Sites with high numbers of removal notices may appear lower in our results.

Wow. Sounds like trouble. Surely that means that YouTube's rankings are about to get torched.

Oh, nope. One quick exemption for the video king:

This data presents information specified in requests we received from copyright owners through our web form to remove search results that link to allegedly infringing content. It is a partial historical record that includes more than 95% of the volume of copyright removal requests that we have received for Search since July 2011. It does not include:

• requests submitted by means other than our web form, such as fax or written letter
• requests for products other than Google Search (e.g, requests directed at YouTube or Blogger)
• requests sent to Google Search for content appearing in other Google products (e.g., requests for Search, but specifying YouTube or Blogger URLs).

Google does not state where the thresholds will be set & grants blanket immunity for themselves, yet they (illegitimately) emphasize that they are being transparent.

Only copyright holders know if something is authorized, and only courts can decide if a copyright has been infringed; Google cannot determine whether a particular webpage does or does not violate copyright law. So while this new signal will influence the ranking of some search results, we won’t be removing any pages from search results unless we receive a valid copyright removal notice from the rights owner. And we’ll continue to provide "counter-notice" tools so that those who believe their content has been wrongly removed can get it reinstated. We’ll also continue to be transparent about copyright removals.

YouTube vs Sites Cleaner Than YouTube

Courts have ruled that embedding a YouTube video is not copyright infringement. The EFF has mentioned that embedding a video is simply a link.

And yet, a UK student faces up to 10 years in jail in the US for founding a crowdsourced site which links to sites that allow you to watch TV online.

Kim DotCom suffered a militant raid on his house & had his assets frozen for running MegaUpload, which was a tiny spec of dirt compared to the size of YouTube.

On the copyright front YouTube was rotten from the start:

• "In a July 19, 2005 e-mail to YouTube co-founders Chad Hurley and Jawed Karim, YouTube co-founder Steve Chen wrote: 'jawed, please stop putting stolen videos on the site. We’re going to have a tough time defending the fact that we’re not liable for the copyrighted material on the site because we didn’t put it up when one of the co-founders is blatantly stealing content from other sites and trying to get everyone to see it.'"
• "Chen twice wrote that 80 percent of user traffic depended on pirated videos. He opposed removing infringing videos on the ground that 'if you remove the potential copyright infringements... site traffic and virality will drop to maybe 20 percent of what it is.' Karim proposed they 'just remove the obviously copyright infringing stuff.' But Chen again insisted that even if they removed only such obviously infringing clips, site traffic would drop at least 80 percent. ('if [we] remove all that content[,] we go from 100,000 views a day down to about 20,000 views or maybe even lower')."
• "In response to YouTube co-founder Chad Hurley’s August 9, 2005 e-mail, YouTube co-founder Steve Chen stated: 'but we should just keep that stuff on the site. I really don’t see what will happen. what? someone from cnn sees it? he happens to be someone with power? he happens to want to take it down right away. he get in touch with cnn legal. 2 weeks later, we get a cease & desist letter. we take the video down.'"
• "A true smoking gun is a memorandum personally distributed by founder Karim to YouTube’s entire board of directors at a March 22, 2006 board meeting. Its words are pointed, powerful, and unambiguous. Karim told the YouTube board point-blank:
  'As of today episodes and clips of the following well-known shows can still be found: Family Guy, South Park, MTV Cribs, Daily Show, Reno 911, Dave Chapelle. This content is an easy target for critics who claim that copyrighted content is entirely responsible for YouTube’s popularity. Although YouTube is not legally required to monitor content (as we have explained in the press) and complies with DMCA takedown requests, we would benefit from preemptively removing content that is blatantly illegal and likely to attract criticism.'"
• "A month later, [YouTube manager Maryrose] Dunton told another senior YouTube employee in an instant message that 'the truth of the matter is probably 75-80 percent of our views come from copyrighted material.' She agreed with the other employee that YouTube has some 'good original content' but 'it’s just such a small percentage.'"
• "In a September 1, 2005 email to YouTube co-founder Steve Chen and all YouTube employees, YouTube co-founder Jawed Karim stated, 'well, we SHOULD take down any: 1) movies 2) TV shows. we should KEEP: 1) news clips 2) comedy clips (Conan, Leno, etc) 3) music videos. In the future, I’d also reject these last three but not yet.'"

Broader Copyright Questions

There still are a lot of murky questions in Google's "transparency."

• If a person embeds an image from Imgur, ImageShack, TinyPic, PhotoBucket or elsewhere & the page that has a hotlink gets a DMCA how does that count?
• If a brand is large enough does it take many DMCAs to get hit?
• Is there any analysis of the underlying business model of the site? What happens to document storage sites like DocStoc & Scribd, or even image sites like Pinterest?
• What happens to sites that link at penalized sites too frequently?
• What happens to ad networks that frequently fund such copyright violations?

HUGE Impact on the Web

Has anyone registered DMCASEO.com & DMCA-SEO.com yet? ;)

In terms of impact on the web for publishers, this change is every bit as big as Florida, Panda & Penguin. It may not seem so at first (as it will take time for market participants to consider the uses) but this is a huge deal. Consider some of the following scenarios...

• You try to create something like YouTube for another form of content (Pinterest?) and it gets hit as spam for following Google's lead.
• You offer a free blogging platform that competes with Blogspot, but it gets hit as spam for following Google's lead.
• You decide to create a project like Google's book scanning project & you get hit as spam for following Google's lead.
• You run an ad network & start growing quickly. As you grow some sketchier publishers enter your ad network. Like Google AdSense, a large portion of your ad network is filled with sites that have copyright violations on them. Suddenly working with your ad network gets people hit as spam because your business model is too similar to Google's.
• You create a new social network & are struggling to compete with Google's preferential ranking & hard coded placements of their own network. You make your network more open to encourage growth & you get hit as spam.
• If You are Amazon or eBay you can afford premium featured content to pull up your other listings. But if you can't afford their cost structure & hire freelance writers or work with outsourced workers to create some of your content & they use some copyright work without you knowing. But does Amazon now have to vigilantly review their reviews for plagiarism?
• A competitor licenses some of their content as Creative Commons for years & doesn't mind wide use of it. Then you use it & one day they see you as a competitive threat and remove their Creative Commons license & bulk DMCA you. Or you have a lifetime syndication deal with a company, they later change the policy & claim that your documents are forged.
• Getty images presumes you didn't license an image that you did & files a DMCA. At some point there is no purpose in targeting the webmaster or host...just go direct to Google knowing that you can create the equivalent of a "patent trolling" styled business model where you create a business model where it is cheaper for people to pay to have the issue resolved the quick way before they lodge a formal complaint. Some organizations might even have a subscription service set up where you pre-pay for immunity.
• A former employee who wrote content for you claims you used it without permission. Or that same former employee used pirated images & longish quotes from other sources that they didn't disclose to you that they now highlight via DMCA.
• You license data from a source & they do a mid-contract change leveraging the small print & have a bot lined up to send 40,000 DMCAs against you if you do not agree to the higher pricepoint.
• Google is considering making an investment in your site & you want too much money. As an edge case near the threshold of this copyright limit you know you have immunity if you join the borg, but lack it if you don't work with them.
• Big media players that play in the gray area will be fine, but smaller sites that try a similar model will be sunk by DMCAs and/or legal fees.
• Your leading competitor realizes that your blog publishes comments by default with editorial review (and that even later has lax review) and then they file DMCA reports against you. Or they could just grab chunks of content from Google's leaderboard of complainers and post them into your web forum, knowing that those companies will file a DMCA report against you.
• A site has some content public & some behind a paywall. With a page partially indexed, how does Google respond to DMCA requests when the alleged infraction is behind a registration wall or paywall?
• A competitor (inspired by Google no doubt) hires off shore "contractors" to copy your site & then file DMCA reports against you in bulk. How long until people start uploading their own content to file their own DMCAs against certain sites with user generated content?
• Even if your site is 100% legal, a combination of ignorance & crowd-driven vigilante justice can still take you down.
• Any site that offers interactive features & has user generated content is at risk of being labeled as spam unless they have tight editorial control over user generated content. And at the same time, Google can enter vertical after vertical with scrape & displace garbage knowing that they don't have those editorial costs due to their self-granted blanket immunity.
• If you do not register your sites with Google & counter claims (even bogus ones) then you are seen as being a spammer. And if you register with Google then when they don't like something one site does they can hit other sites all at the same time. No point going to the host or registrar, go direct to Google & start building up negative karma.

Why did Google feel the need to grant themselves blanket immunity from the policy?

That question was largely missing among the fanboi blogs & journalists who were encouraged by Google's "transparency."

24 Karat Pyrite On Sale for Only $100 an Ounce

If YouTube is going to win big, then that's a great place to invest, right?

Maybe not.

Some venture capitalists are investing in YouTube channels, but that is a fool's game.

• Google is also investing in select channels (like Machinima). It is quite hard to outperform Google in returns while investing into a platform that they control & thus have better data on than you ever could.
• As YouTube's dominance increases (and it will now that competing platforms with a similar business model will be smeared as spam), you can count on them offering premium partners crappier revenue share deals in years to come. They will offer nice deals to Warner Bros. & such, but the independent smaller players will get cut out of the ecosystem in much the same way as they did in Google's organic search results.
• Google, prince of transparency (for everyone but Google), requires that premium publishers *not* disclose the terms of their deals: "The Partner Program forbids participants to reveal specifics about their ad-share revenue. Rates can vary depending on the size and demographics of the partner’s audience and an array of other metrics."

Note that I don't claim YouTube is a bad host for your own content, but that I am skeptical in applying the VC model to it with a belief that you can out-invest Google on their own site; particularly when they own the dominant platform, control the non-public revenue share rates, invest in competing channels & can offer free promotion + higher rates to anyone they invest into in order to dominate the category.

And the issue isn't just video either. The same dynamic can apply to just about any other infrastructural layer. For instance, Google could buy out a torrent site (say like uTorrent) and have that site gain immediately immunity for being part of the borg, while other sites that compete now absorb both greater editorial filtering costs & greater risks that destroy their ROI.

As Google continues to lock down search, you can expect more smart publishers to hedge investments in search and YouTube with investments in proprietary non-search applications that Google can't take away.

The Devil is in the Details

"We are optimistic that Google’s actions will help steer consumers to the myriad legitimate ways for them to access movies and TV shows online, and away from the rogue cyberlockers, peer-to-peer sites, and other outlaw enterprises that steal the hard work of creators across the globe. We will be watching this development closely — the devil is always in the details — and look forward to Google taking further steps to ensure that its services favor legitimate businesses and creators, not thieves." - Michael O’Leary, Senior Executive Vice President for Global Policy and External Affairs of the Motion Picture Association of America, Inc.

The concerned with Google pitching themselves as the preeminent authority on copyright is they have consistently played both sides of the fence.

• Their image search offers thumbnails
• they offer a "quick view" version of PDFs in the SERPs
• their search results offer cached versions of pages
• they have long opposed piracy-related legislation & created a viral campaign against it
• they run a social network (which often outranks the original content source in the search result)
• they run a blog host (which has a long history of being one of the spammiest blog hosts with sketchy content)
• they run YouTube, the video site with the most piracy on it
• they run the ad network which funded the most piracy-driven content (at one point their ads were even on the biggest warez sites)
• they monetized scraper sites like Mahalo for years (still do!)
• they willingly ran AdWords ads for over 50 thousand advertisers selling counterfeit goods
• for years they included piracy-related searches in Google Instant (along with the above mentioned ads) to try to force brands to need to buy their own brand on broad match to try to mask over the problem
• they scanned millions of ebooks without permission (a class-action lawsuit over this is ongoing)
• when they penalize sites with algorithms like Panda or Penguin, they often allow (AdSense funded) scraper sites to outrank the original content source
• they scraped content from Yelp & TripAdvisor to populate their places page while giving those businesses the ultimatum that if they didn't like it, they should block Googlebot (and it took government scrutiny to curtail the practice)
• they continue to scrape in more information from publishers to power the expansion of their knowledge graph

When Google was competing against YouTube, this was how they viewed copyright internally.

Business Objectives Drive "Relevancy" Signals

Google is a big player in business online and off. They can sell private data exclusively & their online profits are so huge that they are now buying auto loan bonds.

Now that Google wants to sell premium content they (sort of) respect copyright (& are willing to hold the rest of the web to a higher standard than themselves to create this impression).

I have long believed that relevancy signals were often politically driven & that internal business development goals often lead or create various signals. Certainly that was obvious when Google+ was hardcoded in the search results. It was equally true when Knol outranked the original content sources. Google frequently pretends to be (belligerently) unaware of externalities, but when the issues impact their own business they gain an elevated sense of importance.

• We saw that when Google mentioned that they created QDF because Google Finance didn't rank: Mr. Singhal started to worry that Google’s balance was off. When the company introduced its new stock quotation service, a search for “Google Finance” couldn’t find it.
• We saw it again when universal search was rolled out only *after* Google bought YouTube.
• We saw once more when their hard stance on (against?) property rights for a decade quickly flipped like a switch after Android Marketplace & Google Play stunk up the joint from not making enough sales to be significant. Google needs a complete media catalog to become a default purchase hub & they can't get that until they display that they respect property rights.

And these business objectives not only influence the relevancy algorithms, but also the editorial guidelines.

• Renting a link to try to rank higher for a relevant keyword is sinful, but offering up undisclosed payola to influencers who manipulate the entire media ecosystem is totally reasonable.
• Info harvesters are not allowed to buy AdWords ads & some forms of lead generation are marketed as being spam, but lead generation forms in the search results & overriding user privacy settings in their browser are totally cool.

And even while Google is rolling out this "copyright violators are spammers" algorithm (which they are exempt from) they still chug on with their ebook offering:

They posted several of my 41 books up as free downloads (some were missing a few pages at most a single chapter) It took several e-mails from me pointing out that they were infringing copyright before they took them down. During the time my books were free on Google my sales of e-books fell dramatically. " - K C Watkins

When Google started scanning books an internal document stated: “[we want web searchers interested in book content to come to Google not Amazon” ... or, as put another way, in that same document, “[e]verything else is secondary … but make money.”

Here are some quick tips on how you can use blogrolls to compose a list of as many related blogs as possible, then checking those blogs to see if any of them return 404s.

A good free tool to use for finding blogs is the SoloSEO link tool, which will pull up blogs (and other assorted advanced search operators) based on the keyword you input.

After going through the ways to build the list, I’ll run through a few ways you can use it.

Step #1: Find a few blogs

Start off by finding a few related blogs that have blogrolls. The more blogs & the longer the blogrolls of each, the better. This is our seed list that will soon multiply itself.

Step #2: Multiply your list

Take the URLs of these blogs and throw them into Buzzstream’s blogroll list builder. In the example below, I just started off with one (an HR blog):

Once you hit Go, it searches the blog(s) and finds every blog in their blogroll (note: I was having issues with it in Chrome, so if you do as well, switch browsers):

Step #3: Rinse and repeat

Next, download the results as CSV. Open it up in Excel, then copy & paste the blogroll URLs back into the list builder tool (you might have to refresh/reopen the page).

Keep doing this until you have a sizeable list. The bigger the better, but as you keep expanding, you’ll run into the issue of irrelevant blogs entering the list, so just keep that in mind.

Step #4: Check the status of the URLs

Next, throw your list of URLs into Citation Lab’s URL Status Checker tool. This will check to see if any of the URLs in the blogroll are 404s.

Once the report is finished, you can export it as a CSV.

Step #5: Pick your poison

Now it’s up to you how you want to use this list for broken link building. Here are a few popular options:

1. Blogroll Links

Go down the list of 404s and plug them into any of the bigger link tools on the market, Open Site Explorer, Ahrefs.Com, or MajesticSEO. Scan their top links for any that are coming from a homepage. These are almost always blogroll links.

Go to these homepages and use the Check My Links chrome extension, because if one link in their blogroll is broken, then there’s usually a few others.

From there, reach out to the bloggers letting them know of the broken links. Then ask them if one of them could be replaced with a link to your blog since it’s related.

2. Dead Content Links

Once again, plug the 404s into the link tool of your choice. This time however, instead of checking their links, click on Top Pages section in the tool.

Find their most linked to content, double check each to make sure the page is no longer available, then plug those URLs into Archive.org to see what content used to be there.

Next you’re going to rewrite the content, but do your best to make it even better. If it’s a little outdated, then update it.

This content will not only attract links on its own with proper promotion (the old one did, the new one probably will as well), but you can now use this for broken link building.

Take the URLs of the broken, linked-to content and plug it into your link tool(s). Go down the list and find the most valuable links to that content, then reach out to the webmaster/blogger of those sites and let them know that page is broken.

Tell them that “you took the burden” of recreating it, and that for the sake of their readers, they should update the broken link by now linking to you.

Other purposes

You can also use the initial list of 404s to see if any of those domains are:

• Expired & available to register
• Available to purchase in auctions
• Available to outright purchase

If they have enough links to them, you can put some content up and include a few links back to you. If you’re going to do this, make sure you put content up on their Top Pages, since these are already loaded with link juice.

Finally, you can take that list of blogrolls, remove all the 404s & duplicates, and use the Mozscape API (with excel) to find the most authoritative blogs in your niche. From there, build relationships, ask for product reviews, or anything else you can think of.

Final thoughts

So many of the tools we have ready at our fingers can be used in various combinations. Don’t be afraid to experiment.

What do you think of this process? What do you think can be improved? I’d love to hear your thoughts below!

Bio

Jon Cooper is an SEO consultant and the author of Point Blank SEO, a link building blog. Follow him on Twitter @pointblankseo.