The rapid changes in the search industry over the last sixteen months have left many web publishers wondering whether they should pivot their business models or exist the industry entirely. This is a difficult question for business owners who have invested years of their lives and much of their wealth in firms which may no longer be viable contenders in the "new" search industry.

SWOT analysis is a technique which business owners can use to strategically analyze their businesses in relation to their competitors and the marketplace as a whole. SWOT stands for Strengths, Weaknesses, Opportunites, Threats.

• Strengths are attributes of the organization which provide an advantage in the marketplace.
• Weaknesses are attributres of the organization which cause a disadvantage in the marketplace.
• Opportunities are actions the organization could take to create an advantage in the marketplace.
• Threats are events which could happen in the environment and cause the organization to be disadvantaged.

The first two areas, Strengths and Weaknesses, focus primarily on the internal attributes of the organization. The last two areas, Opportunities and Threats, focus primary on how the organization may be affected by external events.

Specifics for Web Publishers

Many firms in the same industry will share similar Strengths and Weaknesses. Even more so, most firms in any industry will be responding to similar Opportunities and Threats.

Strengths and Weaknesses

Take a look at your organization. If you feel that your organization has an attribute which makes it stronger than it's competitors, add that to your Strengths list. If you feel that your organization has an attribute that makes it weaker than it's competiors, add that item to your Weaknesses list.

• Access to Funding
• Brand Recognition
• Domain Authority
• Industry Connections
• Technical Skills
• Marketing Savvy
• Vertical Expertise

Examples of Strengths might include:

• We have ready access to venture capital
• We own a widely recognized brand name
• We own a PageRank 8 domain
• I have Matt Cutts on speed dial
• Our technical team members are experts in our platforms, development tools, and applications
• Our marketing team members can make linkbait about lug nuts go viral
• We invented this niche and our competitors have no hope of ever catching up

Examples of Weaknesses might include:

• Our working capital is limited to what's in my wallet
• Our top domain is a hyphenated .us domain
• We're hoping to gain PageRank at the next update
• Matt Cutts blocked me on Twitter
• Our technical team is outsourced to Pakistan
• Our marketing team is outsourced to Bangladesh
• I read a book about this niche and it seems very exciting

Opportunities and Threats

The same event might be an Opportunity or a Threat, depending upon how your organization can respond to it. Search is a zero-sum game. For every winner, there must be a loser.

Take a look at your organization. If you feel that your organization has the ability to benefit from a coming change in the business environment, add that to your Opportunities list. If you believe that your organization is at risk from a coming change, add that to your Threats list.

• Our niche (travel, local, etc...) is being taken over by Google (unless you are Google)
• Our niche is being persecuted (gambling, medication) or promoted (green energy, section 8 housing) by the government
• Our niche is being regulated by the government, which benefits large companies and hurts small ones
• Our niche is being increasingly dominated by the top brands (unless you are one of the top brands)
• Our niche is growing (iPads) or shrinking (Blackberries)
• Profitability in this niche is rising (medical training) or falling (almost everything else)
• Some marketing tactics may be filtered or penalized (directory submissions, blog commenting, profile building)
• Significant competitors are entering (or leaving) the niche

Examples of Opportunities might include:

• Legislation could force consumers or businesses to buy our goods and services
• Government regulation could force small competitors out of the market, and we're a large competitor
• Google is increasingly ranking the top brands for all searches, and we're a top brand
• Our niche is growing
• Profitability in the niche is rising
• Our marketing tactics are being increasingly rewarded by the search engines
• Our niche has significant barriers to entry which prevent competitors from entering the market

Examples of Threats might include:

• Legislation could make our business illegal in our country
• Government regulation could force small competitors out of the market, and we're one of those small competitors
• Google is increasingly ranking the top brands for all searches, and we're not a top brand
• Our niche is shrinking
• Profitability in the niche is falling -- unless you can operate on thinner margins than your competitors and take their market share when they fail
• Our marketing tactics are being increasingly filtered or penalized by the search engines
• One of our competitors just did an interview with Forbes bragging about the high profit margins in this niche

Responding to the Results of Your Analysis

After listing your Strengths, Weaknesses, Opportunities, and Threats you should have a pretty good idea where your business stands. From here, it's time to take advantage of this new knowledge.

The web publishing industry is currently undergoing a major contraction. Some organizations will choose to continue in this business, while others will choose to pivot into related business or to exit the industry entirely. AdSense publishers may decide to move into affiliate marketing or selling white label products. Web publishers may decide to halt development on their own projects and offer their services as SEO's to large enterprises. Entrepreneurs may simply close their companies and accept positions with larger firms.

If your niche is travel, which Google is slowly taking over and Wikimedia is considering a push into, you might consider moving to a different niche, pivoting your web publishing business into an SEO firm, or moving into the nascent eBook market. If your niche is 3D printers, you might seek funding to stake out early market share in a niche that may be about to cross the chasm from the early adopter stage of development.

If you have deep knowledge, experience, and connections in your niche, you might try to stick with it and be the last man standing after your weaker competitors have failed. If your knowledge is less niche focused and more related to publishing and marketing, you might sell SEO services or take a job with one of the huge multinational brands which Google is currently favoring in the SERPs.

If you have access to large amount of venture capital, you might take advantage of that to become one of the large brands which Google prefers to rank. With enough funding, anything can be ranked well in Google. I would caution, however, against entering a niche which is likely to be on Google's roadmap. Google, in being able to control the order of search results, has an unbeatable advantage in promoting their own properties (YouTube, Google+, etc...).

As margins in the industry are falling in our race to the bottom, you may even find a significant competitive advantage in having a lower cost structure than your competitors. Lower costs create larger amounts of retained earnings which can be used to fuel development and growth.

Summary

The two most important aspects of SWOT analysis are to be honest with yourself and to take action based upon your analysis. As Virgil wrote, fortune favors the bold. Be bold in your honesty and your actions and fortune will smile upon you.

Will Spencer is the CEO of MemeBridge and Alpha Geek at The Tech FAQ.

The best part about a growing and very quickly changing industry is the diversity of viewpoints; the worst part is the exact same thing because sometimes 1 always equals 1 and doesn't need bullshit in lieu of evidence. I try my best to stay out of the limelight and just focus on making things happen. However, occasionally a topic will bother me so much that I have to chime in. The last time was over 5 years ago so I figure I'm due to speak up again. Today's topic? Negative SEO. My issue with the topic? Deniers.

There've been several posts on how negative SEO doesn't exist (those are the worst) or that maybe it exists but only weak sites can get hit (in other words, people with opinions that didn't do any testing). I'd like to put those topics to rest as best as a guy that keeps to himself can. I really should be able to do this in one sentence, but in the event what I write as the second half of this sentence doesn't do it for you, I have a couple stories; if crappy SEO of over-optimized anchors and junky links are to blame for ranking drops, how can it be said one cannot do this to someone else, and even if you were to deny this, then why the sudden rush to denounce certain links? On to some anecdotes!

While leading a training session overseas I mentioned a site I watched get hit by some negative SEO activities. I know that it was negative SEO and not a slip up on the SEOs' part by virtue of knowing the history/team behind the site and watching it as part of my normal data routine; the site was managed by the kind of guys that get asked to speak at SEOktoberfest...the kind of people I'd go work for if my bag of tricks ever ran out. Ok, so you're asking how I know it was negative SEO. The easiest explanation is that I watched the site spike heavily with on-theme anchors from junk sites over a one week period and was filtered shortly thereafter. It stayed filtered for just under few months, but 2 days after discussing the site and explaining how I knew the site was hit it magically reappeared (yes, there were googlers in the audience).

If you are skeptical then your first response better be that I'm only loosely describing one example so let me say that in the same industry where I've shared my knowledge of the subject on some more sophisticated methods (first released in the SEObook community), I feel almost like an information arms dealer since even the larger brands have themselves or through affiliated relationships been clubbing each other over the head. You read that right; I explained how I thought negative SEO could be employed and then watched a bunch of people actually do it, repeatedly. Unfortunately, I was hit too, but that's a different issue. In this particular industry, the only people left standing now are some poorly matched local results with fake reviews, a bunch of hacked domains, and the flotsam of macroparasites that gained popularity post Penguin. The only one that came back? The one I publically shared at a conference, explaining exactly how they were a victim based on the link patterns that didn't fit with the site's history over a several year period.

I'll wrap this up with a bit of humor. As a joke a friend of mine asked me to negative SEO him for his name. Let's say his name is John Doe and his domain is johndoe.com. The negative effect was temporary, but I was able to get him filtered for a little while on his name for maybe 120 seconds of my time and less than $50. The site did come back after a few days, but our mutual feeling on the matter is that for an extra $50 double-dose I could probably get the site filtered again. Neither of us wants negative SEO to get any more prevalent than it already is, so I'll skip the details on exactly how it was performed. There are multiple forms of negative SEO significantly scarier than someone with a copy of xrumer and in some cases there is very little you can do to prevent it; if a jerk wants to take you down, it can happen. If your niches begin to look like the wasteland I described above where I shared my thoughts a little too freely, then heaven help you because it doesn't look like Google is going to.

Cygnus has been involved in search since 1997 and loves tackling new and interesting (and of course lucrative) projects. Follow @Cygnus on Twitter for his rants.

(Image Source – my poorly built logo creator)

In my long career as an Online Marketer, I have had to often pick an agency to partner with or to carry out the different mixes of online marketing, such as SEO, Paid Search, Affiliate marketing, Email Marketing, Analytics, Social Media etc etc. Fact is, I am a rounded marketer who, although spends time on SEO the most, understands and works in most online fields. This means I am often the go to person for brands when they want to pick an agency to work with.

One such day, while in the middle of listening to an agency pitch, I felt quite a bit perplexed. The two pitches I heard were vastly different, and I wasn’t happy with either. The core problem I had with agency pitches was around the following observations:

• They tend to be too boiler plate. Replace your business with any other and it may feel that it doesn’t matter.
• They miss the main questions that a business may want the answers for.
• They miss the opportunity to really sell their USP (Unique Selling Proposition)
• If they are customised, they lose some of the generic elements necessary
• They often leave too much room for questions, which can take the process either way.

The above is often true, even if you have issued a clear brief to your agency as to what you would expect to see, or what questions you would want answered. Any agency can follow a brief and answer it, but very few in my opinion see beyond the brief. And as an experienced agency recruiter for brands, I would like to see much more answered within the pitch than I am still seeing.

Many agencies don’t make it CLEAR what they aim to achieve, nor do they try to CLARIFY what the businesses need or want.

Image Source: http://www.flickr.com/photos/nomad9491/2399208582/

So I formulated the CLARITY model for briefs, which could be a frame work for answering pitches – help you answer your brief, while allowing you to demonstrate much more than the questions at hand. CLARITY, in my opinion, is an agency model that would score very highly but would also form the ethos of an agency environment that is really geared to helping their clients.

At the same time, the model has helped me pick the right agencies over and over again, and as such could be used by in house Digital marketers to form their own judgement sheets.

Although many SEObook readers are SEOs, many are in the agency environment themselves having to pitch, or in house and may have to from time to time help pick an agency. Many are like me, interested in SEO, but involved in much more online and offline marketing. As a result, I felt that sharing my model may help at least a few readers.

Warning: This is a rather long post, and could sound a bit preachy.

Defining CLARITY

The model is a mnemonic that covers the 7 elements below:

1. Communication
2. Learning and Development
3. Access to Support
4. Respectable and Responsive
5. Intelligence both in people and processes
6. Technology and innovation
7. Yield Based Approach

Communication

Image Source: http://www.flickr.com/photos/ventodigrecale/315084105/

When working with any outside agency, the type of communication is vital. The overall tone and approach as well as the individual team members all add to a business’s communication strategy. Some businesses like being overly formal, while others find that formal approaches are annoying and could hinder work. When picking the right agency for you, understanding how they communicate with clients and amongst themselves is extremely important to make sure that the working relationship is a healthy one.

For example, how your agency dresses and behaves in meetings is fairly important – it is a subliminal communication signal. As part of a pitch process I was involved in, one very talented SEO turned up, but was wearing ripped cuff jeans.

The Head of Ecommerce was at the meeting and was not impressed that for such a large pitch, the key person delivering was in scruffy jeans.

Result? They didn’t get the gig because the Head of Ecommerce was distracted by the fact that this person hadn’t bothered to dress appropriately.

My tips to people running a pitch:

• Find out what the communication standards are for the business – do they favour email over phone, or vice versa?
• How do the key stakeholders behave, dress or communicate? If they are formal in their communication, you may have to resort to matching their style, or loosen up if they are a team that prefer informal approaches.
• Keep your presentations clear and concise, and ALWAYS identify your communication strategy, especially things like reporting regularity, formats, availability of account holders, and even down to how you would deal with a crisis situation that requires communication out of hours.
• When presenting or pitching, make the objectives clear – many a pitch goes a bit haywire if the summary of the presentation or of the overall service isn’t clear.

Learning and Development

Image Source: http://www.flickr.com/photos/smemon/4984567320/

In the digital world, things change daily. And sometimes small changes make big difference – take the latest Penguin Algorithm update from Google. The change in the way Google is treating a majority of low quality links has caught many an agency unprepared to turn around quickly - and to my knowledge a few, if not a majority, have since drafted communication to their clients about the change and what it means for their SEO.

As part of a pitch process, identifying the potential for such large scale impacts on channels is important – but more important is to show that your team is up to the challenge. It is important to indicate that your team is an ever learning, ever developing beast, and it may be worth showing some examples where you have bucked the trend, or foresaw changes and indicate how you managed to save, support or shift your other clients strategies.

For example, knowledge of your discipline isn’t enough – you have to garner some knowledge about your potential clients industry and changes occurring within it, such as legislation.

In one pitch I was part of, we identified that the client was suffering from Voucher Code site abuse – where the voucher code sites would consistently rank for long tails of the business. Interestingly, the client hadn’t picked up on the fact that the reason that they were losing a lot of organic traffic wasn’t because they had had ranking losses – rankings were all fine. The reason they were losing their traffic was because this voucher code site was ranking immediately below the clients sites with a discount offering! Our strategy tackled that by investigating the legislation, both applied and subscribed to within the voucher code industry in the UK, and as a result managed to craft a communication brief, which would enable the client from stopping the abuse.

We won the contract, and the work we did was implemented. In the end we came to an agreement with the site in question and they stopped. Clients SEO traffic and conversions soared.

A good agency has an arsenal of resources at its disposal – indicating these as well as how you constantly add to the armoury is very important – after all, often agency relationships with clients can span years.

Access to Support

Image Source: http://www.flickr.com/photos/pat-h3215/7500230750/

For any business, support is important. For any business with large budgets and complicated marketing campaigns, support is critical. Although most agencies work in a 9am to 5pm daily shift 5 days a week, many brands don’t see themselves that way. Their business online is churning round the clock, 7 days a week.

Which means a crisis, issue or even an opportunity may raise itself at the least possible convenient time. Although in a pitch these sort of issues aren’t expected by most businesses, I often find that if an agency covers it, they tend to get “bonus points” especially if they highlight likely scenarios and how they would respond to them out of hours. Although this point is a subset of communication, it is also important enough as a winning point to be isolated.

One SEO agency I hired for a holiday business proposed that during peak periods of the business refreshing site wide content, (an annual occurrence) they would send one of their content SEOs to sit with the content team to start optimising content as it gets written, and getting it to the publishing team within a very short period of time. Excellent foresight, and was one of the contributing factors to a contract that still runs 5 years on.

On the flip side, another agency pitching an email support platform worth $100,000 in fees a year to them insisted that they would prefer all the communication via email and had a very complicated tracking system that runs through to first line support, then second line and then finally to a specialist if the first two lines couldn’t solve a situation. This scared the client – sometimes you just can’t wait for three layers of conversations before actioning an urgent change - and as a result they weren’t short listed.

Respectable and Responsive

Image Source: http://www.flickr.com/photos/mafleen/402792862/

It sounds obvious that you have to be both respectable and responsive to potential (and current!) clients. However what you as an agency see as being “respectable” may not be necessarily what they feel the definition of the word may be.

Respectable also implies respecting your clients intelligence. One of the key primary things I teach to agencies is that they should research their potential clients carefully. By making your pitch too simplistic may offend their intelligence and could cost you.

Take for example a UK SEO agency that was pitching to a business I was consulting a few years ago. The pitch was about SEO and how they would help the business grow its SEO. Before hand, they had a list of all the attendees, which included my name and the name of the head of Ecommerce (who would at least have a rudimentary knowledge of SEO).

Now if you are pitching to me, you SHOULD know that I know a bit about SEO, if only you bothered to Google my name :)

Yet, in the pitch, the starting slide was an animated slide, which was a web with spiders running up and down it – explaining to us what a search engine bot was and how it crawls the web(!) apart from the fact that the animation was poor (a gif of a spider running up and down the web), they actually assumed that a multi million pound business that they were pitching to:

• Need to see what a web spider is in a picth presentation
• Be spoken to as if they were total amateurs

In addition, as I was in the audience I found this actually quite insulting – the fact that they hired me to be in the room meant that they were serious about a decent SEO strategy. The Head of Ecommerce had the same horrified response as I had – did the agency think we were complete idiots?

Needless to say, they lost the pitch in the first round.

Intelligence

Image Source: http://www.flickr.com/photos/mukumbura/4043364183/

To be perfectly frank, expect a serious pitch to be faced with some serious questions. At the same time, you would be expected to show real intelligence in the way you present and prepare for the meeting.

Displaying intelligence isn’t showing how many clients you have, or sprouting your company’s internal strap lines. It isn’t displaying how many results you have gotten for other clients.

Intelligence is more about:

• How well you have both, understood and answered the clients brief
• How well you have actually understood the clients business
• Demonstrated a working knowledge of the clients business and THEN demonstrated how your activity would help
• Demonstrated both creative and critical thinking, and looked into trying to future proof campaigns.
• Indicate that the right people would be working on the right portions of the campaign. Make some of those people part of the brief

The worst case scenario would be that you have a really intelligent hands on SEO prepare your presentation, and then instead either get an account manager or sales person actually present it, without the SEO present to field any questions. Often the result is a disaster – yet this a very common approach. Believe it or not, this has happened to me at least three times. Neither the account manager nor the sales person actually knew any SEO (PPC in one PPC agency pitch). Which meant though their presentation was solid, they ability to field questions intelligently was fairly limited to “We can come back to you on that”.

Technology and Innovation

Image Source: http://www.flickr.com/photos/stuckincustoms/3232133635/

In the online world, when data flows (fairly) freely, technology has to be at the forefront to deal with that data, to rationalise, monetise and sanitise it. An agency coming in to pitch within the digital sphere needs to show (to me at least):

• Usage of relevant tools and technology existent in the market
• Development plans for new tools / or custom tools
• An understanding of how technology available can be suited to your campaign

Similarly, an agency that doesn’t innovate is low on my “like” list. I would be willing to spend more time with one that has interesting ideas about innovating, than one that actually just rehashes ideas that exist in the market and brand them as their own.

One agency years ago insisted that they have “market leading” guides on SEO for internal staff - from content strategies to link building. When quizzed what kind of information they would share with the businesses content team for better SEO, we received a document that was clearly well set up, researched and written for the right audience. Sounds great right? Only problem was this was the SEOmoz guide that they simply wrapped up and presented to us. Seeing that I was on one of the top contributors to SEOmoz at that point ( I think I still rank in the top 10) I recognised the document and called them up on it.

Needless to say, I don’t believe they ever repeated that faux pas - and went out and had their own content written.

Similar situations exist when companies tell me of a revolutionary tool A or amazing platform Y - and in most cases they tend to be industry standards that they use and nothing out of the ordinary. Which is fine for a basic pitch – but for a stellar pitch you need to stand out.

Yield Based Approach

Image Source: http://www.flickr.com/photos/cambodia4kidsorg/3290848235/

Any campaign you build has to deliver a return. It doesn’t matter what the campaign is, it has to achieve its objectives. Which means if you have to pick an agency, the agency has to demonstrate the capability to not only come up with a plan or strategy that works for you it has to demonstrate that it understands what your businesses KPIs are.

This doesn’t simply mean an uplift in sales, traffic, but a clever demonstration of how the Return On Investment would be aimed at and achieved. If an agency cannot demonstrate a clear understanding of your businesses goals, and does not take the time to understand what the ROI of the specific channel being discussed should be, then they fail in using a yield based approach.

A few years back, an SEO agency pitched to me for a UK Holiday business. They were big on numbers by their own admission, and had a clear demonstration of how much it would cost us to rank for Keywords, and in fact had a clear chart identifying the top level “Category Killer” keywords.

They then went on to demonstrate how one of their current Holiday clients achieved those rankings with their help by spending the same figures that they demonstrated. The top level Keyword was “Holidays”.

2 problems with that.

First, if they already have a client in the space that they are working with to rank for those exact keywords, then I wonder to myself if the end result would become who spends the most to retain those positions. Which in itself is fine, I have no problem with agencies who have clients in the same niches, BUT, at what point does the competition with one client against the other show a negative return? If spend is the limiting factor, I wouldn’t want a competitor in that space to have the same resources as I do in terms of SEO talent, and then be simply beaten by their capability to throw more money at the campaign. Which wouldn’t be a worry, except if the agency was so willing to tell us exactly how much it cost their other client to rank, how can I trust them not to reveal the same data to our competitor?

The second problem with this scenario was they went straight for the proverbial jugular. They want to work on the money keywords (money for them!). A UK Holiday site may gain some sales on the back of ranking for “Holidays”, but I promise you that the conversion rate would be dreadful, and probably in the third decimal percentages.

If I had to pitch that gig, I would have started at the lower rung, moving upwards towards the chain to the category keyword “UK Holidays”. The spend to rank for most those would have equated to the total that the agency wanted as its fees, but the ROI for ranking for the RIGHT keywords would have been much, much higher. And an easier sell.

So the agency failed t understand the business, and as such failed to demonstrate that they could deliver the right ROI for them.

Conclusion…

If you have stuck with me so far, congrats (and thank you!). I am genuinely hoping that agencies that pitch, take something away from this post, and people who are paid to listen to pitches, do as well. I know that these principles have been successful for a large number of agencies when pitching, despite the fact that the agency didn’t realise that they were following a successful model.

The aim isn’t to follow my thoughts flat out, but learn form a person who has been involved I both sides of a pitch process, with a decent success rate in both, picking the right agency, and being picked for a campaign.

Rishi Lakhani is a freelance Online Marketing Consultant working with a number of brands and agencies in the UK, and spends a large portion of his free time on twitter. Follow him at: https://twitter.com/rishil

Google announced they were going to extend their vertical paid inclusion program to product search queries, where the paid inclusion results are put inline with the organic search results, often driving most (if not all) of the organic search results below the fold.

The layout of the result looks something like this

Or if you put it in Google's browser analysis tool, it looks something like this

And with that move, if you are in ecommerce & you don't rank #1 you are essentially invisible to most searchers.

As John Andrews highlighted on Twitter: "Notice Google tells us "paid relationships improve quality" and then penalizes for paid links?"

As always, it is more profitable to follow Google's biz dev team than Google's public relations pablum.

In some cases Google might include 3 or 4 different types of monetization in a search result. In the below search result Google includes:

• AdWords ads
• Google Offers
• Hotel Comparison ads
• Hotel Price ads

And those are *in addition* to featuring promotional links to Google Maps & Google+ in the search results. Further, some of these vertical results consist exclusively of paid inclusion & then have yet another layer of PPC ads over the top.

As SEOs we focus a lot of energy on "how do I rank 1 spot higher" but when the organic results are displaced and appear below the fold why bother? The issue of the incredibly shrinking organic result set is something that can't be over-emphasized. For many SEOs the trend will absolutely be career ending.

AdWords, product listing ads, brand navigation, product search, local, etc. A result like this has a single organic listing above the fold & if Google decides to rank their local one spot higher then that turns to zero.

If you look at the new TLD announcement Google applied for .MBA & .PhD (as well as many names around entertainment, family & software). Thus it is safe to say that education will eventually be added to local, video, media, shopping & travel as verticals where Google is displacing the organic results with links to more of their fraternal listings. About the only big categories this will leave unscathed will be real estate, employment & healthcare. However those first 2 are still in contraction during our ongoing depression & Google blew a lot of their health credibility by pushing those illegal ads for steroids from a person posing as a Mexican drug lord.

In addition to these fixed vertical that cover the most profitable areas of search, Google is also building a "vertical search on the fly" styled service with their knowledge graph. Their knowledge graph extracts data from 3rd party websites & then can be used to funnel traffic and revenue to Google's various vertical services. To make it seem legit, Google will often start by sending some of the traffic onto 3rd party sites, but the end destination is no different than product search. While it is a "beta" product it is free to justify an inferior product being showcased front & center, but after Google gets enough buy in they monetize.

There is a non-subtle difference between Google's approach and Microsoft's approach to building a search ecosystem.

Sucking the Brains Out of the Internet

After Google was unable to acquire Yelp they offered Yelp (& sites like TripAdvisor) an ultimatum: "either let Google steal your content & displace you with a competing service consisting largely of the stolen content, or block GoogleBot if you don't like it." While Google sucks in the value created by such 3rd party sites, they also explicitly exclude them from various vertical services aligned with the most valuable keywords. Yet at the same time, all this is to be seen as legitimate because there is a computer used somewhere. It as though humans are not making these profitable business decisions at all & so Google hires lawyers to write coin operated legal opinions about how computer generated results are free speech.

Nextag's CEO wrote a scathing article about Google in the WSJ, which promoted a response from Amit Singhal.

If you've wondered why Google keeps appearing before regulators, keeps being called evil, was just sued by the Texas AG, & has their own hate organization the above exemplifies why.

Let's compare that behavior against Yahoo! or Bing.

Yahoo! has long been considered out of the search game, yet when they want to have a competitive advantage they do things like license photos from Getty. They use the content with permission on agreed terms.

Google's approach is more along the lines of "scrape it now & figure out legal later." And after a long enough period has passed they will add monetization & mix it into the core of their offering, like they recently did with books:

This launch transitions the billions of pages of scanned books to a unified serving and scoring infrastructure with web search. This is an efficiency, comprehensiveness and quality change that provides significant savings in CPU usage while improving the quality of search results.

Both Bing & Google are creating knowledge graphs. Bing does things like partner with Britannica, Yelp & Qwiki.

Eric Enge interviewed Stefan Weitz about the new Bing interface. As part of that interview, Stefan described Bing's editorial philosophy on building a search ecosystem

We partner with 3rd party services instead of trying to build or acquire them. There are probably something like a million apps out there today.

I talk to probably two dozen start-ups every week that are doing different cool things on the web. To think that we are ever going to be able to actually beat them, or out-execute them (when they are talking about 12 guys with half a million angel funding building some really interesting apps), it is just not likely.

Ars Technica also has a piece discussing the creation of entity graphs (which is where the "sucking the brains" line came from). A key difference between Bing & Google is that Bing feels they should partner with sources & link out, whereas Google links the results back into more Google searches. What's more, when Google features their own vertical results in many cases links to the data sources are not provided at all & you stay on a fully Google experience, in spite of the cost to 3rd parties in building & maintaining databases that are scraped to power Google's offerings.

Off the start forays into new categories might provide some value to publishers in order to get buy in, but eventually the "first hit free" stuff shifts to paid & Google continues to displace publishers across more and more of the ecosystem, using content scraped from said publishers.

Funding Scraping

When Google or Apple drive cars around the country or fly military-grade planes over cities to create 3D maps of cities they are creating databases & adding new information. Outside of collecting private data (like wifi payload data) there is little to complain about with that. They are adding value to the system.

However, at the same time, Google not only scrapes themselves, but they are a revenue engine that drives a lot of third party scraping. And they design penalties in a way that allows those who scrape penalized sites to outrank them. With batch penalty updates some folks can chain redirects, expired domains & so on to keep exploiting the combination of copyright violations & Google penalties to make a mint. Google also had a long history of funding Traffic Equalizer sites, sites like Mahalo that would take a copy of a search result & auto-generate a page on it, newspaper sites that would hang auto-generated stub preview articles on subdomains, & sites like eHow which integrate humans into the process.

While many sites are still penalized from the first version of Panda, downstream referrals to eHow.com from Google in the US were up over 9% last month. They know "how to create SEO content."

Recently a start up that launched a couple years ago decided to take their thousands of subdomains of scraped databases & partner with authoritative websites to syndicate that content around the web. Some of those get double listings & for some search queries there is the same page (with a different masthead logo) 5 different times. Those sites don't get hit by duplicate content filters or algorithms like Panda because they have enough domain authority that they get a free pass. Including AdSense in the set up probably makes it more palatable to Google as well.

If you have scale you can even auto-generate a bunch of "editorial" questions off the database.

More data = more pages = more questions and comparisons = more pages = SEO alchemy (especially if you don't have to worry about Panda).

The parent scraper site includes links back to itself on every syndicated page, which to some degree makes it a glorified PageRank funnel. WPMU.org got smoked for syndicating out a sponsored theme on one of his own sites, but the above industrial-scale set up is somehow reasonable because it was launched by a person who sold their first start up to Google (and will likely sell this start up to Google too). The site also includes undisclosed affiliate links & hands out "awards" badges to the best casual encounter sex dating sites, which then get syndicated around the web & get it many inbound links from "high quality" porn sites.

I won't name the site here for obvious reasons, but they are not doing the above in a cloak of darkness that one has to look hard to find & do deep research to patch together. For some search results they are half or more of the search result set & they even put out press releases when they add new syndication partners, linking to numerous new automated subdomains or sections within sites related to various categories.

When the search results look like that, if you do original in-depth reviews that are expensive there is zero incentive structure to leaving your content and ratings open to Google and these sort of scraper/syndicaters.

There is always a new spin on the mash up low end content with high trust websites and try to feed it into Google. So long as Google biases their algorithms toward big brands & looks the other way when they exploit the ecosystem that trend will not end.

The Illusion of Choice

It is hard to see & feel the cost of a dominant market participant unless you have to do business negotiations with them:

The Independent Publishers Group, a principal distributor of about 500 small publishers, recently angered Amazon by refusing to accept the company’s peremptory demand for deeper discounts. Amazon promptly yanked nearly 5,000 digital titles. Small-press publishers were beside themselves. Bryce Milligan of Wings Press, based in Texas, spoke for most when, in a blistering broadside, he lambasted Amazon, complaining that its actions caused his sales to drop by 40 percent.

However, even when companies are brutal in some aspects they do amazing things in other areas, so one has to weigh the good with the bad.

Now more than ever we are drowning in perceived choice, but if you look at market after market they are far more consolidated on the business side.

Into hipster indie music? Those labels are heavily reliant on the bigs. The increased flow of online streaming royalties will further increase the consolidation as big businesses prefer to negotiate with other big businesses & small players lack the resources needed to move the needle.

At any point Google can fold one vertical into another or extend out a new model. The Android Marketplace feeds into Google Play, Google local feeds into Google+, Google search force feeds just about everything else & even free offerings on sites like YouTube will eventually become pay to play stores.

Where Google lacks marketshare & forced bundling isn't enough to compete they can buy the #2 or #3 player in the market & try to propel it to #1 using all those other forms of bundling.

Part of what made search competitive against other platforms was its openness & neutrality. But if the search results are Wal-Mart over and over again (or the same scraped info 5 times in a row, or a collection of internal listings) then the system becomes more closed off & the perception of choice becomes an illusion. John Andrews wrote a couple great Tweets expressing the shift in search:

• "Google SEO is no longer worth the effort for those who are not writers, artists, speakers, trainers, or promoters. What happened to Search?"
• "If you want to see what Google will look like after it locks up, look at Apple. ipad users are already "managed" very tightly."

When companies try to expand the depth of their platform with more features it is a double edged sword. At some point they capture more value than they create and are no longer worth the effort. When they get to that stage it becomes a race to the bottom with scrapers trying to outscrape one another. Then in turn the company that created the ecosystem problem uses the pollution they rewarded to further justify closing off the system, guaranteeing only more of the same. Those who actually add value move on looking for greener pastures.

Protecting Privacy

Google promotes that they make browsing safe & Firefox will soon stop passing referrer data. Apple was granted an anti-Big Brother patent. StopBadware partnered with Google, Facebook & others to create a self policing industry organization named the Ads Integrity Alliance.

When these companies are not busy "protecting" users they acquire recognition technology, collect a treasure trove of personal data, deliver fake endorsements, provide false testimonials & sell off the data to third parties.

Microsoft filed a patent for serving mood-based ads & there is research on how depressed people use the internet.

These companies compete on both the hardware & software level, collecting more data & creating more ad formats.

A label or an interest is a vector for ad targeting. There is no need to worry about de-anonymizing data for ad targeting when it is all in-network and you monitor what someone does, control which messages they see, & track which ones they respond to. Tell someone something often enough and they may believe it is true.

The Contempt Large Companies Have for their Customers

There is a sameness to customer service from a lot of big companies. They spend loads & loads to track you and market to you, but then disappear the moment things go wrong, as they are forbidden to care.

Perhaps the only thing worse that AOL's customer support is the unmoderated comments on the YouTube page.

Google will rate YOUR customer service, but when it comes to customer service FROM them you are on your own:

Denise Griffin, the person in charge of Google’s small customer-support team, asked Page for a larger staff. Instead, he told her that the whole idea of customer support was ridiculous. Rather than assuming the unscalable task of answering users one by one, Page said, Google should enable users to answer one another’s questions.

Even their official blog posts announcing that they are accepting customer feedback for your applications go unmoderated.

This sort of contempt exists at essentially all large companies.

Everything seems on the up & up, but that "private listing" was maybe for a counterfeit product.

If it isn't a counterfeit & you get too good of a price you are threatened with a lawsuit, and the branded network falls behind a "oh we are just a marketplace and can't be bothered to give a crap about our customers" public relations angle.

If a company has size there is a limit to how much they can invest in any individual transaction. And so ebooks made of YouTube comments invade Amazon.com.

Apple creates "beautiful" products designed around forced obsolescence:

The Retina MacBook is the least repairable laptop we’ve ever taken apart: unlike the previous model, the display is fused to the glass—meaning replacing the LCD requires buying an expensive display assembly. The RAM is now soldered to the logic board—making future memory upgrades impossible. And the battery is glued to the case—requiring customers to mail their laptop to Apple every so often for a $200 replacement. The design may well be comprised of “highly recyclable aluminum and glass”—but my friends in the electronics recycling industry tell me they have no way of recycling aluminum that has glass glued to it like Apple did with both this machine and the recent iPad. The design pattern has serious consequences not only for consumers and the environment, but also for the tech industry as a whole.
...
Every time we buy a locked down product containing a non-replaceable battery with a finite cycle count, we’re voicing our opinion on how long our things should last. But is it an informed decision? When you buy something, how often do you really step back and ask how long it should last? If we want long-lasting products that retain their value, we have to support products that do so.

One last bit of absurdity on the YouTube front. Google recently threatened to sue a site designed to convert YouTube videos into MP3s.

• How does Google's "computers deserve free speech rights" & shagging 3rd party content to fill out their own vertical search services compare against their approach when someone uses YouTube content in a way Google does not desire?
• There are AdWords ads promoting free unlimited MP3 downloading & song burning bundled with shady adware.
• Google's AdSense for domains funds boatloads of cybersquatting. While Google threatened to sue this particular site, they could have just took the domain due to it cybersquatting on the YouTube trademark. The fact that they chose to turn this into a press event rather than simply fix the issue shows that this is more for posturing.
• Further aligned with the above point, while Google singled out a specific MP3 conversion site, there are other sites designed around doing the same exact thing which are PREMIUM ADSENSE PARTNERS, with the body of the page looking like this:
  

How Small Companies Are Taxed With Uncertainty

When Google decided to move away from direct marketing to brand advertising things that are often associated with size, scale & brand recognition became relevancy signals.

For big brands there is no shortage of companies trying to service the market that Google is favoring. For smaller companies it's a struggle. There are so many things to know:

• how to create & pitch feature content
• what do unnatural link warnings mean & how do I interpret reinclusion request replies?
• how much to invest in marketing, where to invest it, how to balance the need for short term cashflow with the required reinvestments to build real (or fake) brand signals
• how long does the market have left before Google enters the niche and destroys the opportunity that organic SEO once represented
• should you run 1 website, or many to hedge risks? and how many is optimal?
• how big should your site be?
• if one of your sites gets penalized, should you try to fix it up, should you start over with a new site, or should you consider SEO to be a pointless goal?

Google mentions that they want people to do what is best for the user & not worry about Google, but that advice is a recipe for pain

If you do not run a large & authoritative website there are so many landmines to trip over with the increasing complexity of SEO. And any of Google's "helpful" webmaster messages can suspend a webmaster in fear, leading them to an eventual bankruptcy.

Small companies need to do all sorts of canonicalization hoops & prune content and such to hope to avoid algorithms like Panda. Then Google changed their host crowding preferences to let some large sites get up to 8 listings in a single search result page for their LACK OF effort. Those larger sites can then partner with glorified scraper sites that syndicate databases feeding on domain authority with no risk of Panda.

Due to how Google penalizes smaller sites, those that rewrite their content will outrank them when they get hit. These horrible trends are so obvious that even non-SEOs like Tim Carter (who was a Google golden boy for years) highlights how the tables have tilted away from what is most relevant to what pays Google the most.

The promise of the web (especially search) was that it could directly connect supply and demand. However, just like propaganda promoting the superiority of certain countries in the physical world, it is unfortunately fast becoming a myth.

Bing refreshed their webmaster tools offering & now allows you to look up link data for 3rd party sites.

We recently interviewed Bing's Duane Forrester about the new SEO tools & their product roadmap.

Here is a screenshot of their new link explorer, but I highly recommend setting up an account and checking it out firsthand.

For a long time Yahoo! provided great link data, but most other search engines were more reserved with sharing link data for competing sites. What were some of the driving forces behind Bing opening up on this front?

Bing values the power of strong partnerships as one way to spur innovation and deliver compelling experiences for our users. For any partnership to be effective, remaining as transparent as possible is critical, including those we forge with agency and publisher partners. Sharing link information was something very clearly asked for by tool users, so after doing the internal work to see if we could provide the information, it was an easy decision to build this tool when the answer came back positive. You wanted it, we had it and could share it. Done.

As a search engine your web index is much much larger than most SEO tools. On Twitter Rand mentioned that the index size of Bing's new Link Explorer was fairly comparable to Open Site Explorer. Is the link data offered in the tool a select slice of the index? Were you trying to highlight the highest quality link sources for each site?

We see the entire index, or at least "can" see the entire index and link ecosystem. We’re limited to the actual number we can show at any given time, however.

Currently it appears as though the tool lists link source URLs & page titles. Will the tool also add anchor text listings at some point?

On the list – sometimes we run into data sourcing issues, so when we hit those walls, it takes us longer to add features. Bing WMT pulls data from all the sources available within Bing Search, and sometimes those have limits imposed for other reasons. In those cases, we must abide by those rules or seek to influence changes to increase our own access/capacities. A search engine is a complex thing it turns out… J

There are filters for "anchor text" and "additional query." What are the differences between these filters?

Anchor Text is pretty clear to most SEOs. "Additional Query" allows you to look for, as an example, a page with "N" text appearing on it. So text not just as "anchor text", but simply appearing on the page.

Currently if I search for "car" I believe it will match pages that have something like "carson" on it. In the future will there be a way to search for an exact word without extra characters?

I’m going to split this answer. Users can enable “Strict” filtering to only see “cars” data by selecting the “Strict” box. To your point, however, this is what some of our tools are Beta. We will continually refine them as time goes on, adding features folks find useful.

Will you guys also offer TLD-based filters at some point?

First time anyone's mentioned it, so I’ll add this to our list for consideration.

A few years ago my wife was at a PPC seminar where a Bing representative stated that the keyword search data provided in the tools matched your internal data. Is this still the case?

Bing Advertising is completely separate from Webmaster Tools. I’m not sure if that rep was meaning data within the adCenter tools matches data or what. Bing WMT does import CPC data to showcase alongside keywords which sent traffic to your site. That data matches as we pull direct from adCenter. The data we show through our tools comes direct from Bing Search, so that’s a match if this is what you’re referring to.

Bing's Webmaster tools offers an API with keyword research & link data. Bing's Ad Intelligence is easily one of my 3 favorite SEO tools. Will Bing eventually offer a similar SEO-oriented plugin for Excel?

No plans on the roadmap for an Excel plugin.

At SMX Derrick Connell suggested that there was a relevancy perception gap perhaps due to branding. What are some of the features people should try or things they should search for that really highlight where Bing is much stronger than competing services?

Without doubt people should be logging in and using the Facebook integration when searching. This feature is tremendously helpful when you’re researching something, for example, as you can reach out directly to friends for input during your research process. While searching, keep your eyes open for the caret that indicates there is more data about a specific result. Hovering over that activates the “snapshot” showing the richer experience we have for that result. Businesses need to make sure they focus on social and managing it properly. It’s not going away and those who lag will find themselves facing stiff, new competition from those getting social right. Businesses also need to get moving adopting rich snippets on their sites. This data helps us provide the deeper experiences the new consumer interface is capable of in some cases.

You have wrote a couple books & done a significant amount of offline marketing. One big trend that has been highlighted for years and years is everything moving online, but as search advances do you see offline marketing as becoming an important point of differentiation for many online plays?

In a way yes. In fact, with the simplification of SEO via tools like our own and many others, more and more businesses can get things done to a level on their own. SEO will eventually become a common marketing tactic, and when that hits, we’re right back to a more traditional view of marketing: where all tactics are brought to bear to sell a product or service. Think of this…email marketing is still one of the single best converting forms of marketing in existence. Yet so many businesses focus on SEO (drive new traffic!) instead of email (work with current, proven shoppers!).

In fact, neither alone is the "best" strategy for most online businesses. It’s a blend of everything. Social happens either with you or without you. You can influence it, and by participating, the signals the engines see change. We can see those changes and it helps us understand if a searcher might or might not have a good experience with you. That can influence (when combined with a ton of other factors, obviously) how we rank you. Everything is connected today. Complex? Sure, but back in the day marketers faced similar complexity with their own programs. Just a new "complex" for us today. More in the mix to manage.

What is the best part about being an SEO who also works for a search engine?

On Wednesday, June 6th at 10AM PST, I was part of the team that brought a new level of tools forward, resetting expectations around what Webmaster Tools should deliver to users. Easily one of the proudest moments of my life was that release. While I’m an SEO and I work for the engine, the PM and Lead Engineer on the WMT product are also SEOs. ;) To say Bing is investing in building the partnership with SEOs is no mere boast. Great tools like this happen because the people building them live the life of the user.

What is the hardest part about being an SEO who also works for a search engine?

Still so few people around me that speak this language. The main difficulty is in trying to understand the sheer scope of search. Because everything you thought you knew as an SEO take son an entirely different dimension when you’re inside the engine. Imagine taking every SEO conversation and viewing it through a prism. So many more things to consider.

And, finally, nothing against Matt here, but why are dogs so much better than cats?

1 – they listen to you and execute commands like a soldier
2 – generally, they don’t crap in your house
3 – you can have a genuine conversation with a dog
4 – one of my dogs drives
5 – when was the last time your cat fetched anything for you?
6 – your dog might look at you funny, but won’t hiss at you
7 – guard cat? Hardly… you’d be better off with peacocks in the yard.
8 – dogs make great alarm clocks
9 – even YOU know you look strange walking your cat on a leash…
10 – dogs inspire you to be a better person

-----

Thanks for the interview Duane & the great new tools. :)

Duane also did a video review of their new tools on SEOmoz, which highlights how they show rank & traffic data on a per keyword & per page basis. To learn more about Bing, subscribe to their search blog & their webmaster central blog. Duane also shares SEO information on Twitter @DuaneForrester & via his personal blog.