As an SEO one of our primary goals is to get more search traffic for targeted search terms. Search traffic is typically far more valuable than other traffic sources because it is so targeted. But non-search traffic is perhaps the single most reliable sign of quality. As Google controls a larger portion of the overall traffic flow across the web, they risk creating self fulfilling prophecies where low quality sites continue to rank only because they already rank.
If you were Google, and discovered that 98% of a site's traffic comes from Google.com might you want to give that site a bit less exposure? I would. Maybe those algorithms do not exist now, but eventually they could.
If you have a site that earns far beyond your living costs, and it is almost entirely reliant on search for income, then one of the best moves you can make for the sustainability of that site is to lower the percentage of traffic that comes from search by creating other traffic sources. The other traffic sources may not be as profitable on a CPM basis, but as you diversify you lower your risks. It doesn't matter how the algorithms shift if your site is strong in every signal of trust they could possibly measure.
Question: How do I track the progress of competing sites over time? How do I know what keywords my competitors are ranking for, and which ones they are improving on?
Answer: There are a way variety of means to track competing sites.
Online Ranking Checkers
Online tools such as Digital Point's keyword ranking tool, the ShoeMoney SERP tool, the our rank checker, or the new tool at ZoomRank show where keywords rank. The Digital Point tool also shows you where a keyword ranks over time, but the problems are who wants to check a lot of these keywords one at a time, and where do you get the opening list of keywords?
Competitive Research Tools
There are numerous organic search competitive research tools on the market. SpyFu is a paid tool which offers limited free data, and SEODigger and URLTrends are free tools which shows you keywords that a site ranks for. AdGooroo also takes a look at similar data, with more focus on paid search. If you can afford to spend $10,000+ you may also want to consider trying HitWise.
Keyword Research Tools
After the competitive keyword research tools you may also want to look at the keyword tools promoted by the search engines (Google, Yahoo, MSN) and companies like Wordtracker and Keyword Discovery. The Google Traffic Estimator will tell you what terms Google thinks are most valuable, and you can also check keywords in bulk and track them over time by buying software like Agent Web Ranking.
Looking for a few more keywords that the competitor may have found but the market missed? Beyond those competitive research tools the easiest spots to look for keywords on competing sites are
the internal navigation of competing sites (especially look for pages that are bizarrely over-represented in their navigation)
a competitor's homepage page title and headings (you can see the page titles with Xenu and the last two data points with OptiSpider)
abnormal patterns in their inbound anchor text (use a tool like Backlink Analyzer)
As far as general traffic trends for a site go, Compete.com, Quantcast, and Alexa all give snapshots of site traffic trends. That data tends to be rough though...especially for small sites. The two big ways to track site growth from a market penatration and search representation standpoint are to look for changes in the rate of citation by using the following data points and tools
Yahoo Site Explorer for link count (pay attention to the quality of the top links, also use SEO for Firefox to dig in for more general linkage data)
Technorati and Google Blogsearch links for links from blogs (pay attention to rate of growth, and the what percent of the links are of decent authority)
and look for the rate of the growth of the site's content, reach, and trust using the following metrics
number of RSS subscribers (use Bloglines subscribers as a sample estimate if they don't have any other numbers public) and/or number of comments on blog posts for sites with feeds
how often you see well read channels mentioning the competitor
how often you see (search, contextual, or affiliate) ads for the competitor
Frank mentioned this NYP article about how some companies are buying sites outright rather than increasing their AdWords bid prices. I expect this to be a large and growing trend for at least a couple years. As Google gets more efficient at pricing the ads they increase the value of the top ranked sites that sit alongside those ads. Internet Search Metrics, quoted in the NYP article as Internet Search Management, is providing audits on the competitive landscape of search
ISM's audits track the top 4.5 million search phrases on Google and Yahoo!, a total of 7.3 billion searches a month, to determine which companies across 50 business sectors pop up most frequently in the top three or four positions in natural search. ...
The ISM audits, to be released in London, break down which of 50 business sectors are locked up - that is, have large chunks of natural search dominated by a handful of companies - and which are wide open.
I have not yet seen any of the reports, but the network is still young. If you love marketing, are in tune with web trends, and are well funded I am guessing that many of the markets that appear locked up are still wide open.
As long as Google allows webmasters to report spam and report paid links, few will question how much webspam Google sponsors. As long as they are the lead corporate sponsor of Stop Badware few will think of their Toolbar as spam.
Some of the A list bloggers who trashed paid reviews, are talking up the virtues of conversational marketing, forgetting what they just wrote. The hard part of being a well known blogger is that as one gains exposure, influence, experience, friends, enemies, and a large archives it is easy to appear hypocritical. This is especially true on a rapidly changing network, where successful people change with the network, and advertising techniques that were once unethical are mainstream a year or two later.
It is easy to appreciate how under-priced some SEO services are when you look at what people are willing to pay for other traffic sources. $11 million in domains changed hands at the recent TRAFFIC conference, and $52 million more would have if the owners will willing to let them go! Once domains get properly developed their value quickly increases. Business.com was claimed to be overpriced in 1999 for 7.5 million, but now is up at auction for $300 million plus. Business.com is not much more than a thin arbitrage play, with a great name, great branding, and great marketing, all funded by the power of search.
To appreciate how much marketing and sales they do, consider that they only employ 6 editors to run the entire directory, while having around 100 employees. Without SEO/SEM that domain would not have the leverage needed to get syndication distribution partnerships, have strong revenues, custom ad deals with Google, or command such a high multiple.
I have spent $10,000+ buying a non-type-in domain name that someone paid $8 for. I have done it more than once in the last month. Why? Synergistic value in branding, associated perceived trust, and market differentiation...which are all important in a crowded marketplace.
Are domains overpriced? If you pay retail and don't have a plan for them, maybe. If you have a good plan and develop them (or buy below retail in a growing market) you should do well. As ad dollars continue to move online almost all markets are growing markets.
Sure a lot of people think you are at the mercy of Google as an SEO, but in certain markets people EXPECT to find certain sites. Google has to rely on showing some partners in their organic results to sell paid search ads against. If you look at Google's personalized homepage tabs they also offer a I'm Feeling Lucky button that adds the most popular items to that tab. If you create a tab for your keywords does Google push your feed? If you get the right name and market the hell out of it, I think your rankings, traffic, and business model are at least as defensible as type in traffic or just about any other business model.
As SEOs we tend to undervalue some of our assets because the work we do for clients is so valuable relative to the rates we charge. After a few hours of work you can offer most clients strategies that increase their non brand traffic by at least 10 to 20%. But when you hear that Business.com might get $350 million for something making $15 million a year, it makes it easy to want to heavily reinvest into creating a real brand of your own, instead of working on a group of smaller projects.
If you get a scalable business site up to $30,000 a month in free cash flow, then reinvest all of that into marketing and branding for a couple years, it is possible to build something that generates at least a few million a year and sells for at least 8 figures. At that point you can retire if you want to, but I will be 30 in a few years, so I need to start pushing harder. :)
If you see a hot story spreading don't be afraid to jump on it, especially if you have specific details related to the story, or your view is counter to the popular view. Journalists want sources, numbers, and to appear unbiased.
Blogging, email, IM, and telephone make it easy to keep in touch with your real friends. Social networks are hyped, but tend to have low value traffic because they don't effectively separate signal from noise. The only people who have time for them are people hawking crap, people looking to waste time, and spam bots.
Sure Google owns Orkut, but they don't need it. Google already is a social network, and became one by targeting and partnering with the power users and influencers, but few people think of them as a social network. If you are a webmaster there is a good chance that Google delivers your ads, controls your feed, is your analytics provider, and is a leading source of traffic to your site. News sites and other trusted editorial partners are rewarded with high Google rankings, and become addicted to Google's traffic the way LookSmart was addicted to Microsoft's traffic a few years ago.
On top of having touch-points with most webmasters, Google also has touch-points with mass information consumers. Google's Toolbar gives them web history, which knows where you have been and how long you were there. Google Reader is the most popular feed reader service on the web. They know what voices you trust and subscribe to, and allow you to share your subscriptions or favorite items with friends.
Google's personalized homepage is also the most functional, customizable, and useful service by any of the major portals. Google gadgets are easy to create and syndicate, and Google allows you to publish gadgets or tabs with anyone.
YouTube makes it easy to upload and syndicate videos, and now those are highly ranked in Google's search results. Look at YouTube's TestTube to see the community features they are planning.
Google offers an auto-complete feature, which makes it easy to refine your search based on the searches of others. They also show related sites and hot searches, and let you note sites you like via Google Notebook. If you don't find their results entirely relevant they make it easy to create a custom search engine using their data.
Keep track of time and your projects using Google Calendar and Google Docs.
Sure Google has Orkut, but they don't need it. By default Google keeps hold of your attention so they can use it to improve their relevancy algorithms and ad targeting. Tie in search personalization with automated ad optimization and their information sharing tools and it is easy to think of Google as the web's largest social network, though most people don't think of them that way, largely because they put information at the center of it, and are so good at separating signal from noise.
Along with branding, public relations is one of the few things that save you and I from commoditization. Every business (and business model) has flaws, hidden costs, value propositions, and has stories to express the delivery of value. PR aims to minimize the downsides of the flaws while making the upside look much larger than it is. Alternatively, public relations can also be used to diminish the upside of competitors while making their flaws look much larger than they are.
The Idea of a Fair Market
Is buying links fair? Is buying and holding domains fair? Is linking to a friend's site or your own new site fair? Is buying out competing sites fair? Is syndicating your spin through your own media outlets fair? Business does not care about the concept fair. It only cares about results.
The word fair is typically used to manipulate people. Markets are not fair. Humas have a bias toward that which they have an affinity to, and business is self-serving and inherently dirty.
PR aims to exploit the media and our inherent biases to create an affinity for a brand or product while viewing other brands or products lowly. Low kicks are allowed, though not recommended unless you thought through the potential consequences ahead of time.
Like governments, most successful businesses use PR.
Search engines are so powerful largely because they are so good at public relations. Every week you see Google partnering with someone, fighting for a cause, or talking down competing companies like Microsoft.
The Wikipedia's PR article and Paul Graham's essay The Submarine offer good introduction articles into how prevalent public relations is in the mainstream media.
People and search engines have to trust something. Good public relations campaigns target the trusted parts of the web, by targeting either general authorities or related topical experts.
PR is hard to duplicate. Each story has a main storyteller. Another people retelling your story wont make the right people want to talk about them. Other SEO techniques, such as link buying, are much easier to duplicate and much easier to penalize.
Some of the best PR stories get to be told over and again by the main storyteller, surfacing that person as a topical expert whenever their field comes in focus of the media. Awareness builds relationships, which allows you to spread other stories.
Some things are popular only because they are popular. Good public relations stories can go viral and produce Justin TImberlake-like results. Using large seed sets makes it easier to ensure success, even if the story is not as viral as you would like.
Media exposure gives a sense of credibility. My landlord called me to tell me he read about me in the Wall Street Journal. It is much easier for him to view me as a topical expert after reading that article.
We Love You
Good public relations campaigns spread so well because they make the target want to share the story, by making them feel important, sharing their bias, and/or giving them some incentive to spread the story.
Salary.com created a story about how much work at home mothers should be paid, high-balling the numbers. Every year they re-release the same story and the media eats it up as though it is new.
All of the blog value calculators high-ball the value of the blogs to get people to want to talk about how great their blog is.
Even if you fail to spread to spread these types of stories right away, you can still passively target the right audience using AdWords for less than 10 cents a click.
We Hate You
Many smaller companies make a name for themselves by stating how impure competing businesses are. Creating a common enemy makes it easy for people to identify with you.
The key is not to rant, rave or bash the enemy, but to provide an underlying theme that shows you’re all in it together against the enemy. When framed that way, you’re not a salesperson; rather, you’re a comrade who can lend a hand. Establishing a thematic enemy allows you to focus on providing solutions without coming across like you’re hard selling, and is a perfect technique for white papers, tutorials and blogging in general.
In some cases small market players can garner support when businesses attacks them. Once lawsuits are filed you never know how much support the competitor will get. When I was sued by Traffic Power my fight was for freedom of speech online and saving blog comments. It was an easy story to want to share, so people did. Within days of my blog post about it, the story was featured in the Wall Street Journal.
When we submitted a story about the fall in the value of the US Dollar to Netscape the story was titled How Bush Devalued the Dollar. They like political stuff on Netscape, so the story quickly shot to #1 on their homepage, stuck there all day, and sent over 15,000 visitors to our site.
Please Hate Us
Some public relations ideas play both sides of the coin - creating controversies then fixing problems they created. For example, PayPerPost went lowbrow with their marketing, offering unmarked editorial blog posts as a service, then came up with their Disclosure Policy site to correct the problem they created. They got press on the way down and the way up. They probably would have never received VC funding if they were not so lowbrow with their marketing.
Jason Calacanis
Love him or hate him, he is great at public relations. Most Weblogs Inc. content is at best average, yet he got a nice payout for it, and he used the PR machine again to launch Mahalo.
Weblogs Inc. worked because it got so much link equity from the media, which wanted to tell a story on blogging.
Jason maintained that all you needed to be successful was great content, but they had first mover advantage, paid low rates, and scraped by on profitability by selling spammy links.
Jason got a lot of press for Mahalo by claiming the death of SEO. Mahalo is a human compiled scraper & the URLs are seo friendly.
Just like SEO, public relations can be used to push things that are good or things that are bad. Seth recently published my favorite marketing rant post ever. Here is a quote:
I believe that every criminal, no matter how heinous the crime, deserves an attorney. I don't believe that every product and every organization and every politician deserves world-class marketing or PR.
If you get asked to market something, you’re responsible. You’re responsible for the impacts, the costs, the side effects and the damage. You killed that kid. You poisoned that river. You led to that fight. If you can’t put your name on it, I hope you’ll walk away. If only 10% of us did that, imagine the changes. Imagine how proud you’d be of your work.
PR Watch highlights some of the misuse of and abuses by the public relations industry. They also publish videos to YouTube. I marketed some really dirty stuff when I was new to the web. As I learn more about the power of marketing, I am less willing to market things that only sound good when ignoring the hidden costs.
Rupert Murdock is trying to trade Yahoo MySpace for a 25% stake in the combined company. If Yahoo goes through with that, Rupert's $580 million MySpace investment will be worth about $10 billion. But should Yahoo do it?
Everyone Wants Dow Jones
Rupert is also trying to buy Dow Jones for $5 billion.
The reporting at the WSJ may be better than other places, but even more importantly are the relationships that are in place and the perception that it is better. As many of the newspapers see their margins erode the top few will have more leverage over the market, because the smaller players will be forced to rely more on community created news (mostly noise or something Google could easily replicate) or syndicating news from companies like Dow Jones.
As content quality and relevancy algorithms improve and Google (or similar outfits) control more of the traffic supply the noise content will become less and less accessible (because traffic sources will rank the higher quality stuff to sell ads against and clone the low level stuff to keep hold of that traffic stream). The MySpace experience is not hard to clone.
Yahoo Does Not Need MySpace
Yahoo could grab MySpace and get a bunch of low value inventory in a spam filled network on the decline, or they could get the #1 financial newspaperfor less. Yahoo already has a lot of traffic. They don't need another layer of noise. If they could innovate in the social space they ought to be able to do it with their current assets and traffic stream. As the web gets better at filtering signal vs noise, quality will beat out quantity nearly every time.
The Solidification of Markets
As offline players wake up to the online world the following is happening
search will get more relevant and become harder to manipulate (unless you already have significant offline influence or other assets you can leverage)
As markets evolve the threshold between signal and noise changes. Is your site the type of site that would be easy for Google to clone? Is your content the type of contet Google created the supplemental index for?
Look how bad some of the top ranked content is that still ranks because it is old and was considered high quality content years ago. Imagine how much harder it will be to crack into markets a couple years from now, when people are working so much harder to make higher quality citation worthy content today in so many formats.
The later you start the harder it gets. An hour of focused energy building citation worthy and brand building content today is worth 2 hours next year and 4 hours a year after that.
The effort spent building two parallel sites targeting the same keywords would be better spent creating one stronger brand. Markets are self reinforcing and exposure leads to more exposure.
To stay competitive independent webmasters will increasingly need to chose fewer high quality projects over a large quantity of cheaper lower quality information. Top trusted editorial channels have far more value than bottom feeding networks.
I tend to hate link aggregator posts, but I have read a lot of good stuff recently, and do not want to write 30 posts today or regurgitate other's info verbatim, so here is a link list of useful stuff I recently came across.
