Compete.com Search Analytics is Amazing

I recently got a beta account to the upcoming Compete.com Search Analytics tool. I am not sure of their pricing yet, but Jay Meattle, from Compete.com, told me "the price points will be extremely attractive to small business owners."

You can get leading category based keywords, top competitors for a given keyword (exact or broad match), compare competing sites head to head on keywords, and get the breakdown of traffic sent to any website.

How Accurate is Compete.com?

Their model of data collection is going to make their data more accurate for frequent search terms and larger sites, but I tested their keyword data against some of my smaller sites and it was surprisingly accurate.

How to Clone Smaller Competitors

This is yet another way authority sites will pick off smaller competing sites. It is a simple process. If your site is one of the most authoritative sites in your space you can clean up.

  • Use Google's site targeting ads stats to check what sites are running AdSense and getting a lot of traffic (you don't even need to buy ads on them to do this)

  • Go to Compete.com to get the top keyword phrases competing sites rank for and create content targeting the same keywords. You can also run the keywords through Google's Traffic Estimator to sort the keywords by Google's value estimates.
  • Beyond that, you can run Google's site targeted ads or general contextual AdSense placement reports to find out what pages are the most popular on competing sites, and then create content covering the same topics.

Lowbrow webmasters are fast becoming the outsourced market research department for bigger, more technologically advanced, and cash flush companies.

The Effect of Better Competitive Data

All of these analytic services are going to increase the value of domain authority (since it can be easily leveraged for greater profit) and force webmasters to move themselves up the value chain (since models like AdSense give away too much competitive data, especially when combined with Compete.com).

Other Ways to Use Compete.com

  • Compete can be used to see how strong a brand is in its field. The top keyword in the credit card category is Capital One. Both www.capitalone.com and capitalone.com are also in the top 5 keywords. They are obviously a leader in that space. You can also see what percent of a website's traffic originates from its brand related keywords.

  • You can search for a broad match phrase to see how established your site is inside a vertical, how consolidated a vertical is, and how much potential upside you have by increasing your share of search traffic.
  • Compete can also show you the if a competing site is heavily reliant on a few strong keywords or if their traffic distribution is wider. This can be used to see sites worth investing in (especially if you understand search relevancy algorithms) or sites which have a lot of risk and are worth avoiding.

Cory Doctorow Speech at Google

Cory Doctorow spoke at Google a few months ago. His speech covers IP protection, copyright law, DRM, and international trade laws. It is well worth a listen for any web entrepreneur, especially those considering getting published.

Speaking at the Domain Roundtable

Jay Westerdal recently invited me to speak at the Domain Roundtable, a domaining conference held in Seattle from August 13th through 15th.

If you would like to attend here is a code for $100 off attending: domainseo. That is not an affiliate code, just a coupon code.

Would You Trust a Business Domain Registered via Proxy?

Someone recently left a comment on my blog promoting a new keyword research tool that is registered via proxy. The competitive analysis keyword research tool has been marketed heavily via comment spam, and currently shows itself as bidding on 0 keywords, per its own competitive measures. The site gives no data about who owns it. Could it appear any less legitimate? How do marketers create market research tools espousing the value of something they are not doing themselves, then market it via blog comment spam? It isn't hard to send an email or buy a review. If their service is worth $90 a month (their current price) their marketing budget should include some money for paid search and public relations. They could at least have a blog comparing seasonal data and data from different companies the way Hitwise does.

The easiest way to show the value of your offering is to eat your own dog food. That is why ReviewMe bought a bunch of its own ads to help the site go viral at launch. There are so many ways to market ad networks or competitive research tools that there is no point creating one if the marketing strategy starts with the likes of blog comment spam and/or cold calling.

Comparing the ROI of Online vs Offline Investments

I am sure I have made similar posts before, but I live in an 8 unit townhome, and just down the street they are hauling away part of a mountain to make room for another one of these. Each of the 8 units has a rent of $2100 a month, which comes to a total of $16,800 a month.

Their costs include digging up the mountain, hauling away the dirt, permits and licensing fees, property taxes, cleaning, land, raw building materials, construction, interest on a loan, etc. By the time they are done I am sure they will have spent at least $5,000,000 to create a small revenue stream. A website I started a year ago took about $50,000 to develop, and already makes more than that building will. The virtual real estate investment required no loans, and the ROI is over 100x greater. If I build a strong enough brand my income stream will grow much faster than inflation and be nearly as stable as the real estate, while owning a more liquid asset with lower fixed costs. If you are aggressive enough, you can scale an affiliate site to a billion dollars.

The best investments you can make are in your own education and projects. As time passes the web will be more like the offline world and that 100x greater ROI will start heading closer and closer to 1. The only ways to prevent that from happening are to

  • keep testing and learning

  • heavily reinvesting in growth and brand building
  • be willing to fail and move on to better markets
  • use technology to automate
  • build a following and create social relationships with like-minded people
  • leverage your current assets to optimize and promote future business

I hope they have fun tearing down the mountain. I am off to get a few more links. ;)

How to: Move a Website...Should You Fear 301 Redirects Hurting Your Rankings?

SEO Question: I am considering moving my site to another domain name. Do I have anything to fear in moving it? What is the proper way to move a website to a new URL?

Answer: The best way to permanently move a site is to 301 redirect it. If you have a small site you will likely see few small changes with your rankings. The bigger your site is the longer it takes to move and the more drastically the shakeup will be.

301 Redirecting a Small Site Versus a Large Site

When I redirected my article about Search Engine History the pickup was almost immediate because it was only moving one page to another one page site. My friend Daniel recently 301 redirected the old scholarship site to College Scholarships.org. It was a good test to see how the various search engines would react to moving a 1,000+ page website. The site move started on May 30th. By June 3rd Google indexed 385 pages, and on June 6th Google indexed 509 pages. The old URLs ranked until the associated new ones took their place in the index.

When moving a large site make sure to use a find and replace feature to change internal links to point at the new site location. You can do so inside of HTML editing software like Dreamweaver or using freeware like ReplaceEm. Rebranding a site is also a good time to fix broken links. You can find broken links using Xenu.

Why Our Search Traffic is Still Down

Traffic volumes are still noticeably down from their all time highs, but that is largely a function of 4 factors

  • The site had not been actively marketed for month and we were busy creating other sites so we didn't add much content to the site for months. We were busy building out other high growth potential properties.

  • Right around when we moved the site I believe the traffic volume for that keyword universe dropped (people out of school are not looking for scholarships or grants as much, and people use the Internet less in early summer).
  • Before the site moved we had ads on the blog portion of the site. We removed those ads because we want our blog to be a more organic part of the web than it would otherwise be if it were cluttered with ads. Our page-view traffic stats below are for pages that had ads on them. The blog currently represents a small minority of our traffic, but we hope to change that ASAP.
  • MSN search sucks at 301 redirects!!!!!

Traffic Trends

The site was moved on the 29th of May, and you can see how the traffic was at its all time high in May (the red is the old site and the blue is the new site). In addition to a nice third party trend graph, here are some traffic numbers by date

Date Page impressions
Thursday, March 1, 2007 8,082
Friday, March 2, 2007 5,853
Saturday, March 3, 2007 4,321
Sunday, March 4, 2007 5,424
Monday, March 5, 2007 8,244
Tuesday, March 6, 2007 7,844
Wednesday, March 7, 2007 8,061
Thursday, March 8, 2007 7,594
Friday, March 9, 2007 6,387
Saturday, March 10, 2007 4,579
Sunday, March 11, 2007 4,936
Monday, March 12, 2007 7,877
Tuesday, March 13, 2007 7,898
Wednesday, March 14, 2007 7,739
Thursday, March 15, 2007 7,707
Friday, March 16, 2007 6,344
Saturday, March 17, 2007 5,090
Sunday, March 18, 2007 6,112
Monday, March 19, 2007 9,259
Tuesday, March 20, 2007 9,359
Wednesday, March 21, 2007 8,708
Thursday, March 22, 2007 8,924
Friday, March 23, 2007 7,102
Saturday, March 24, 2007 5,304
Sunday, March 25, 2007 5,169
Monday, March 26, 2007 9,061
Tuesday, March 27, 2007 8,955
Wednesday, March 28, 2007 9,068
Thursday, March 29, 2007 8,565
Friday, March 30, 2007 6,770
Saturday, March 31, 2007 4,760
Sunday, April 1, 2007 5,360
Monday, April 2, 2007 8,483
Tuesday, April 3, 2007 8,671
Wednesday, April 4, 2007 8,552
Thursday, April 5, 2007 7,882
Friday, April 6, 2007 6,390
Saturday, April 7, 2007 5,052
Sunday, April 8, 2007 5,100
Monday, April 9, 2007 9,579
Tuesday, April 10, 2007 9,863
Wednesday, April 11, 2007 9,768
Thursday, April 12, 2007 9,180
Friday, April 13, 2007 7,734
Saturday, April 14, 2007 5,874
Sunday, April 15, 2007 6,533
Monday, April 16, 2007 15,370
Tuesday, April 17, 2007 11,534
Wednesday, April 18, 2007 10,605
Thursday, April 19, 2007 9,895
Friday, April 20, 2007 7,375
Saturday, April 21, 2007 5,201
Sunday, April 22, 2007 5,597
Monday, April 23, 2007 9,955
Tuesday, April 24, 2007 10,064
Wednesday, April 25, 2007 10,461
Thursday, April 26, 2007 9,942
Friday, April 27, 2007 7,689
Saturday, April 28, 2007 5,129
Sunday, April 29, 2007 6,097
Monday, April 30, 2007 10,442
Tuesday, May 1, 2007 11,262
Wednesday, May 2, 2007 10,005
Thursday, May 3, 2007 9,262
Friday, May 4, 2007 7,486
Saturday, May 5, 2007 4,627
Sunday, May 6, 2007 5,744
Monday, May 7, 2007 9,642
Tuesday, May 8, 2007 9,968
Wednesday, May 9, 2007 9,686
Thursday, May 10, 2007 8,865
Friday, May 11, 2007 7,248
Saturday, May 12, 2007 4,858
Sunday, May 13, 2007 5,407
Monday, May 14, 2007 10,878
Tuesday, May 15, 2007 10,478
Wednesday, May 16, 2007 9,974
Thursday, May 17, 2007 9,568
Friday, May 18, 2007 7,724
Saturday, May 19, 2007 5,015
Sunday, May 20, 2007 5,579
Monday, May 21, 2007 10,053
Tuesday, May 22, 2007 10,725
Wednesday, May 23, 2007 10,085
Thursday, May 24, 2007 8,650
Friday, May 25, 2007 7,383
Saturday, May 26, 2007 4,997
Sunday, May 27, 2007 5,328
Monday, May 28, 2007 6,113
Tuesday, May 29, 2007 9,977
Wednesday, May 30, 2007 10,039
Thursday, May 31, 2007 9,650
Friday, June 1, 2007 7,424
Saturday, June 2, 2007 5,111
Sunday, June 3, 2007 6,016
Monday, June 4, 2007 8,876
Tuesday, June 5, 2007 7,378
Wednesday, June 6, 2007 6,599
Thursday, June 7, 2007 5,354
Friday, June 8, 2007 4,019
Saturday, June 9, 2007 2,809
Sunday, June 10, 2007 2,953
Monday, June 11, 2007 4,550
Tuesday, June 12, 2007 4,124
Wednesday, June 13, 2007 3,903
Thursday, June 14, 2007 3,824
Friday, June 15, 2007 3,231
Saturday, June 16, 2007 2,158
Sunday, June 17, 2007 2,459
Monday, June 18, 2007 4,441
Tuesday, June 19, 2007 5,154
Wednesday, June 20, 2007 4,978
Thursday, June 21, 2007 4,468
Friday, June 22, 2007 4,057
Saturday, June 23, 2007 3,086
Sunday, June 24, 2007 3,675
Monday, June 25, 2007 5,853
Tuesday, June 26, 2007 5,666
Wednesday, June 27, 2007 5,684
Thursday, June 28, 2007 5,214
Friday, June 29, 2007 4,032
Saturday, June 30, 2007 2,852
Sunday, July 1, 2007 3,501
Monday, July 2, 2007 5,461
Tuesday, July 3, 2007 4,925
Wednesday, July 4, 2007 3,402
Thursday, July 5, 2007 6,016
Friday, July 6, 2007 4,769
Saturday, July 7, 2007 3,351
Sunday, July 8, 2007 4,022
Monday, July 9, 2007 6,780
Tuesday, July 10, 2007 7,047
Wednesday, July 11, 2007 90
Totals 916,061
Averages 6,887

How Google Reacted to the Redirects

Google picked up on the 301 redirects for the core pages almost immediately. For core terms, that were heavily emphasized on-site and off-site, the rankings dropped slightly then rose back to their typical positions within about a month. Some of them dropped for as little as a week, while others stayed at #1, but for athletic scholarships the rankings are still a bit lower than they were before, with 4 sites still outranking the site, but then again I doubt we should have beat the NCAA for that term. ;)

At the low point, search traffic was about 50% of its high point, and now it is back up to about 70%.

For some portions of the site, Google took quite a while to pick up the 301 redirect. The sections of the site about state scholarship and grants programs, state student loans, and study abroad scholarship programs were slow to move because they each contained many pages, and were numerous few links from the homepage, requiring a bot to go from the homepage, to a category page, to the subcategory page, to each of the 50 or so pages in each of those sections. To help speed along that movement links to some of the deeper sections were added to the homepage.

Helping Deeper Content Get Re-indexed Quickly

We got about a half dozen average quality links after the site moved and changed the URLs in a few of our directory listings like Business.com to help show the transition of the site. The exact match domain name helped improve rankings for the core matching term, and changing the site architecture to place more emphasis on the deeper pages not only got them crawled again quickly, but also helped those pages rank better as well.

In addition to placing more emphasis on lower nodes in the site structure we also blocked some of the duplicate blog content in the robots.txt file and lowered the number of site-wide outbound links from about a half dozen to a couple. These helped focus more of the PageRank on lower pages to get them indexed quicker.

How Did the Engines Do?

Yahoo! Search

1 month later Yahoo traffic is about the same as what it was before the site moved, which likely represents a slight increase in traffic, given the seasonal trend.

Google

Google traffic is down slightly, but I think that is partly due to seasonality factors, as the site ranks where it used to for most of its competitive search terms.

Microsoft's Live Search

The big shocker is that MSN is so bad at following 301 redirects that they now send the site 0 traffic. The old site is listed URL only, while the new site is not listed.

MSN's failure to follow redirects can also be appreciated by seeing how they indexed 10,000 more pages on the SEO Book site than Yahoo! or Google do. Most of those additional pages are affiliate redirects. As of writing this, MSN also failed to index SearchEngineHistory.com.

I filed a re-inclusion request with MSN about a week and a half ago, but have got no reply so far. If it is still hosed near the end of this month I will ping a few people I know who work there.

What Should I Do Before & While Moving My Site?

  • If you have a seasonal site you can mitigate risk by moving your site early in the off season.

  • Buy ads for any brand related queries or other keywords that you feel your site must rank for.
  • Look for low information pages, and eliminate their ability to waste PageRank by blocking them from being indexed.
  • Take inventory of your most profitable sections of your site (from a search standpoint) and ensure your current internal link structure places emphasis on them. If you have a large site and some of your most profitable sections of your site are many clicks from your homepage consider raising their prominence in the site's navigational scheme to help them get re-indexed quickly. Heavily cross link toward the valuable sections whenever and wherever it makes sense.
  • If you are afraid of risking your rankings, you can try 301 redirecting one piece of your site to see how well the engines receive that, then push the rest of your site a month later if the results are good.
  • If your new site launch is a big deal ensure you submit a press release, and come up with a few good marketing / promotional ideas you can do when you launch your new site to help you build link authority and make the transition as smooth as possible. Also changing some of your better directory listings, your internal links, and any other links under your direct control to point at the new location is ideal.
  • I like to buy a bit of StumbleUpon traffic and ads on topical traffic source sites when the site moves.

Local Data is Worthless (Unless You Have a Community)

Backfence died because it was made obsolete by Google's relevancy algorithms and older local community sites. The commoditization of local data is only going to get worse. iBegin Source allows you to search or browse local business information for free, or buy an entire state for $1,000. Today Google announced they are allowing people to overlay mapplets, which will likely make Google the default source for local information inside of 2 years.

How Do I Write About & Market Important News When Legal Has to Review All Our Content?

Question: We have a multi-week turnaround on our content which makes it hard for us to write about fresh news. Is there an easy way to get past compliance, legal, and information only formatting requirements?

Answer: This is in no way legal advice, but here are a few ideas that might work...

If you have this limitation so do most of your competitors, so in that regard it probably does not hurt you much when competing with other large players. If your business model is solid you should still have a similar or higher visitor value. Where it might hurt is competing with smaller players, but since they are smaller they are typically easier to influence.

You can still attract the traffic streams of the smaller players by using affiliate programs, hiring them as consultants, buying ads on their site, sponsoring special reports they create, creating things they would want to talk about (free tools, widgets, etc.).

I am not sure of the legalities of it, but you may also want to turn some of your star content producers into people you are buying ads off of, or spin the content portion of your business off into its own business then work to cross promote, or publish a few independent blogs with legal disclaimers. Look to see what competitors are doing, and if it is legal try to do it better than they do.

Create and sponsor an independent non-profit organization that speaks for your industry. At least one of the non-profit organizations in the search marketing space is a complete joke, but it still helps its leaders gain brand recognition because they are affiliated with it. I bet more than one car donation charity was created by an auction house.

11 Unique Content Sources in Saturated Markets

Hamlet Batista commented on my last post that a problem with blogging is that many people look to the same sources for inspiration and information to blog about. Even in the most saturated markets there are a wide array of unique data sources. Here are some of my favorites:

  • your own experiences - how you save time, your favorite work tips, things that slow you down, interactions with others, how your perspective has changed over time, etc.

  • encourage readers to ask you questions - use a button like this one, and run open threads where you answer questions
  • find common questions in forums that are typically poorly answered - why doesn't Google show all my backlinks and why hasn't my PageRank changed? etc.
  • common misconceptions in your marketplace which are spread as fact - new websites can't rank, etc
  • interview other experts, create interactive contests, offer awards, etc. - these touches the ego of others and uses their reach to help market your site
  • bring back old classics that are not talked about too much in the current marketplace and compare them to the current market. You can't go wrong bringing up Orwell's Politics and the English Language, Bush's As We May Think, Hardin's The Tragedy of the Commons
  • show how your topical language has shifted - compare news from the 1950s to today, there are many stories from the 1980s and earlier which are easily accessible via Google News, but have not been pushed heavily on the web graph
  • compare information from different formats - blogs, magazines, books, dvds, conferences, stuff in Google Scholar, etc.
  • content behind firewalls - bring these ideas into the active parts of the web
  • find ideas that were popular and spread on other networks, seamlessly aggregate the best parts in a new way which allows the structure to add significant value
  • parallel markets - read information from other markets and relate it to your current market

Less Writing, Higher Quality

Value Blogging is In

Brian Clark recently highlighted that while valuable blogs continue to gain traction, the bloggers who were only popular because they were early are seeing diminished traffic and are fading in relevancy.
It makes sense that many of the original bloggers would fade because blogging about oneself is narcissistic and highly irrelevant to most readers, while blogging about technology and the web is quite easy, and there are thousands of people doing it.

What Makes Content Valuable?

The difference between value and non-value content is how unique the content and thoughts are, and how actionable the content is. If everyone else reads the same channels you and I do then us posting about their information has little value. In an ironic twist, that is one likely reason this is a low value post.

The Importance of Formatting & Framing

Almost all content ideas are recycled. The key is to target your message to an audience and format your message in a way that gives you credit as being the thought leader who came up with it. Jakob Nielson continued his (well structured) rants against blogging with Write Articles, Not Blog Postings. He didn't have to call thin low value information blog posts, but he did to help target his message at bloggers and have them spread his message.

Information Pollution

From Jakob's article

Even if you're the world's top expert, your worst posting will be below average, which will negatively impact on your brand equity. If you do start a blog despite my advice, at least screen your postings: wait an hour or two, then reread your comments and avoid uploading any that are average or poor. (Even average content undermines your brand. Don't contribute to information pollution by posting material that isn't above the average of other people's writings.)

The best channels, the ones worth paying attention to, filter. They are valuable as much for what they DON'T publish as they are for what they do publish. If you have an ad supported business model then information pollution is an effective means to increase profit margins, but if you sell consulting and/or content a different approach is required:

Elite, expertise-driven sites are the exception to the rule. For these sites, you don't care about 90% of users, because they want a lower level of quality than you provide and they'll never pay for your services. People looking for the quick hit and free advice are not your customers. Let them eat cake; let them read Wikipedia.

The reason TropicalSEO is so good is that Andy only publishes every once in a while. If you publish everyday eventually you run out of stuff to say. Blogging is just like writing songs or books. Each writer only has so much in them before they have to take a break to gather their thoughts and find more material to write about.

Jakob also mentioned that by writing longer articles that you create content which is not only of greater value, but hard to duplicate. It is why my book is read more frequently than my best blog posts, and part of what makes writing 20 page articles fun and worthwhile .

Are New Bloggers Experts?

I think we are all experts at things we have experienced, but any field worth being in takes a while to become an expert. To become a publicly recognized expert you have to

  • garner attention and keep it

  • develop many social and business relationships
  • build a personal brand
  • have thick skin

The hard part about writing in depth stuff when you are new to a market is that if you are still learning there is little upside to trying to write beyond your knowledge level. When I did that people took time out of their day to email me reminding me of what a horrible human being I am. I still get some of that.

A better approach to getting traction for a new blog is to add an element of social interaction to it, leveraging the brand and reach of others. Awards, interviews, and contests work great. After you get a bit of attention make sure to follow that up with some higher value content to turn one time readers into subscribers. It is hard to imagine a blog market more saturated than SEO, and yet in 3 weeks Patrick Altoft built 10,000 links.

How to Lose Relevancy

One can talk about the current hot memes, like Squidoo spam is right now, but ultimately nobody cares to read 31,843 people blogging about Terry Semel stepping down. The only way to build a brand talking up memes is to be the person who started the meme, or have such influence that you can re-frame the meme and gain ownership of it.

Deleting Garbage

As content quality improves short me too posts end up costing more than what they are worth. I have known of people who deleted a year and a half of archives because it wasn't worth the link equity the content was wasting, when it could be spread across higher value content.

The threshold for usefulness will continue to increase as more content is available online in richer and more interactive formats.

When Garbage Content is an Effective Monetization Strategy

Internet marketing advice is rarely universally useful. Here are 3 cases where low value information pollution is an effective strategy:

  • If you are in a market full of garbage it is not hard to beat it by being slightly different and then bolting a bit of linkbait onto it. In many consumer finance markets just rewriting the affiliate feed is all you need to start getting traction.

  • If you have an older authoritative site and are not effectively monetizing it you can add a related offer sections.
  • If you are a blog or a media website that regularly publishes news you can backdate commercially oriented posts or publish special advertisement sections without adding noise to your main channel (blog, newsletter, RSS feed, homepage, etc).

Pages