Yahoo! Search Update

Yahoo! recently did another search index update with little fanfare. To me it looks like they put more weight on exact match domain names, inbound link anchor text, and domain authority.

What is Content? When is Publishing a Commodity?

People Tolerate (and Expect) Ads:

As Google's ad network gets more efficient and Google controls more bits, almost everything in any commercial market is going to sell something, subsidize another commercial interest, or have ads on it. To appreciate how much this trend will grow, just read the comments on my post about using custom search engines. Many of the comments are insightful, but almost nobody appreciated that not having ads in your internal site search result was a value add for a commercial site. If you look at blog search results for shower gel you will see Google's version of the web. Google and Microsoft already own in game advertisement firms. With Google bidding billions of dollars for part of the US wireless spectrum you can bet that there are going to be even more ads between content producers and consumers.

It doesn't matter what ads appear on the publisher site if they didn't sell the ads directly themselves. Somehow the publisher is off the hook because they didn't know, and it wasn't Google's fault because the system is partially automated.

Newspapers Practicing Arbitrage

Google killed some of the scraper AdSense garbage, but now arbitrage is going mainstream, with newspapers leveraging their brands to publish thin content from freelance writers. Think of newspapers buying ISP data or mining their server logs and suggesting writers create content about crap that got a lot of traffic when Google featured their story on the homepage in the past.

Newspapers Practicing Automated Garbitrage

The above trend is going to make the most competitive keywords even harder to rank for as there will always be at least one fresh news story about every high value topic.

  • This just in...make money with the best forex platform!

  • This just in...mortgage rates are at all time lows!
  • This just in...online education is more affordable than you thought!

Google has already proved that they don't mind large publishers creating robotic content and building its authority with spammy links. Did you know that BizJournals offers Google searchers a credit card application service? If I posted similar content on this site Google would kill it.

Anyone Can do Public Relations

Beyond the recycled content, the movement will also be fueled by lazy underpaid writers who will heavily rely on social media and others work for story ideas. If you are good at spamming social media then the mainstream media will act as your megaphone without requiring you to hire a PR firm.

When is a Link Buy Legitimate?

Most profitable publishing enterprises are moving toward producing lower quality content. As a publisher, a brand is nothing more than something that allows you to practice arbitrage while appearing that you provide a valuable service. It allows you to get extra exposure and charge a higher rate for your ads.

The only difference between a smart strategic ad buy and a text link sale that Google wants you to report is the publisher's rate card.

The $10,000 Robots.txt File...Ouch!

I recently changed one of my robots.txt files pruning duplicate content pages to help more of the internal PageRank flow to the higher quality and better earning pages. In the process of doing that, I forgot that one of the most well linked to pages on the site had a similar URL as the noisy pages. About a week ago the site's search traffic halved (right after Google was unable to crawl and index the powerful URL). I fixed the error pretty quickly, but the site now has hundreds of pages stuck in Google's supplemental index, and I am out about $10,000 in profit for that one line of code! Both Google and Yahoo support wildcards, but you really have to be careful when changing a robots.txt file because a line like this
Disallow: /*page
also blocks a file like this from being indexed in Google
beauty-pageants.php

Unless you are thinking of that in advance it is easy to make a mistake.

If you are trying to prune duplicate content for Google and are fine with it ranking in other search engines, you may want to make those directives specific for GoogleBot. If you make a directive for a specific robot, that bot will ignore your general robots directives in favor of following the more specific directives you created for it.

Google's webmaster guidelines and Yahoo!'s Search Blog both offer tips on how to format your robots.txt file.

Google also offers a free robots.txt test tool, which allows you to see how robots will respond to your robots.txt file, notifying you of any files that are blocked.

You can use Xenu link sleuth to generate a list of URLs from your site. Upload that URL list to the Google robots.txt test tool (currently in 5,000 character chunks...an arbitrary limit I am sure they will eventually lift).

Inside the webmaster console Google will also show you what pages are currently blocked by your robots.txt file, and let you view when Google tried to crawl the page and noticed it was blocked. Google also shows you what pages are 404 errors, which might be a good way to see if you have any internal broken links or external links pointing at pages that no longer exist.

People Want to Be Inspired

As I have watched brands grow and die I have come to the conclusion that the concept of value is an important ingredient in a successful site, but as an editorial content provider, value alone is not enough to grow much past sustainability.

People want to have something they can talk about and share, something they can relate to, and something that inspires them and brings them hope. Emotional appeal is typically far more important than logical appeal. In some cases the hope is for change of things they do not like, but that has three potential outcomes

  • the fight is one that will not be won (and the ideas you are fighting against may be gaining ground in other markets)

  • change is brought about and the people creating change eventually forget where they came from, walk on other people without care, and become the people they once despised (relevancy comes and goes like the seasons)
  • change is brought about and the cause no longer exists

After a battle is over you still have to have something worth talking about and create something inspiring if you are to keep attention and mindshare.

How much money is spent marketing fear to people? Do you need health insurance? Would you still need it if you ate healthy? People gravitate toward their interests for an escape from fear-mongering and other injustices that are baked into society.

Fear is easy to forget, and is a message that needs to be marketed over and over again to cause response. In most cases, spreading fear also has limited upside social potential...it is more commonly used to exploit people, so we learn to tune it out. We take off our shoes before going through the metal detectors, but how many people still sleep on the airplane? I know I do. And people live on the sides of mountains, in fire zones, and where earthquakes are common. I know I do.

Why do award programs work so well? They are easy to relate too and make people feel good. Nearly daily I get emails about how a person quit their job or sold their business because I inspired them and helped them want to go out on their own.

When I think of the most explosive growth periods in brands I have helped build they are typically associated with periods of time when the owner was passionate about what they were doing. That passion attracts people who share it and want to spread it. When I think of the periods that they lost marketshare they are more in-line with the times they lack creativity and passion.

In a saturated market packaging and formatting are often more important than the message. Sustainable value systems change with the times.

As an entrepreneur most of us have many failures before we have any successes. I recently made the Technorati top 100 blogs, but have already dropped to 102, and that position will likely fade unless I can be a bit more inspiring than I have been recently.

Hiring a Google Gadget Developer

I am looking to hire someone to create a few Google Gadgets for the the iGoogle homepage. If you are good at that sort of stuff please contact me. My email address is seobook@gmail.com.

Losing Self Respect, One Conversion at a Time

If you mimick leading marketing in your filed, then deflect the blowback from the effective marketing onto others who gave you the tips that helped you increase your conversion they may not appreciate that:

We also have inline ads, ala Aaron Wall's suggestion (so blame him if you're angry )

If you have a professional copywriter suggest revising your content, then state that using similar marketing to their past work is below you they may not appreciate that:

Aaron Wall told me his conversion rate after switching to a page format like this has sent his sales skyrocketing to unbelievable proportions. Yes, this saddens me, but it's also inspiring - if they can do it, so can we, right?

Well, maybe. But we've got to be willing to sacrifice a bit of self-respect and dignity to get it done.

Then if you hold a marketing contest, saying anything goes you are bound to raise a few eyebrows

Anything goes, dude. We're looking for the best-converting page, and if you think the best way to get conversions is with flying monkeys and marquee tags, then have at it!

In every market controlling conversations is associated with trust and authority. That is why framing issues, business models, and sales techniques is so important if you want to prove your model is better than competing models. Few people claim their own businesses are bad.

Few people think of the Google Toolbar as spyware if it is marginally useful. Stop Badware, the non profit reporting on spyware, is not likely to make a report criticising Google's Toolbar or search helper redirection so long as Google is their lead corporate sponsor. The attempt to portray honesty and openness is sometimes more important than being either of them.

Rand recently created a landing page contest, but the problem with this sort of contest is that the very act of making noise increases sales, but it also makes your traffic stream dirtier and less consistent. After Brian Clark rewrote my salesletter he quickly became one of my top affiliates. His reach and brand increased my reach and brand, but it also led to many others writing about me, and more people searching for my brand...some of those traffic sources were really clean while others were less so...perhaps less interested and more driven out of curiosity or following what was popular at that point in time. Plus those that were pre-sold on Brian's site probably didn't need any salesletter at all to convert. The fact that he wrote it made his readers convert exceptionally well though because his copy is in tune with his readers, and they already trust him.

I had to wait on testing the results because I had too much temporal noise around my brand to provide accurate short term testing results. Inadequate data sampling provides inconclusive data. Even A/B split test results that are allegedly 95% certain are wrong 1 out of 20 times.

As John Andrews stated landing pages are only one part of the marketing mix. It is hard to track conversion for high touch and high trust products or services using multivariate testing while allowing others to write your conversion copy. Unless they write your ad copy, a sales letter, conversion funnel, and bonus offers that are consistent with your brand they are throwing darts at a wall. It may create conversation, builds your reach, boosts your brand exposure, give the perception that you are open, and can give you a few good ideas on how to improve conversions, but likely the test will bear false results, especially since brand positions and sales techniques are so temporal and few people willing to write free copy understand all the nuances of effective sales techniques.

If your are going to "sacrifice a bit of self-respect and dignity" you might as well get meaningful results out of it Rand. ;)

Ad Networks as Spyware Providers

I recently posted about the online security wars, a trend which will continue to grow as spam filtering improves. The WSJ recently ran an interesting article about ad networks distributing spyware:

In May, a virus in a banner ad on tomshardware.com automatically switched visitors to a Web site that downloaded "malware" -- malicious software designed to attack a computer -- onto the visitor's computer. ScanSafe Inc., one of the first security firms to discover the virus, estimates the banner ad was on the site for at least 24 hours and infected 50,000 to 100,000 computers before Tom's Hardware removed it.

As traffic streams consolidate and ad networks improve in efficiency many people who get marginalized are going to get more insidious in their attempts to make money. This fear will further consolidate web traffic toward trusted brands and place a premium on central ad networks.

The Benefits of Using Google's Custom Search Engine

I recently installed the new business edition of Google's custom search engine. It took about 5 minutes to set up and will likely pay for itself many times over. You can use it by using the search box in the right rail. Google's custom search engine is easy to customize, fast to implement, works across multiple domains and is cheap. It only costs $100 a year for up to 5K pages and $500 a year for 50K pages. It also allows the owner to see search frequency and your most popular queries.

Rather than doing Google site specific searches, now I can see Google flavored results customized to my site right from my own domain. If I am searching for something on my own site and can't find it, the odds are pretty good that I am not going to show up in Google's global search either.

Using Google's search will likely also help you figure out how well they trust different pages on a site or a subset of sites, as it likely factors in link popularity and other off site relevancy measurements (unlike most site search services). If your site is easier to search it is easier for you and others to cite your archived content.

Using search results with the same format as Google's will also show you how compelling your documents look to customers on Google, from a searcher's perspective. Some consumers may also view your business as being more Google friendly if you use a Google site search service. Their awareness of and affinity toward Google's brand may help increase your conversions.

When Niche Market is the Wrong Advice

Andrew Goodman wrote a delightfully good column over at SEL about how and when going niche is wrong. He noted that the advice to go niche typically comes from niche market thought leaders and people selling information products who tend to think everyone else is just like them.

The reason people push the niche idea are:

  • it is easy advice to give

  • overestimating the value of our own experience
  • the web has many self-reinforcing effects that favor market leaders
  • it is hard to conquire a big field until you have proven success and gained confidence from conquering a smaller niche
  • it doesn't take much to get past self sustaining if you focus on something you are passionate about
  • it is harder to get burned out working on something you are passionate about

It is a mistake I have made many times, but it is easy to speak of your own experiences as truth, especially if you are expected to write nearly every day. Focusing too tightly on a niche for an extended period of time makes it easy to become inauthentic, inaccurate, boring, and/or pessimistic. Those are obvious to anyone who reads frequently, and they are counter to what allows companies to expand:

We can certainly see how small to midsized retailers—online and off—who undergo recognizable and surprising spurts of growth, seem to have something in common beyond the executional wisdom and recruiting skills of the larger enterprises that Collins tracks in Good to Great. That intangible seems to be: contagious excitement.

The market for something to believe in is infinite.

If you are making 50 websites it is likely that you will be more successful if you focus on one or two of them before building out all the others, such that you incorporate learning from your first few sites into your later sites. Why repeat all the mistakes? In the same sense, it helps to conquer one market position before going too broad. When I started on the web I wanted to know a lot more about search than I could have profitably done, especially since the market already had clear market leaders with momentum, social relationships, media relationships, strong brand, capital, and knowledge advantages over me.

After I niched down to SEO and built a brand I was able to create numerous revenue streams (consulting sales, ebook sales, direct ad sales, contextual ad sales, affiliate commissions, link sales, referral commissions, speaking engagements, etc). It is also easy to further leverage my knowledge gained building this site to provide more revenue streams away from this site. I have a network idea I want to launch soon, but it still needs some work.

I have long been a fan of parallel markets after attaining self reinforcing authority status in a core market.

Google Ringtone Search (Beta)... the Utility of Search & the Value of Relationships vs Data

Google recently announced they are offering an ad free search service to small businesses for as little as $100 a year. They are also rumored to be working on creating a mobile search offering to search for ringtones, cell phone games, and other high margin cell phone services:

With the new system, users would search for a piece of content -- such as ringtones -- and would get back a list of companies that provide it, with links letting them easily purchase the material. Eventually, Google would charge companies for high placement in the search results, much as it does with "sponsored links" on Web search, the people familiar with the company's plans said.

If this proves effective, why wouldn't Google create search services for real estate and student loans? Off the start they likely wouldn't be any dirtier than those markets already are.

The local sites (or other niche sites) that die are the ones that don't have a supportive community...the ones that can roughly be described as data. But as you move your site away from data toward community building and user experience you increase its marketability, profitability, and longevity.

John Scott recently said

when the Internet becomes a community more than a marketplace, the community oriented merchants have the advantage.

Most general music lyric sites will lose over 90% of their traffic and value in the next 2 years. Niche value add sites like SongMeanings.net will increase in value.

Some newspapers are fighting off their own irrelevancy by publishing more ads, and becoming more irrelevant. The reason WSJ.com is worth $5 billion is because it will be one of the last newspapers surviving after many competitors are marginalized. Its brand and business relationships make it unique.

The next step for search is moving away from age and citation based trust toward user satisfaction based rating. In a recent interview Jakob Nielson said:

Of course, the big thing that has happened in the last 10 years was a change from an information retrieval oriented relevance ranking to being more of a popularity relevance ranking. And I think we can see a change maybe being a more of a usefulness relevance ranking. I think there is a tendency now for a lot of not very useful results to be dredged up that happen to be very popular, like Wikipedia and various blogs. They’re not going to be very useful or substantial to people who are trying to solve problems. So I think that with counting links and all of that, there may be a change and we may go into a more behavioral judgment as to which sites actually solve people’s problems, and they will tend to be more highly ranked.

Part of that will include collecting user feedback and personalizing results. For the commercial parts of the web, much of this will be done by search engines pushing more aggressive ad units (while using non-ads to test how far they can go), licensing content from premium publishers, and making more transactions direct to funnel traffic internally while minimizing the downside of bait and switch marketing. If Google controls the transaction they know the lead value AND the customer satisfaction stats. If something that was once good becomes spam Google controls distribution and monetization, and can kill it without a second thought.

If you are in a market that is likely to get commoditized what does your site do that makes it more than just data?

Pages