Google Keyword Research Tool: Not Popular

At a recent SMX conference, Baris Gultekin, Group Product Manager for Google AdWords, put the cat amongst the pigeons when he said the Google Keyword Tool only provides keyword data for the terms Google deems “commercial".

Teething problems? New policy? Bit of both? Regardless, it's fair to say there has been a backlash against the changes made to the keyword tool.

For example, Marty Weintraub points out:

“Facebook” Must Not Be “Commercial” Do Google users really only articulate 12 semantic permutations of “Facebook” at phrase, broad and exact match? Eeesh… Obviously that’s a laughable proposition. These 12 keywords are what Google wants to sell as they productize Facebook related queries into AdWords inventory"

Google's Business

It shouldn't come as a surprise that Google is only showing webmasters what it wants webmasters to see. Google will show data that works to Google's advantage.

There's no advantage to Google in revealing all their keyword data - a valuable asset - especially the data that Google thinks can't be monetized as profitably via Adwords. Adwords research is, after all, what the Keyword Tool is for, at least as far as Google is concerned. As much as SEOs like keyword data, Google isn't there to make SEOs lives easier.

Adwords advertisers might argue that we know which terms provide value, but that's a slightly different issue. Google may prefer to force more bid competition on keyword terms Google deems work best - in terms of searcher relevance, clickability, and for Google's bottom line. There's some merit in this, given their number crunching ability, although they don't have end revenue data for sites using Adwords. Well, not unless you give it to them.

There may well be bugs Google are working out, or we're seeing a a change in the PPC game - i.e. encourage advertisers towards the most profitable terms. At SES San Jose last year Google's Nicholas Fox highlighted that Google had about 30 million words in their ad auction. For advertising purposes, Google figures they do not need to give you a deep set of data, just the core relevant keywords and the ability to taste them via a broad match or phrase match AdWords campaign and refine with negative keywords.

As predicted, Google instant has had a significant impact on keyword diversity in some markets: "While organic traffic levels have risen about 5% for all Drive users since Instant was introduced, keyword variety has fallen more than 15%!"

However, there is still a big keyword tail, and the Google keyword tool is but one keyword resource. ;)

Other Ways To Research Keywords

There are many ways to discover keywords. But first, let's back up and focus on the user.

In a user-driven environment, like search, everything centers on typical user behavior, or, more specifically, what's in their head. Those who don't understand this seemingly innocuous piece of information often go wrong in SEO.

For a user to conduct search, they must already be aware of a concept. In this respect, search is reactive. It is difficult - although not impossible - to break a new idea or brand using the search channel, as the searcher isn't already aware of the new concept, therefore is unlikely to search on it. These type of "awareness generation" campaigns are generally better suited to interruption media, such as banners, videos and such.

Is your product/service/concept already known? Is it a brand? If so, it's a good candidate for search marketing. Listen to the way your customers talk. What phrases do they use? What questions do they ask? What problems do they have? Read the sites/magazines/publications they read and look for common terminology and reference points. Keep an eye on social networks and see what news they discuss. Feed all this information - the phrases, questions and terminology - back into your keyword list. Chances are, many of these terms will not appear on keyword research tools.

The next step is to consider searcher behavior.

82% of searchers will rephrase their query if they don't find what they are looking for on their first attempt. Combine this with the fact that 55% of queries use more than three terms, and a staggering 20 to 25% of the queries have never been seen before i.e. they are unique.

This means that there are many more keywords permutations than a keyword tool will ever give you.

If you focus on multiple low traffic terms, this can result in more traffic than can be gained from a single high traffic term. You can often achieve this simply by knowing the topics your audience are interested in, and writing about them. Is this SEO? Of course. Your language matches that of your intended audience.

So publish often. Each page you publish is a keyword net.

Look deep into your web analytics / log file. Use keyword terms found in your logs as topic/titles/starter ideas for new pages. Repeat indefinitely. You'll eventually build your unique own body of keyword data that people using keyword research tools are unlikely to find.

Always listen and adapt to your audience. Always listen and adapt to your site's analytics, as it is the purest (and most relevant) data you will ever get to use in your search marketing campaigns.

Free Keyword Research Tools

We're going to blow our own horn here and recommend the SEOBook keyword tool, powered by Wordtracker. It's free, and provides a lot data across various search services. The SEOBook members section has some very cool tools, too, including a Competitive Research tool based on SEMRush data. This data can list keyword value distribution i.e. keyword value * estimated traffic. Aaron did a thorough review of SEMRush here.

But enough about us.... :)

Google still offer a range of great freebie tools, including:

Google Trends

Google trends for websites

Insights for search

Google Sets

Microsoft's Ad Intelligence is too good to not mention.

Don't forget to use a Thesaurus - such as Thesaurus.com. A Thesaurus can often cough up synonyms the keyword research tools miss. Aaron has a video and a few more keyword tools listed here.

And virtually anything can be a source of data to explore

• a competing site's structure
• "most popular" articles highlighted on websites
• "most viewed" or "most commented" blog posts & forum threads
• sites which share their stats publicly
• statistically improbable phrases from a book
• the chapter & section titles of a book
• putting a competing page through a keyword density analyzer
• customer questions via email

The well is deep!

There is a ton of data out there, whether Google chooses to share it or not.

The very best keyword data is seldom shared intentionally ;) though sometimes when people sell their site they do offer "free milk."