Editorial Objectivity
Should search engines be able to preferentially promote their own services in their search results?
Nearly 3 in 4 people think that search engines should not be able to preferentially promote their own services.
| vote | All (1226) |
|---|---|
| no, results should be objective | 74.1% (+3.1 / -3.4) |
| yes, it is their search results | 25.9% (+3.4 / -3.1) |
There was essentially no split between men & women.
| vote | Men (827) | Women (399) |
|---|---|---|
| no, results should be objective | 73.7% (+3.1 / -3.4) | 74.4% (+5.2 / -6.0) |
| yes, it is their search results | 26.3% (+3.4 / -3.1) | 25.6% (+6.0 / -5.2) |
Older people tend to prefer/want more editorial objectivity, whereas younger people are more fine with search engines preferentially promoting their own services. Older people tend to be more fixed in their ways & younger people are much less so.
| vote | 18-24 year-olds (338) | 25-34 year-olds (269) | 35-44 year-olds (158) | 45-54 year-olds (209) | 55-64 year-olds (169) | 65+ year-olds (83) |
|---|---|---|---|---|---|---|
| no, results should be objective | 65.0% (+4.9 / -5.2) | 76.0% (+5.1 / -6.0) | 74.0% (+6.5 / -7.7) | 71.2% (+5.7 / -6.5) | 71.4% (+6.5 / -7.5) | 87.2% (+6.1 / -10.4) |
| yes, it is their search results | 35.0% (+5.2 / -4.9) | 24.0% (+6.0 / -5.1) | 26.0% (+7.7 / -6.5) | 28.8% (+6.5 / -5.7) | 28.6% (+7.5 / -6.5) | 12.8% (+10.4 / -6.1) |
Geographically, people in the south & midwest tend to be slightly more trusting, perhaps due to the lower cost of living & less competitive markets. However, any differences here are fairly minor & are within the margin of error.
| vote | The US Midwest (244) | The US Northeast (367) | The US South (352) | The US West (263) |
|---|---|---|---|---|
| no, results should be objective | 72.2% (+6.4 / -7.4) | 77.7% (+4.5 / -5.3) | 72.1% (+6.0 / -6.9) | 75.9% (+5.7 / -6.9) |
| yes, it is their search results | 27.8% (+7.4 / -6.4) | 22.3% (+5.3 / -4.5) | 27.9% (+6.9 / -6.0) | 24.1% (+6.9 / -5.7) |
People who are rural tend to be slightly more accepting of Google doing as it wishes, though this is also a small sample size & well within the margin of error.
| vote | Urban areas (647) | Rural areas (106) | Suburban areas (453) |
|---|---|---|---|
| no, results should be objective | 74.3% (+4.3 / -4.9) | 71.9% (+8.5 / -10.5) | 74.4% (+4.2 / -4.7) |
| yes, it is their search results | 25.7% (+4.9 / -4.3) | 28.1% (+10.5 / -8.5) | 25.6% (+4.7 / -4.2) |
There isn't a strong correlation with income on this issue either. People cared a bit more at higher income levels, but there was also a wider margin of error due to small sampling size.
| vote | People earning $0-24K (142) | People earning $25-49K (677) | People earning $50-74K (316) | People earning $75-99K (75) | People earning $100-149K | People earning $150K+ |
|---|---|---|---|---|---|---|
| no, results should be objective | 72.0% (+7.8 / -9.4) | 76.8% (+3.7 / -4.1) | 68.7% (+6.1 / -6.8) | 83.1% (+6.9 / -10.2) | Insufficient data | Insufficient data |
| yes, it is their search results | 28.0% (+9.4 / -7.8) | 23.2% (+4.1 / -3.7) | 31.3% (+6.8 / -6.1) | 16.9% (+10.2 / -6.9) | Insufficient data | Insufficient data |
Comments
results speak for themselves, bit of a no-brainer really. I would like to know how objective Google REALLY is when displaying search results.
The data is essentially useless because of the framing of this question. Not that it will stop anyone from using this as certifiable proof! But really, it would be nice if survey design was at given at least a quick thought.
...but if the purpose is to provide constructive feedback you may as well give an example of what you would do differently & how. It is pretty hard to convey some of the complexity of search in a 100 letter question and about a 30 letter answer.
Cough. It would be nice if comment feedback usefulness was given at least a quick thought. :D
True! If you're going to complain offer a solution. That's fair.
So, first I'll go to another industry since that might be a little less controversial. So lets say you shop at Safeway. They happen to make and sell their own food, or sell food marketed under their Safeway Select brand.
How do people feel about that? If I wanted to get a certain answer I could write the question as:
Should a grocery store be prevented from promoting their own products on store shelves? Even better, Should a grocery store be forced to promote other products over their own on their own store shelves? I'm guessing most would say no. But I've used the word preventing and forced which are signal words.
Your question uses preferentially as a signal word. Now amid a battery of other questions it might work, but in isolation I have no idea how people are interpreting that signal word. What does 'preferentially promote' mean? And what services are we talking about?
Me? I might start with a very simple question. Should a search engine be able to present their own services on search results?
I'd then try specific questions. Should Gmail be one of the results returned for a search for 'email'?
Then perhaps something like: On Google, should Gmail be the first result returned for a search for 'email'?
Then maybe: On Bing, should Gmail be one of the results returned for a search for 'email'?
Now there are other ways Google can promote their own properties. If you wanted to understand the impact of Places integration you'd need to see if people understood it first, right? Getting results from people who don't actually understand the question won't work.
I'm still not doing much of this justice really. You need to fight like hell against bias to construct good questions and a great survey. Otherwise you just wind up with something like in this video: http://www.youtube.com/watch?v=G0ZZJXw4MTA
The issue of concern is not presentation though...it is over-promotion that becomes a core issue, as it is what destroys economic incentives for other publishers & that forced economic leverage is how the search engines try to buy leverage to create relevancy in their new offerings. If they can offer premium placement to their new niche offerings then in any vertical they can partner with players 2 through x to create a new thin Google layer & then lambast player #1 for not being "open." This is just a general function of 1 to many marketplaces where a single entity dictates terms of engagement for almost all other players.
Some examples:
Information is unlike physical products in that the marginal cost of reproduction is zero. In large part the information *is* the product, so if it is scraped and reproduced without payment & ranked preferentially then it is like more than just selling a similar cookie, it is more akin to cloning the entire supply chain & being willing to run it at negative margins until you collapse a competitor.
BTW...am running a bunch of other questions trying to get at user understanding of search results. Some of them are picture-based, some are text-based, etc. Some questions offered 5 choices & a picture, some offered 2 pictures side by side, some were other background questions as well. The thing is...some of the data is still coming in & I thought it was better to put out these 1 at a time than to make a mega-post that is so long that nobody reads it.
Also note that in the above asked question I did not single out any individual search engine. And, for what it is worth, Ask and Yahoo! are far more extreme with this stuff than Google is & if Microsoft had Google's marketshare I am sure they would be worse than Google too.
AJ
Your safeway analysis misses the point for 2 reasons.
1. Most searches don't even think of results as a product. We all know safeway dogfood is both a product and sold under the safeway label before we buy it.
2. Search results are the defacto "reality" for most searchers. Google gets to define what is true. That is implicit in Google's position of grave moral concern about manipulating health care results; why is it any less true if the search is for a browser, a social network or the "best" air fare?
It is pretty easy to reduce this point to the ridiculous on either side of the argument from Matt Cutts declaring search results are protected by the first amendment to Rick Santorum taking offense at being defined by a scatological neologism. Reality is more nuanced and the fact is that users appear to want objective results, privacy, security while they want everything to be free. What they actually get at the moment is a many headed hydra that is influenced by corporate objectives, advertisers, fear of regulation and some true believers fighting for "quality" and "truth"
Aaron,
I understand the argument and actually agree on a couple of points. The issue for me is getting data that really supports and reveals. It's a pet peeve of mine.
For instance the first report Google put out saying that 89% of paid clicks were incremental was complete bunk and I said so, loudly. Or at least I said it where I thought it would count: Google+.
The follow-up study was better but still has some holes in it. They could and should go farther. Now to the idea of over-promotion.
You Panda examples are irksome for sure and I am not a great fan of Panda because it treats great content the same as lousy content.
http://www.blindfiveyearold.com/farmer-update-about-sites-not-content
There are plenty of eHow articles that I've actually found personally helpful. (The garbage disposal works again!) Now, overtime, I would say that results have gotten marginally better. And Google is still the best we have.
The vertical integration is interesting. I've had a number of clients in those crosshairs. It's tricky though. If they weren't using content from others then no one would have a problem with it, right? So if they just buy those competitors and then integrate, do we feel better? (aka Zagat)
If Google can improve on the experience within a vertical it keeps people using Google which will ensure that it continues to draw revenue. Is the Places integration a better experience? Today, I'd argue yes. One of the issues is that the presentation Google can provide Places is not afforded to others. So if someone else did what Google did it still might not have the same impact.
That sucks but it's Google's search engine. I mean, some people seem to believe that Google is a public service.
https://plus.google.com/u/0/115106448444522478339/posts/2wHFhCQsTsJ
Should Google be prevented from bringing more efficiency to these verticals? I'm not saying it feels good and there are a number of people on the short end of the stick but we don't see a lot of milkmen these days either.
What about the 'weather' results. I can get it right on Google. But there are links to three sites. Do those sites enjoy that placement, thinking of the weather results presented as the freemium offering? And what about other weather players who aren't in the weather onebox?
By the way, many of your other questions are quite good and producing interesting results.
Jonah,
You think everyone knows that the 'Organics' brand is Safeway? You have FAR more faith in consumers than I.
And one of the points I make which is related to any survey or data is that what people say and what they do are VERY different. If privacy were such a huge deal why is Facebook still humming along?
People quickly give away address, phone number and other personal information for a chance to win a free iPad. Privacy is a paper tiger. I wish it wasn't but the evidence is ... everywhere. (Why are people giving their driver's license to cashiers with their credit cards?)
Same thing with 'objectivity'. Show people personalized results and they like them MUCH better. Their behavior is clear. Ask them whether they want personalized results? They say no and cry foul. (And then promptly go back to enjoying the non-objective experience.)
To me the answer to that question is "yes." There is a significant difference between investing capital into a market to compete in it vs just shagging someone else's work and passing it off as one's own.
When it is from shagging 3rd party content & displacing it then that isn't making the market more efficient. That is destroying some of the highest quality sources by bankrupting them. Does it thin the herd after long enough? Sure. But then so did agent orange ;)
Speaking more in terms of economics, this is (quite literally) a play of Gresham's law: "When a government compulsorily overvalues one type of money and undervalues another, the undervalued money will leave the country or disappear from circulation into hoards, while the overvalued money will flood into circulation."
Even from the consumer perspective, in many cases it isn't more efficiency, but more friction. Consider what the booking process through Google flight search looks like for airlines that are not paying Google. Users get higher prices & less selection. Further, companies that were selling the airline tickets at no profit (and perhaps sometimes loss) were prohibited from participating in the new channel, even though their years of experience created many layers of efficiency for consumers.
In the forums today someone shared an example of Google estimating the population of a city where they were off by a couple orders of magnitude. And if that stays for very long suddenly that becomes a "fact" that works its way into eHow styled content around the web. ;)
One topic where there were issues (image sizing, etc.) I asked questions 4 different ways in order to get results on. It is taking a while to collect all the feedback...but it is an interesting process.
One thing I just realized today is that I should have used "I don't know" as an answer to more of the questions, such as to have another lens on the feedback.
Also agree 100% on the huge gulf between what people like & what they say they like.
I forgot to mention that one of the reasons I referred to Google and specific queries was to try to get the opinion on a subject without asking about that subject directly.
Concrete and specific examples allow people to quickly respond without thinking about the 'bigger picture'. If you get enough of those questions under your belt (not cheap I know) then you can start to tease out user attitudes instead of asking for those attitudes directly.
Sort of like asking whether you're a liberal or a conservative versus asking specific questions about topics or policy.
Flight Search does not improve the user experience enough for me right now. Then again, for a long time Places was the same way and now I think it does.
What's interesting is that many of these verticals rely heavily on Google. In some ways they'd gotten pretty comfortable and the offerings weren't getting much better.
It's a symbiotic relationship. Many of these sites may still get traffic, just less of it. Google becomes the middleman more often because they think they can do it better. Now, clearly how they've behaved with Yelp and TripAdvisor is pretty unsavory. They DO throw their weight around.
It's fascinating to watch how Southwest has navigated their Internet strategy. They gave the finger to all the online aggregators. They'll probably give Google the finger too.
I'm not saying everyone can do this but if you're in an industry where Google thinks they can do it better, you'd best figure out how to up your product AND find other sources of traffic.
As for accuracy, I think Google takes that pretty seriously. It's just difficult. I'm not sure our research skills are getting better - might actually be getting worse because of our reliance on Google. But there ARE people at Google concerned about this, most notably Dan Russell.
Whether that shows up consistently in the product is another story.
Add new comment