All

Posted on

Online Marketing at Big Skinny (teaching materials) with Scott Kominers

Edelman, Benjamin, and Scott Duke Kominers. “Online Marketing at Big Skinny.” Harvard Business School Case 911-033, February 2011. (Revised February 2012.) (educator access at HBP. request a courtesy copy.)

Describes a wallet maker’s application of seven Internet marketing technologies: display ads, algorithmic search, sponsored search, social media, interactive content, online distributors, and A/B testing. Provides concise introductions to the key features of each technology, and asks which forms of online marketing the company should prioritize in the future. Discusses similarities and differences between online and off-line marketing, as well as issues of marketing campaign evaluation.

Supplement:

Online Marketing at Big Skinny — slide supplement – PowerPoint Supplement (HBP 912006)

Teaching Materials:

Online Marketing at Big Skinny – Teaching Note (HBP 911034)

Posted on

In Accusing Microsoft, Google Doth Protest Too Much

In Accusing Microsoft, Google Doth Protest Too Much. HBR Online. February 3, 2011.

Google this week sparked a media uproar by alleging that Microsoft Bing “copies” Google results. But is that actually the best characterization of what happened? In fact Google’s engineers intentionally clicked bogus listings they had previously inserted into Google’s results, and they did this on computers where they had specifically authorized Microsoft to examine their browsing in order to improve Bing.

Strikingly, Google’s own Matt Cutts previously endorsed the use of Toolbar and similar data to improve search results — calling this approach “a good idea.” And Google’s own Toolbar Privacy Policy allows Google to perform the same analysis Bing used. So I don’t have much sympathy for Google’s allegations of impropriety. Quite the contrary: With Bing’s small market share, this data is important in improving Bing search results and building a viable competitor to Google’s dominant search offering.

Details, including what exactly happened, Google’s prior statements, and Google’s widespread use of others’ intellectual property:

In Accusing Microsoft, Google Doth Protest Too Much

Posted on

Measuring Bias in “Organic” Web Search with Benjamin Lockwood

By comparing results between leading search engines, we identify patterns in their algorithmic search listings. We find that each search engine favors its own services in that each search engine links to its own services more often than other search engines do so. But some search engines promote their own services significantly more than others. We examine patterns in these differences, and we flag keywords where the problem is particularly widespread.

Even excluding “rich results” (whereby search engines feature their own images, videos, maps, etc.), we find that Google’s algorithmic search results link to Google’s own services more than three times as often as other search engines link to Google’s services.

For selected keywords, biased results advance search engines’ interests at users’ expense: We demonstrate that lower-ranked listings for other sites sometimes manage to obtain more clicks than Google and Yahoo’s own-site listings, even when Google and Yahoo put their own links first.

Details, including methodology, analysis, and policy implications:

Measuring Bias in “Organic” Web Search

Posted on

Bias in Search Results?: Diagnosis and Response

Edelman, Benjamin. “Bias in Search Results?: Diagnosis and Response.” Indian Journal of Law and Technology 7 (2011): 16-32.

I explore allegations of search engine bias, including understanding a search engine’s incentives to bias results, identifying possible forms of bias, and evaluating methods of verifying whether bias in fact occurs. I then consider possible legal and policy responses, and I assess search engines’ likely defenses. I conclude that regulatory intervention is justified in light of the importance of search engines in referring users to all manner of other sites, and in light of striking market concentration among search engines.

Posted on

Adverse Selection in Online ‘Trust’ Certifications and Search Results

Edelman, Benjamin. “Adverse Selection in Online ‘Trust’ Certifications and Search Results.” Electronic Commerce Research and Applications 10, no. 1 (January-February 2011): 17-25.

Widely used online “trust” authorities issue certifications without substantial verification of recipients’ actual trustworthiness. This lax approach gives rise to adverse selection: the sites that seek and obtain trust certifications are actually less trustworthy than others. Using an original dataset on web site safety, I demonstrate that sites certified by the best-known authority, TRUSTe, are more than twice as likely to be untrustworthy as uncertified sites. This difference remains statistically and economically significant when restricted to “complex” commercial sites. Meanwhile, search engines create an implied endorsement in their selection of ads for display, but I show that search engine advertisements tend to be less safe than the corresponding organic listings.

Posted on

Knowing Certain Trademark Ads Were Confusing, Google Sold Them Anyway — for $100+ Million

Disclosure: I serve as a consultant to various companies that compete with Google. But I write on my own — not at the suggestion or request of any client, without approval or payment from any client.

When a user enters a search term that matches a company’s trademark, Google often shows results for the company’s competitors. To take a specific example: Searches for language software seller "Rosetta Stone" often yield links to competing sites — sometimes, sites that sell counterfeit software. Rosetta Stone think that’s rotten, and, as I’ve previously written, I agree: It’s a pure power-play, effectively compelling advertisers to pay Google if they want to reach users already trying to reach their sites; otherwise, Google will link to competitors instead. Furthermore, Google is reaping where others have sown: After an advertiser builds a brand (often by advertising in other media), Google lets competitors skim off that traffic — reducing the advertiser’s incentive to invest in the first place. So Google’s approach to trademarks definitely harms advertisers and trademark-holders. But it’s also confusing to consumers. How do we know? Because Google’s own documents admit as much.

Today Public Citizen posted an unredacted version of Rosetta Stone’s appellate brief in its ongoing litigation with Google. Google had sought to keep confidential the documents that ground district court and appellate adjudication of the dispute, but now some of the documents are available — giving an inside look at Google’s policies and objectives for trademark-triggered ads. Some highlights:

  • Through early 2004, Google let trademark holders request that ads be disabled if they used a trademark in keyword or ad text. But in early 2004, Google determined that it could achieve a "significant potential revenue impact" from selling trademarks as keywords. (ref)
  • In connection with Google’s 2004 policy change letting advertisers buy trademarks as keywords, Google conducted experiments to assess user confusion from trademarks appearing in search advertisements. Google concluded that showing a trademark anywhere in the text of an advertisement resulted in a "high" degree of consumer confusion. Google’s study concluded: "Overall very high rate of trademark confusion (30-40% on average per user) … 94% of users were confused at least once during the study." (ref)
  • Notwithstanding Google’s 2004 study, Google in 2009 changed its trademark policy to permit the user of trademarks in advertisement text. Google estimated that this policy change would result in at least $100 million of additional annual revenue, and potentially more than a billion dollars of additional annual revenue. Google implemented this change without any further studies or experiments as to consumer confusion. (ref)
  • Google possesses more than 100,000 pages of complaints from trademark holders, including at least 9,862 complaints from at least 5,024 trademark owners from 2004 to 2009. (ref)

Kudos to Public Citizen for obtaining these documents. That said, I believe Google should never have sought to limit distribution of these documents in the first place. In other litigation, I’ve found that Google’s standard practice is to attempt to seal all documents, even where applicable court rules require that documents be provided to the general public. That’s troubling, and that needs to change.

Posted on

Hard-Coding Bias in Google “Algorithmic” Search Results

I present categories of searches for which available evidence indicates Google has “hard-coded” its own links to appear at the top of algorithmic search results, and I offer a methodology for detecting certain kinds of tampering by comparing Google results for similar searches. I compare Google’s hard-coded results with Google’s public statements and promises, including a dozen denials but at least one admission. I conclude by analyzing the impact of Google’s tampering on users and competition, and by proposing principles to block Google’s bias.

Details, including screenshots, methodology, proposed regulatory response, and analogues in other industries:

Hard-Coding Bias in Google “Algorithmic” Search Results

Posted on

A Closer Look at Google’s Advertisement Labels

Google's tiny 'Ads' labelGoogle’s tiny ‘Ads’ label

The FTC has called for “clear and conspicuous disclosures” in advertisement labels at search engines, and the FTC specifically emphasized the need for “terms and a format that are easy for consumers to understand.” Unfortunately, Google’s new advertisement labels fail this test: Google’s “Ads” label is the smallest text on the page, far too easily overlooked. (Indeed, as I show in the image at left, the “Ads” label substantially fits within an “o” in “Google.”) Meanwhile, Google now merges algorithmic and advertisement results merged within a single set of listings; Google’s “Help” explanations are inaccurate; and Google uses inconsistent labels mere inches apart within search results, as well as across services.

Details, including the shortfalls, screenshots, comparisons, and proposed alternatives:

A Closer Look at Google’s Advertisement Labels

.

Posted on

Labels and Disclosures in Search Advertising with Duncan Gilchrist

Disclosure: I serve as a consultant to various companies that compete with Google. But I write on my own — not at the suggestion or request of any client, without approval or payment from any client.

Search engines have long labeled their advertisements with labels like “Sponsored links”, “Sponsored results”, and “Sponsored sites.” Do users actually know that these labels are intended to convey that the listings are paid advertisements? In a draft paper we’re posting today, Duncan Gilchrist and I try to find out.

“Sponsored Links” or “Advertisements”?: Measuring Labeling Alternatives in Internet Search Engines

In an online experiment, we measure users’ interactions with search engines, both in standard configurations and in modified versions with improved labels identifying search engine advertisements. In particular, for a random subset of users, we change “sponsored link” labels to instead read “paid advertisement.” We find that users receiving the “paid advertisement” label click 25% to 33% fewer advertisements and correctly report that they click fewer advertisements, controlling for the number of advertisements they actually click. Results are most pronounced for commercial searches, and for users with low income, low education, and little online experience.

We consider our findings particularly timely in light of Google’s change, just last week, to label many of its advertisements as “Ads.” On one view, “Ads”” is an improvement – probably easier for unsophisticated consumers to understand. Yet it’s a strikingly tiny label – the smallest text anywhere in Google’s search results, and about a quarter as many pixels as the corresponding disclosure on other search engines. As our paper points out, FTC litigation has systematically sought the label “Paid Advertisement, and we still think that’s the better choice.