Ad Classification at Right Media (teaching materials)

Edelman, Benjamin. “Ad Classification at Right Media.” Harvard Business School Case 909-032, December 2008. (Revised June 2009.) (educator access at HBP. request a courtesy copy.)

Right media considers systems and policies to make sure that ads are only shown on web sites where they are appropriate, and vice versa. Setting standards is particularly challenging given the large and growing marketplace, the numerous participants, their diverse requirements, and the dynamics of policy enforcement when market participants are competing intensely.

Teaching materials:

Ad Classification at Right Media – Teaching Note (HBP 909037)

Ad Classification at Right media – Slide Supplement (HBP 911038)

Ad Classification at Right media – Slide Supplement (widescreen) (HBP 914054)

Ad Classification at Right media – Pre-Class Slides (HBP 911037)

CPC/CPA Hybrid Bidding in a Second Price Auction

Edelman, Benjamin, and Hoan Lee. “CPC/CPA Hybrid Bidding in a Second Price Auction.” Harvard Business School Working Paper, No. 09-074, December 2008.

We develop a model of online advertising in which each advertiser chooses from multiple advertising measurement metrics–paying either for each click on its ads (CPC), or for each purchase that follows an ad-click (CPA). Our analysis extends classic auction results by allowing players to make bids using two different pricing schemes, while the driving information for bidders’ endogenous selection–the conversion rate–is hidden from the seller. We show that the advertisers with the most productive sites prefer to pay CPC, while advertisers with lower quality sites prefer to pay CPA–a result that may be viewed as counterintuitive since low quality sites cannot proudly tout their conversion rates. This result holds even if an ad platform’s assessment of site quality is correct in expectation. We also show that by offering both CPC and CPA, an ad platform can weakly increase its revenues compared to offering either alternative alone.

Hydra Media’s Pop-Up Problem — Ten Examples

Late last month, I posted an example of Vomba using a Hydra Media affiliate link to defraud VistaPrint — charging VistaPrint for traffic VistaPrint would otherwise have received for free. This was only the second Hydra Media advertising fraud example I had posted on my public web site. (The first showed similar Blockbuster fraud in spring 2007.) So some might think Hydra Media doesn’t have a big adware, spyware problem. Indeed, that’s exactly what Hydra claimed in a comment to ReveNews.

Despite Hydra’s claims of appropriate and ethical behavior, my observations indicate the contrary. Looking back to June 2007, across all my AutoTester’s browsing, my AutoTester has seen a remarkable 1,343 instances of spyware sending traffic to/through Hydra Network — 56 incidents in the past two weeks alone.

Ten Specific Examples

Using my Automatic Spyware Tester, I recently found the following Hydra Media spyware/adware incidents.

Overwrites cookies of any other affiliates previously slated to receive commission for making a referral to the advertiser.

# Date Spyware Advertiser Traffic flow Hydra ID References
1 10/1/08 Zango Survey Club Zango > Hydra > Survey Club 27352 video, packet log
2 10/2/08 Outerinfo Bidz Outerinfo > MediaTraffic > Hydra > Bidz 17203 video, packet log
3 10/4/08 Vomba Gevalia Vomba > Hydra > Gevalia 15387 video, packet log
4 10/4/08 Vomba Gevalia Vomba > Offerweb > Hydra > Gevalia 5830 video, packet log
5 10/4/08 Vomba Video Professor Vomba > Hydra > Video Professor 6102 video, packet log
6 10/11/08 Zango Gevalia Zango > Hydra > Gevalia 11427 video, packet log
7 10/11/08 Vomba Gevalia Vomba > Doubleyourctr > Hydra > Gevalia 9136 video, packet log
8 10/11/08 Vomba Reunion.com Vomba > Artur2 > Hydra > AdShuffle > Reunion 28138 video, packet log
9 10/11/08 Targetsaver Reunion.com Targetsaver > Kchuentracking > Hydra > AdShuffle > Reunion 27039 video, packet log
10 10/12/08 WhenU Omaha Steaks WhenU > MediaTraffic > Tcshoppingdeals > Hydra > Omaha Steaks 7386 video, packet log
Effects: Targets advertiser with its own affiliate link — thereby charging the advertiser for traffic it would otherwise have received for free. See extended discussion in Auditing Spyware Advertising Fraud: Wasted Spending at VistaPrint.

These are just a fraction of the Hydra incidents my AutoTester observed during the past two weeks. But as the “Effects” entry notes, each of these incidents entails charging an advertiser for traffic the advertiser would otherwise have received for free — a strikingly poor deal for the advertiser. Moreover, each of these incidents entails a distinct Hydra affiliate ID, as shown by the ten unique values in the “Hydra ID” column.

Covering Their Tracks

It is difficult to know whether Hydra and the targeted merchants were aware that these affiliates were using spyware/adware to claim commissions on traffic merchants would otherwise have received for free. In principle it is possible that the affiliates told Hydra and the merchants what they were doing — though I find that unlikely at best. But in each instance, the packet logs reflect that these affiliates’ traffic to merchants did not affirmatively indicate that the traffic came from spyware or adware. In principle such designation could be provided by “sub=” tags on affiliate links, by HTTP Referer headers, or by other indications. But these packet logs include no such disclosure.

In incidents 9 and 10, it seems these affiliates and their spyware/adware partners took additional steps to cover their tracks. In incident 9, Targetsaver invoked the affiliate’s link to LynxtTrack and onwards to Reunion.com, without an on-screen Reunion window appearing, whether as a popup, popunder, Taskbar entry, or otherwise. See the incident 9 video — showing only a brief blip at 0:37 when Internet Explorer briefly loses then regains focus. (Notice the change in color of the Internet Explorer title bar.) With no meaningful on-screen display to report what occurred, even a sophisticated tester might fail to notice that an affiliate link had been invoked and affiliate cookies had been dropped. Incident 10 also reflects significant obfuscation: WhenU opened the affiliate’s link in a window that was initially blank (0:25-0:28). WhenU then moved the window off-screen, and even when I manually clicked the window’s Taskbar entry (video at 0:33), the window did not appear. Only by right-clicking and choosing Maximize (0:38) was I able to force the window to appear in the active screen space, letting me demonstrate and confirm that the window did indeed load the Omaha Steaks site through a Hydra affiliate link.

Taking from Other Affiliates

Not only do these affiliates charge merchants for traffic merchants should have received for free, but these affliates also take commissions that should have flowed to other affiliates. Suppose an ordinary web site affiliate (“A” for short) recommends, e.g., Gevalia. If a user clicks A’s affiliate link to Gevalia, and if a user later makes a purchase from Gevalia, then A is supposed to receive a commission on the sale. But if one of these spyware/adware-using affiliates jumps in with its own link, A gets nothing.

I first demonstrated this commission-stealing in July 2004. See my proof of Zango (then “180solutions”) claiming commissions that would otherwise be paid to other affiliates, as to traffic for Crucial, Freshpair, TGW, and Valuemags. This problem remains in full effect.

Legitimate rule-following affiliates rightly disdain spyware and adware for, among other reasons, their tendency to take commissions that would otherwise flow to legitimate affiliates. For example, my VistaPrint piece last month prompted a spirited response from Linda Buquet at the 5 Star Affiliate Programs Blog (“adware also steals from Vista Print’s HONEST AFFILIATES!”) and a discussion at affiliate forum ABestWeb.

Next Steps

In a recent MediaPost article, Hydra claimed it is “complying with the instructions [it has] been given.” Perhaps a few aggressive marketers are willing to look the other way on spyware and adware issues. But all of the advertisers listed above? All these companies are happy to pay commission on traffic they would otherwise have received for free? Pay commission for placements through spyware known to arrive on users’ computers without users’ consent? It strains credibility. By posting these examples, I intend to alert the corresponding advertisers to the nature of the traffic Hydra is sending them — letting the advertisers decide for themselves whether this is a suitable allocation of their marketing budgets. As detailed in my Wasted Spending at VistaPrint piece, my firm view remains that these placements offer advertisers no bona fide benefit, and that no fully-informed advertiser would willingly pay for such traffic.

Meanwhile, others are also observing Hydra placements through spyware and adware. In a comment at ReveNews, ShareASale CEO Brian Littleton noted that he sees Hydra affiliates using spyware and adware to cover and supersede traffic his company provides to advertisers — reducing earnings of ShareASale and ShareASale’s affiliates. Brian generously offers to provide Hydra with reports of these practices, and I encouraged Brian to post his findings on the web for all to see.

Hydra’s “AdControl” service promises “positive, proactive protection” to provide “control over where [advertisers’] ad[s] [are] placed.” Hydra says it “guards against compliance problems from every angle” to assure that ad placements are “safe[,] secure [and] profitable.” Furthermore, Hydra claims to provide “tough affiliate pre-screening and policing to assure quality.” I applaud these objectives, but it seems Hydra has more to do in order to deliver the ethical, compliant, profitable placements it has promised.

CPA Advertising Fraud: Forced Clicks and Invisible Windows

At first glance, conversion-contingent advertising (cost-per-action / CPA, affiliate marketing) seems a robust way to prevent online advertising fraud. By paying partners only when a sale actually occurs, advertisers often expect to substantially eliminate fraud. After all, if commissions are only due when a user makes a purchase, what can go wrong? Unfortunately, this view is overly simplistic and, on balance, overly optimistic.

I’ve previously written at length about spyware and adware programs that watch a user’s web browsing in order to claim commission on sales that would have happened anyway. See last week’s examples of six different affiliates cheating VistaPrint through exactly this technique.

But CPA fraud does not require the use of spyware or adware on a user’s computer. To the contrary, I’ve seen plenty of CPA fraud that is entirely web based. Below I present three examples representative of this ongoing problem.

The Basic CPA Relationship

CPA advertising generally oblige an advertiser to pay a commission if three events occur:

  1. A user browses an affiliate’s web site;
  2. A user clicks a specially-coded link to a participant CPA merchant; and
  3. A user makes a purchase from that merchant.

The purchase in step 3 may occur immediately, i.e. within a single browsing session. But even if the purchase occurs shortly thereafter, e.g. a day later or even a few weeks later, a merchant will typically credit this purchase to the corresponding affiliate — on the view that the affiliate at least introduced the user to the merchant. This extended credit period is typically known as the “return-days period.”

Example 1: Couponcodesmall Forces Clicks to Drop Buy.com Cookies

The Couponcodesmall Site - Cookie-Stuffing Invisibly The Couponcodesmall Site – Cookie-Stuffing Invisibly

Some affiliates seek to bypass the user-click requirement (event 2 above) by simulating a click on an affiliate link using JavaScript. When the user merely visits the affiliate’s site, the affiliate forces the user’s browser to load an affiliate link — thereby placing affiliate cookies on the user’s PC, and claiming an affiliate commission if the user subsequently makes a purchase from the corresponding merchant.

In 2004, I presented 36 such examples in Cookie-Stuffing Targeting Major Affiliate Merchants, But the problem is ongoing.

In testing this month, I requested a page from Couponcodesmall, a top organic result for Google searches for “buy.com coupon” (without quotes). Couponcodesmall sent more than 65KB of HTML, followed by the following IFRAME:

<iframe SRC=”http://affiliate.buy.com/gateway.aspx?adid=17662&#038;aid=10389736&#038;pid=2705091&#038;sid=&#038;sURL=http%3A//www.buy.com/” WIDTH=5 HEIGHT=5 frameborder=”0″ scrolling=”no”></iframe>

I preserved a full packet log that shows this IFRAME in context. (Edit-Find on “IFRAME” to skip to the key section.) I also preserved a screen-capture video showing the cookies created after I requested this page — confirming the IFRAME‘s effect. As the HTML instructs, the IFRAME yields no visible on-screen indication — for the IFRAME‘s 5 pixel by 5 pixel size (blue highlighting) leaves too little space for the Buy.com site to be recognized.

Buy.com’s agreement with affiliates requires that affiliates comply with Commission Junction’s Publisher Service Agreement (PSA), and PSA rule 3.a grants credit only when a user “clicks through [a] Link[] to [an] Advertiser.” This affiliate’s IFRAME-delivered forced clicks exactly violate that requirement. If a user merely views this affiliate’s page, without clicking an ad or taking any other action, then this affiliate will receive a 3% to 5% commission on any purchase the user makes from Buy.com within the next 14 days, even though the user never clicked an affiliate link as required under the PSA.

I notified the affiliate program manager for Buy.com, and I gather that Buy.com is taking appropriate action.

Similar infractions remain easy to uncover. My automated testing systems typically uncover a dozen or more violations in a day of searching. I’ve also seen all manner of advances over the popups, popunders, and IMG tags I observed in 2004. For example, I now often observe cookie-stuffing using EMBED tags, OBJECT tags, HTML entity encoding, and doubly-encoded JavaScript.

Example 2: Allebrands Banner Ads Invisibly Load Affiliate Links

Other affiliates load affiliate links and drop affiliate cookies as users merely view a banner ad. From a rogue affiliate’s perspective, this attack is more effective than the attack in Example 1, for the affiliate need not get the user to visit the affiliate’s site. Instead, merely by viewing a banner ad on a third party web page, the affiliate can drop its cookies and obtain a commission on purchases users make from the targeted merchants within the return-days period.

That is, the affiliate bypasses both the user click requirement (event 2 above) as well as the browsing requirement (event 1 above). Removing this additional requirement lets the affiliate claim commission on more users’ browsing that much more easily.

To targeted merchants, this attack is importantly worse than the attack in Example 1. In particular, through this kind of attack, a merchant receives no promotional benefit whatsoever. Under this attack, merchants pay out commission only on sales that would have happened anyway — so every commission paid is entirely wasted.

I recently observed such an attack via a banner ad running on the Yahoo RightMedia Exchange. Merely by viewing an ad from Allebrands, a user’s computer was instructed to load three affiliate links, each in a 0x0 IFRAME. Below is the relevant portion of the HTML code (formatted for brevity and clarity):

GET /iframe3? …

Host: ad.yieldmanager.com

HTTP/1.1 200 OK
Date: Mon, 29 Sep 2008 05:36:02 GMT

<html><body style=”margin-left: 0%; margin-right: 0%; margin-top: 0%; margin-bottom: 0%”><script type=”text/javascript”>if (window.rm_crex_data) {rm_crex_data.push(1184615);}</script>
<iframe src=”http://allebrands.com/allebrands.jpg” width=”468″
height=”60″ scrolling=”no” border=”0″ marginwidth=”0″
style=”border:none;” frameborder=”0″></iframe></body></html>

GET /allebrands.jpg HTTP/1.1

Host: allebrands.com

HTTP/1.1 200 OK

<a href=’http://allebrands.com’ target=’new’><img src=’images/allebrands.JPG‘ border=0></a>
<iframe src =’http://click.linksynergy.com/fs-bin/click?id=Ov83T/v4Fsg&offerid=144797.10000067&type=3&subid=0′ width =’0’height = ‘0’ boder=’0′>
<iframe src =’http://www.microsoftaffiliates.net/t.aspx?kbid=9066&p=http%3a%2f%2fcontent.microsoftaffiliates.net%2fWLToolbar.aspx%2f&m=27&cid=8′ width =’0’height = ‘0’ boder=’0′>
<iframe src =’http://send.onenetworkdirect.net/z/41/CD98773′ width =’0’height = ‘0’ boder=’0′>

The three IFRAMEs (green highlighting) load three separate affiliate links in three separate windows. Because these windows are each set to be 0 pixels wide and 0 pixels tall (blue highlighting), they are all invisible.

I preserved a full packet log of the entire HTTP sequence — showing traffic flowing from the underlying Smashits web site to Right Media to Allebrands to the target affiliate programs. (Edit-Find on “allebrands” to skip to Allebrands’ code.) I also notified the targeted merchants — McAfee, Microsoft, and Symantec. They’re taking appropriate action.

Allebrands' Decoy Ad Allebrands’ Decoy Ad

Notice Allebrands’ tricky use of the misleadingly-named /allebrands.jpg URL (yellow highlighting). In particular, Allebrands instructed Right Media to send traffic to http://allebrands.com/allebrands.jpg — a .JPG extension, so seemingly an ordinary JPEG compressed image. But despite the URL’s extension, the URL actually provided ordinary HTML — creating the A HREF, IMG, and IFRAME‘s set out above. Meanwhile, if a user happened to look at this ad, the user would see only the http://allebrands.com/images/allebrands.JPG image specified by the IMG tag (pink highlighting; image shown at right). Because the IFRAMEs are invisible (blue highlighting), the IFRAMEs yield no on-screen display whatsoever.

In my testing, Allebrands distributed its rogue banner ad via a variety of web sites. One that particularly caught my eye was Smashits, a spyware-delivered banner farm which buys widespread pop-up traffic and shows voluminous ads. Beyond Smashits’ dubious traffic origins, Smashits is also notable for its placement of ads in invisible windows: Via the two-row FRAMESET presented below, Smashits creates a 0-pixel-tall “part1” frame of /audio/empty.html, which in turn ultimately displays the Allebrands ad at issue.

<FRAMESET ROWS=”0,*” FRAMEBORDER=0 FRAMEPADDING=0 FRAMESPACING=0 BORDER=0>
  <FRAME name=part1 SRC=”http://ww.smashits.com/audio/empty.html” NORESIZE MARGINWIDTH=0 MARGINHEIGHT=0 SCROLLING=”no”>
  <FRAME name=part1a SRC=”http://ww.smashits.com/spindex_02.html” NORESIZE MARGINWIDTH=0 MARGINHEIGHT=0 SCROLLING=”yes”>
</FRAMESET>

Reviewing the packet log in the context of my prior observations of Smashits’ spyware-originating traffic, the full sequence of relationships proceeds as follows: A variety of spyware sends traffic to Smashits (often via the MyGeek / AdOn Network / Mynaagencies run-of-network ad loader), and some users may affirmatively request the Smashits site. Smashits creates a 0-pixel-tall FRAME row in which to load ads off-screen. In that frame Smashits sends traffic to Traffic Marketplace, which redirects the traffic to Theadhost, which redirects it to RightMedia Exchange, which selects an ad from Allebrands, which stuffs cookies to claim commission from the three target affiliate programs.

Who is Allebrands? Allebrands’ web site offers no contact information, and Allebrands’ Whois is equally uninformative. But Allebrands’ DNS servers reside within creativeinnovationgroup.com, and Creativeinnovationgroup’s Whois references a Simon Brown at 700 Settlement Street in Cedar Park, Texas. Google Maps confirms that this is a bona fide address — seemingly a residential unit in a development.

Example 3: Avxf Stuffing Amazon and Hostgator Cookies through Signature IMG Tags in DealOfDay Forum

In Example 1, Couponcodesmall managed to lure a user to its own web site — in part through successful search engine optimization. In Example 2, Allebrands bought traffic from Right Media. In this Example 3, affiliate rogue Avxf manages to stuff cookies using others’ traffic — without paying for that traffic.

To get traffic, Avxf places images in the footer of a message it posts to a DealOfDay.com forum discussion. The associated HTML:

Originally Posted by <strong>somerset1106</strong> …

Ditto. I am still researching some other sites that are similar. If I find out any information I will keep ya posted. …

<img src=”http://www.avxf.com/img16.jpg” border=”0″ alt=”” /><img src=”http://www.avxf.com/img17.jpg” border=”0″ alt=”” />

Avxf’s footer specified two .JPG URLs, /img16.jpg and /img17.jpg — seemingly image files based on their use of the standard .JPG file extension. But in fact these URLs redirect to affiliate programs for HostGator and Amazon:

GET /img16.jpg HTTP/1.1

Host: www.avxf.com

HTTP/1.1 302 Found

Location: http://secure.hostgator.com/cgi-bin/affiliates/clickthru.cgi?id=dsplcmnt01

GET /img17.jpg HTTP/1.1

Host: www.avxf.com

HTTP/1.1 302 Found

Location: http://www.amazon.com/?Fencoding=UTF8&tag=qufrho-20

Avxf Cookie-Stuffing in DealOfDay Forum - The Resulting On-Screen Display Avxf Cookie-Stuffing:
The Resulting On-Screen Display

The resulting two pages then go on to drop affiliate cookies as usual. Thus, if a user makes a purchase at Amazon or Hostgator within their associated return-days periods, then Avxf gets paid a commission. The only on-screen indication of cookies being dropped is the two “broken image” icons shown at right — indications that something is missing, but in no way sufficient to inform a typical user (or even many advertising professionals) of what is occurring. Nonetheless, if a targeted user makes a purchase from Amazon within 24 hours of receiving Avxf’s forced click, or if a targeted user signs up with Hostgator within 30 days, then Avxf receives a commission.

I preserved a full packet log of the underlying HTML and redirects, showing Avxf’s images and redirects in context. (Edit-Find on “avxf” to skip to the code at issue.) I also preserved a screen-capture video confirming the destinations of the broken images.

Avxf’s practices violate applicable policies at Amazon and Hostgator. Amazon’s Associates program allows credit only if a customer “click[s] through” a special link (agreement 4¶1), whereas no click occurs in the example shown above. Furthermore, Amazon specifically prohibits atempts to “caus[e] any page of the Amazon Site to open in a customer’s browser other than as a result of hte customer clicking on a Special Link on [an affiliate’s] site” (agreement 4¶4). Similarly, the HostGator Affiliate Agreement prohibits the similar practice of forcing clicks through IFRAMEs (except “on pages or sites in which the other content represented on the site is related to HostGator” — an exception unavailable here, since the DealOfDay site is entirely unrelated to HostGator).

Who is Avxf? The Avxf web site offers adult content, but no mailing address on its Contact Us page. However, the site’s Whois offers a name and address: Kyle Hahn of Muncie, Indiana. Google Maps confirms the existence of the specified address, 480 W Skyway Drive.

Consequences – Winners and Losers

I see five basic consequences of these commission schemes:

  1. Fraudsters win from the bogus commission they receive, despite failing to provide merchants with a bona fide marketing benefit.
  2. Merchants pay extra commissions without getting anything in return. In particular, merchants pay commission on sales they would have made anyway. Moreover, merchants overestimate the effectiveness of their CPA marketing programs: Merchants mistakenly conclude that their CPA programs yielded sales that in fact would have happened anyway.
  3. Legitimate affiliates lose commissions that are seized by fraudsters. Whenever an ordinary affiliate was about to receive a commission, but one of these fraudsters jumps in to claim the commission instead, the first affiliate loses a commission it had fairly earned.
  4. Advertising intermediaries profit from the additional commissionable sales that purportedly occur. Affiliate networks typically charge merchants in proportion to the number (or dollar value) of commissionable sales. So every time a rogue affiliate claims commission improperly, the merchant must pay additional fees to the affiliate network.
  5. Affiliate marketing staff typically benefit, directly or indirectly, from growth in the reported size of their affiliate programs. For example, an affiliate manager might earn a bonus for rapid quarter-over-quarter growth in affiliate program size.

In principle, merchants’ losses to fraud should encourage merchants to prevent such scams. But in practice, many merchants fall victim to these attacks. Why?

For one, enforcement requires fact-intensive technical investigation — examining HTML code and packet logs to uncover infractions. The required skills have little overlap with the relationship-building and communication that otherwise drive affiliate marketing.

For some merchants and networks, mixed incentives further hinder efforts to prevent these fraudulent practices. In the short run, affiliate networks and merchants’ in-house affiliate marketing staff stand to lose from rigorous enforcement — reducing their commissionable base, reducing the size of their marketing programs, and distracting their attention from activities that more directly increase their respective short-run compensation. Thus, in the short run, both groups may perceive that they can increase their profits by deemphasizing fraud prevention.

Of course, in the long run, affiliate networks have reputations to protect. Similarly, affiliate marketing staff must consider their duties to their employers; in the long run, employers may learn about these scams and think unfavorably of marketing staff who failed to take effective action to uncover improper practices.

Large Merchants at Heightened Risk

For many cookie-stuffing attacks, large merchants are at highest risk. For example, Avxf is essentially betting that the users who read DealOfDay will subsequently go on to make purchases from Amazon. As to Amazon, that’s a safe bet, for many users buy from Amazon with remarkable regularity. But if Avxf were to target a lesser-known merchant, it would face tougher odds and lower earnings.

Thus, these random cookie-stuffing attacks (as in Examples 2 and 3) tend to target large merchants. In contrast, SEO-based attacks, as in Example 1, can prey on CPA merchants of any size.

Prevention and Response

For merchants and networks seeking to uncover and prevent these practices, I see three clear ways forward:

  • Analyze statistics already on hand . Look for unusually high click-through rates, unusually low conversion rates, blank or unexpected HTTP Referer headers, unusual HTTP User-Agent headers, long delays between clicks and sales, and other errata. But beware of affilates who manage to manipulate these statistics.
  • Provide a report / complaint page. It’s surprisingly difficult for independent affiliates, users and researchers to report fraud to many online marketers. But such reports can be extremely useful — particularly when gathered by those with a special interest in catching these scams. There’s ample evidence that affiliates enjoy reporting scams: In the ParasiteWare forum at ABestWeb, affiliates and others analyze and reveal improper marketing practices; some merchants pay bounties to anyone reporting fraud by their affiliates (1, 2).
  • Conduct hands-on testing. Browse the web looking for such scams. Run a network monitor to detect any unexpected “click” events. Or, design appropriate software to conduct such tests automatically.

Separately, merchants and networks can sensibly deter violations through tough penalties. At present, affiliates face little downside to attempting to defraud most merchants. In Deterring Online Advertising Fraud Through Optimal Payment in Arrears, I suggest a different approach — paying affiliates more slowly so that they face greater losses if they are found to be cheating. Meanwhile, some merchants have resorted to suing fraudulent affiliates. See eBay v. Digital Point Solutions (accusing affiliates of cookie stuffing through invisible code claiming unearned commissions — like the examples above) and Lands’ End v. Remy (accusing affiliates of typosquatting on Lands’ End trademarks and redirecting to Lands’ End’s LinkShare affiliate links).

More generally, merchants ought not assume infallibilityof their online marketing schemes. Certainly CPA marketing programs avoid some of the more obvious problems of pay-per-click marketing (e.g. click fraud), but CPA campaigns remain vulnerable to other kinds of abuse. Shrewd merchants should anticipate what can go wrong, and design and audit accordingly.

Auditing Spyware Advertising Fraud: Wasted Spending at VistaPrint

“VistaPrint is disciplined in operation … [VistaPrint’s] marketing [uses] highly analytically driven fact-based decision-making … [W]e manage those [marketing partners] tightly.”

– VistaPrint CEO Robert Keane in a January 2008 earnings call

For more than four years, I’ve been monitoring online advertising — alerting advertisers, ad networks, and the general public when ad spending finds its way to spyware vendors and when advertisers are getting cheated. (Examples: 1, 2, 3, 4, 5) Every day, my Automatic Spyware Tester browses the web on multiple spyware-infected PCs, watching for spyware-delivered advertising and recording its observations in videos and packet logs.

Although VistaPrint’s Robert Keane claims to effectively oversee VistaPrint’s marketing practices, I emphatically disagree. To the contrary, I’ve seen ample evidence of VistaPrint promoted by spyware and adware programs that sneak onto users’ computers without consent (including through security exploits) and through ruse and deception. In many instances, including as detailed in the examples that follow, the corresponding affiliates trick marketing analytics — claiming commission on sales that would have happened anyway, and thereby overstating the true effectiveness of their marketing efforts.

When VistaPrint is cheated by rogue marketing partners, the costs fall in the first instance to VistaPrint shareholders. Every dollar wasted on worthless advertising leaves that much less for corporate profits, and VistaPrint’s advertising budget is already strikingly large: In 2008, VistaPrint marketing consumed 31.9% of revenue (more than $125 million) while profits were just 9.9% ($39.7 million). Meanwhile, fraud against VistaPrint also harms the general public: Consumers suffer unwanted installations of spyware programs funded, in part, by theft from VistaPrint.

The following table summarizes my recent observations of fraud against VistaPrint:

Ad network Example incident Rogue VistaPrint incidents observed
August – September 2008 January – July 2008
Number of affiliates Number of dates Number of observations Number of observations
Lynxtrack Vomba, Hydra Network Affiliate 19934 6 13 18 32
Clickbooth Vomba, Clickbooth Affiliate 14941
WhenU, MediaTraffic, Iadsdirect, Clickbooth Affiliate 7781
5 13 14 14
CPA Builder (including traffic from Revenue Gateway, from OptInRealBig / CPAEmpire, and from XY7) Zango, Revenue Gateway Affiliate 12489, CPA Empire, CPA Builder 2 8 9 21
CX Digital Media (Incentaclick) Vomba, Weclub, CX Digital Media Affiliate 13736 2 2 2 18
Performics (Google) Deluxe Communications, Smartyseek, Performics 1 5 5 5
direct relationships & other networks
not yet tabulated in full – some examples on file

During August-September 2008, my AutoTester repeatedly observed VistaPrint facing rogue traffic coming from five different ad networks. In the sections that follow, this piece presents an example of fraud by an affiliate from each of the specified networks. But I’ve seen plenty more. My AutoTester has been running for more than a year — preserving tens of thousands of records of online advertising fraud, including 133 other spyware incidents arising out of traffic to VistaPrint. These many incidents confirm the breadth of improper practices by VistaPrint’s marketing partners.

Example 1: Vomba, Hydra Network Affiliate 19934 Claiming Commission on VistaPrint’s Organic/Type-In Traffic

Vomba, Lynxtrack Affiliate 19334 Targeting VistaPrintVomba, Hydra Network Affiliate 19334 Targeting VistaPrint

In testing on September 12, my AutoTester browsed VistaPrint’s site on a computer with Vomba (from Integrated Search Technologies, makers of Slotchbar, XXXtoolbar, WhenU, AdVantage, and more). Vomba popped open a window that sent traffic to Hydra Network (LynxTrack) (affiliate 19934), and Hydra Network in turn forwarded the traffic back to VistaPrint. The result was the screen shown at right — the original VistaPrint window at left/back, with a new popup at front/right.

Crucially, both web browser windows share a single set of cookies. Whether the user buys from the original VistaPrint window or from the popup, cookies tell VistaPrint that this Hydra Network affiliate caused the sale. So VistaPrint will pay this affiliate a commission — even though, in fact, the affiliate did nothing whatsoever to facilitate the sale. I call this tactic “self-targeting” — reflecting that Vomba covers VistaPrint with its own ad. All of the examples presented on this page entail spyware/adware performing this kind of self-targeting attack.

My AutoTester preserved a video of this incident and a packet log of the underlying network traffic.

My AutoTester observed this same affiliate using the same method on three different dates in August-September 2008. My AutoTester also observed five other Hydra Network affiliates similarly defrauding VistaPrint. All told, in August-September, my AutoTester observed 18 such incidents on 13 distinct dates.

My AutoTester’s records indicate that Hydra Network receives substantial spyware-originating traffic. Looking back to June 2007, across all my AutoTester’s browsing, my AutoTester has seen a remarkable 1,287 instances of spyware sending traffic to/through Hydra Network.

Example 2: Vomba, Clickbooth Affiliate 14941 Claiming Commission on VistaPrint’s Organic/Type-In Traffic

In testing on September 12, my AutoTester browsed VistaPrint’s site, again on a computer with Vomba. Vomba popped open a window that sent traffic to Clickbooth (affiliate 14941), and Clickbooth in turn forwarded the traffic back to VistaPrint.

Because both web browser windows share a single set of cookies, this Clickbooth affiliate gets paid a commission whether the user buys from the original VistaPrint window or from the popup. This commission gets paid even though, in fact, the affiliate did nothing whatsoever to facilitate the sale.

My AutoTester preserved a video of this incident and a packet log of the underlying network traffic.

My AutoTester observed this same affiliate using the same tactics on eight different dates in August-September 2008. My AutoTester also observed three other Clickbooth affiliates similarly defrauding VistaPrint. All told, my AutoTester observed 13 such incidents on 12 distinct dates.

My AutoTester’s records indicate that Clickbooth receives substantial spyware-originating traffic. Looking back to June 2007, across all my AutoTester’s browsing, my AutoTester has seen 917 instances of spyware sending traffic to/through Clickbooth.

Example 3: WhenU, MediaTraffic, Iadsdirect, Clickbooth Affiliate 7781 Claiming Commission on VistaPrint’s Organic/Type-In Traffic

In manual testing on September 28, I browsed VistaPrint’s on a computer with WhenU. WhenU opened a popunder that flashed briefly on screen (video at 0:15) but then forced itself to an off-screen location where I could not see it even if I minimize other windows. (See video at 0:24 to 0:30, when I attempted to find the popunder.) By manually right-clicking and choosing “maximize,” I managed to make the popunder visible — confirming that it loaded VistaPrint and noting the affiliate ID number.

Packet log analysis reveals that traffic flowed from WhenU to MediaTraffic (a pay-per-view advertising marketplace also operated by Integrated Search Technologies) to Iadsdirect to Clickbooth (affiliate 7781) to VistaPrint.

As in prior examples, both windows share a single set of cookies. Thus, the WhenU popunder causes the corresponding affiliate to receive a commission if the user makes a purchase — even though the affiliate did nothing to encourage or facilitate a purchase.

I preserved a video of this incident and a packet log of the underlying network traffic.

This advertising fraud by WhenU is particularly notable because WhenU previously claimed to have reformed all unsavory practices. (See e.g. “WhenU CEO Bill Day Cleans House.”) Moreover, WhenU previously touted a TRUSTe Trusted Download certification, and TRUSTe specifically prohibits Trusted Download programs from defrauding advertisers. (See Certification Agreement, Schedule A (“Program Requirements”), provision 14.k.) That said, WhenU has silently left the Trusted Download whitelist. Furthermore, in separate testing of WhenU software, I have recently seen repeated self-targeting fraud improperly claiming commissions from a variety of advertisers.

Example 4: Zango, Revenue Gateway Affiliate 12489, CPA Empire, CPA Builder Claiming Commission on VistaPrint’s Organic/Type-In Traffic

VistaPrint
money viewers
   CPA Builder    
money viewers
   CPA Empire    
money viewers
   Revenue Gateway    
money viewers
Zango

The Money Trail and Traffic Flow

In testing on September 21, my AutoTester browsed VistaPrint’s site on a computer with Zango. Zango popped open a window that sent traffic to Revenue Gateway (affiliate 12489), which redirected to CPA Empire (formerly OptInRealBig), which redirected to CPA Builder, which in turn forwarded the traffic back to VistaPrint.

The chain of intermediaries adds additional complexity to the relationships. But traffic flows in a continuous forward path: From Zango to Revenue Gateway to CPA Empire to CPA Builder and finally back to VistaPrint. Conversely, revenue flows in the opposite direction: From VistaPrint to CPA Builder to CPA Empire to Revenue Gateway to Revenue Gateway affiliate 13425 to Zango. The diagram at right summarizes the flows of traffic and money.

My AutoTester preserved a video of this incident and a packet log of the underlying network traffic.

During August-September 2008, my AutoTester also observed other incidents wherein spyware waited for a user to browse the VistaPrint site, then sent the user back to VistaPrint via CPA Builder. Beyond this Zango / Revenue Gateway / CPA Empire example, I also observed incidents wherein CPA Empire’s relationship with XY7 was the source of the tainted traffic. All told, my AutoTester has preserved more than 600 incidents of spyware sending traffic to/through CPA Empire, as well as at least 24 incidents of spyware sending traffic to/through Revenue Gateway (though I have reason to believe that some Revenue Gateway incidents were not preserved).

Example 5: 8/17/08 – Vomba, Weclub, CX Digital Media (Incentaclick) Affiliate 13736 Claiming Commission on VistaPrint’s Organic/Type-In Traffic

Vomba, Weclub, CX Digital Media Affiliate 13736 Targeting VistaPrint Vomba, Weclub, CX Digital Media Affiliate 13736 Targeting VistaPrint

In testing on August 17, my AutoTester browsed VistaPrint’s site on a computer with Vomba. Vomba popped open a window that sent traffic to Weclub, which immediately redirected to CX Digital Media (Incentaclick), which in turn forwarded the traffic back to VistaPrint.

See the screenshot at right. My AutoTester preserved a video of this incident and a packet log of the underlying network traffic.

During August-September 2008, my AutoTester also observed another CX Digital Media affiliate using spyware to claim commission on VistaPrint’s organic traffic. All told, my AutoTester has preserved more than 200 different incidents of spyware sending traffic to/through CX Digital Media.

Example 6: Deluxe Communications, Smartyseek, Performics Claiming Commission on VistaPrint’s Organic/Type-In Traffic

In testing on September 14, my AutoTester browsed VistaPrint’s site on a computer Deluxe Communications (which I have repeatedly observed installed through security exploits and otherwise without user consent). Deluxe Communication popped open a window that sent traffic to Smartyseek, which immediately redirected to Performics, then back to VistaPrint.

In typical Deluxe Communications fashion, the popup window entirely covered the window the user had been browsing. But because both windows showed VistaPrint, some users might not notice.

My AutoTester preserved a video of this incident and a packet log of the underlying network traffic.

My AutoTester observed this same affiliate using the same tactics on five different dates in August-September 2008, and my AutoTester also observed Performics traffic during VistaPrint browsing on five other (prior) occasions.

Responsibility and Causation

It’s easy to present VistaPrint as perpetrator: VistaPrint fails to adequately oversee its marketing partners. As a result, VistaPrint’s advertising spending helps fund spyware and adware programs that sneak onto users’ PCs, with serious harms to performance, reliability, and privacy.

But I also see an important sense in which VistaPrint is a victim: VistaPrint’s marketing partners are defrauding VistaPrint by claiming commissions on sales they actually did nothing to cause. Such commissions are entirely wasted, yielding no bona fide marketing benefit to VistaPrint.

By all indications, VistaPrint faces significant difficulties in supervising its marketing partners. Yet other major retailers handle such challenges with greater success. For example, it is comparatively rare to see spyware or adware promoting, defrauding, or attempting to defraud Amazon — even though Amazon spends nearly three times as much on marketing as VistaPrint ($344 million to $125 million).

What could VistaPrint do differently? For one, I question VistaPrint’s choice of marketing partners: As the preceding statistics indicate, I have repeatedly and widely seen spyware and adware sending traffic to many of the partners VistaPrint works with. VistaPrint might face less fraud if it favored marketing partners with a track record of successful supervision of their affiliates.

More generally, an affiliate currently faces little real downside to attempting to defraud VistaPrint. If an affiliate gets caught cheating, VistaPrint will terminate that affiliate, but I see little indication that VistaPrint exacts any meaningful penalty to make the affiliate (or the network providing that affiliate) regret its transgression. In Deterring Online Advertising Fraud Through Optimal Payment in Arrears, I suggest a different approach — paying affiliates more slowly so that they face greater losses if they are found to be cheating. Alternatively, VistaPrint might sue affiliates it learns are cheaters, as in eBay v. Digital Point Solutions and Lands’ End v. Remy.

Yet Keane’s remarks (“highly analytically driven fact-based decision-making”) reveal that VistaPrint is at least attempting to supervise its marketing partners to optimize its spending. How, then, could VistaPrint end up facing so much fraud? I suspect VistaPrint’s analytics actually lead the company astray. Consider the tactics presented above, from the perspective of the information easily available to VistaPrint’s marketing staff. Because these affiliates target users who are already interested in VistaPrint, the affiliates’ conversion rates are likely to be well above average. Moreover, because these affiliates incur limited costs, they can accept payments far below what Google might require. Thus, VistaPrint’s staff are likely to assess these affiliates favorably — without realizing that the traffic at issue is traffic VistaPrint would otherwise have gotten for free. Put differently: Although VistaPrint’s measurements may be very precise, they’re inaccurate because VistaPrint misunderstands the sources of affiliates’ traffic.

In attempting to prevent such fraud, VistaPrint should also examine its ad networks’ incentives. Ad networks often mark up affiliates’ fees: For every dollar VistaPrint is slated to pay to a given affiliate, that affiliate’s network takes another (say) $0.20. As a result, ad networks have a clear incentive to tolerate rogue affiliates: Networks make money from each sale credited to an affiliate, so ejecting rogue affiliates would directly reduce the network’s earnings.

The Big Picture

Spyware-based advertising fraud extends far beyond VistaPrint. Most merchants operating affiliate, CPA, or other conversion-contingent programs face similar fraud. But VistaPrint is a large and, purportedly, sophisticated advertiser. So VistaPrint could appropriately lead by example.

I’m overdue to present further examples of spyware and adware continuing to defraud major merchants. Historically my articles have tended to emphasize the largest US affiliate networks — Commission Junction, LinkShare, Performics. But there’s plenty of fraud through smaller networks too, as well as through networks based outside the US. I’ll present additional examples later this fall.

In January, an Anti-Spyware Coalition workshop asked “Is adware dead?” Some panelists responded substantially in the affirmative. But my AutoTester indicates otherwise. I’m pleased to see that big advertisers no longer advertise directly with major adware vendors. Yet a chain of indirection — adware sending traffic to one ad network, which forwards to another, then finally to an advertiser — continues to promote top brands. Furthermore, spyware-delivered banner farms and ad-loaders are becoming increasingly widespread. This month I saw adware still promoting American Express, Apple, and AT&T — to name just a few of the A’s. There’s plenty of work left to be done.

Typosquatting: Unintended Adventures in Browsing

Edelman, Benjamin. “Typosquatting: Unintended Adventures in Browsing.” Cybercrime Gets Personal, McAfee Security Journal (fall 2008): 34-37.

Typosquatting is the practice of registering domain names, identical to or confusingly similar to trademarks and famous names, in hopes that users will accidentally request these sites–whereupon they will receive, typically, advertisements. This piece presents the basic typosquatting business model, based on my analysis of more than 80,000 typosquatting domain names. I analyze the advertising intermediaries that make typosquatting profitable, and I assess the legislation and litigation that are beginning to put a check on this practice.

Consumer Payment Systems — Japan (teaching materials) with Andrei Hagiu

Edelman, Benjamin, and Andrei Hagiu. “Consumer Payment Systems — Japan.” Harvard Business School Case 909-007, August 2008. (Revised May 2009.) (educator access at HBP. request a courtesy copy.)

In 2008, the Japanese consumer payments landscape featured ongoing widespread use of cash, limited use of credit cards and rapid rise of e-money systems based on contactless technology embedded in cards and especially mobile phones. The case details the alliances that created new products, as well as the regulations that sometimes stood in the way. Throughout, the case identifies incentives for both consumers and merchants, including direct costs, efficiency benefits, rebates, and treatment in case of loss or fraud.

Teaching Materials:

Consumer Payment Systems — United States and Japan – Teaching Note (HBP 909039)

Consumer Payment Systems — United States (teaching materials) with Andrei Hagiu

Edelman, Benjamin, and Andrei Hagiu. “Consumer Payment Systems — United States.” Harvard Business School Case 909-006, August 2008. (Revised July 2011.) (educator access at HBP. request a courtesy copy.)

In 2008, the U.S. consumer payments landscape was characterized by the ongoing prevalence of credit and debit card networks, the decline of checks, the rise of stored value cards, and the growth of new payment methods such as PayPal, Bill Me Later, and decoupled debit. This case presents the structure of these payment methods, focusing on incentives for both consumers and merchants, including direct costs, efficiency benefits, rebates, and treatment in case of loss or fraud.

Teaching Materials:

Consumer Payment Systems — United States and Japan – Teaching Note (HBP 909039)

Competition among Sponsored Search Services

Edelman, Benjamin. “Competition among Sponsored Search Services.” U.S. House of Representatives, Committee on the Judiciary, Task Force on Competition Policy and Antitrust Laws, 2008. (Hearing cancelled.) (Reprinted in Working Knowledge: Google-Yahoo Ad Deal is Bad for Online Advertising.)

Last month I was asked to testify to the United States House of Representatives Committee on the Judiciary Task Force on Competition Policy and Antitrust Laws about competition among paid search providers, particularly the proposed Google-Yahoo partnership.

At the last minute, the hearing was cancelled, and I won’t be able to testify at the rescheduled session. Rather than let my draft written statement languish, I’m taking this opportunity to post the prepared testimony I had planned to offer:

Competition among Sponsored Search Services.

PPC Platform Competition and Google’s "May Not Copy" Restriction

By all indications, Google AdWords features far more advertisers than competing PPC platforms such as Yahoo Search Marketing and Microsoft adCenter. (Consider: Search for "thinkpad x60 power supply" at Google, and there are six relevant ads from vendors who actually sell that product. Search at Yahoo Search or Microsoft Live Search, and there are various ads from indexers and aggregators, but only one or two from vendors directly selling the product. Other searches for offbeat, unusual or region-specific keywords show similar patterns.)

Why do more advertisers choose Google? Because more users search at Google, Google can offer wider ad distribution than any single competitor. So if an advertiser had to choose just one ad platform, Google would be the natural choice.

But in principle advertisers can easily use multiple ad platforms. Ads are trivially small plaintext data, and In principle ads can be copied from platform to platform without restriction. So why don’t more Google advertisers use Yahoo, adCenter, and others too?

One possible answer comes from a little-noticed Google AdWords API Terms & Conditions restriction which substantially hinders advertisers’ efforts to use multiple providers — exactly prohibiting software vendors from helping advertisers copy AdWords campaigns to competing platforms. In this article, I identify the restriction at issue, analyze its effects on advertisers and competing ad platforms, critique response from Google, and compare this restriction with Google’s commitment to “data portability” in other contexts.

The Restriction at Issue

To use the Google AdWords API, a software developer must accept Google’s AdWords API Terms & Conditions. The T&C’s include the following requirement:

"Any information collected from an input field used to collect AdWords API Campaign Management Data may be used only to manage and report on AdWords accounts. Similarly, any information or data used as AdWords API Campaign Management Data must have been collected from an input field used only to collect AdWords API Campaign Management Data. For example, the AdWords API Client may not offer a functionality that copies data from a non-AdWords account into an AdWords account or from an AdWords account to a non-AdWords account." (emphasis added)

Sure enough, searching the web for commercial tools to synchronize PPC campaigns or to export data from Google to competing platforms, I found none.

The "May Not … Cop[y] Data" Prohibition: Effect on Advertisers

The quoted restriction prevents advertisers from easily exporting ads from Google to a competing paid search provider. Consider: The essence of an export procedure is to copy data from an AdWords account to a non-AdWords account — exactly what the restriction prohibits.

Indeed, available export procedures are strikingly complex. For example, to import a Google AdWords campaign into Microsoft adCenter, Microsoft offers a 17-step procedure (with some steps made more complicated by the presence of multiple sub-steps).

Microsoft’s procedure is necessarily convoluted because Google’s "may not … cop[y]" restriction prevents Microsoft, or any other vendor, from writing a tool that connects to the Google API, downloads an advertiser’s ads, and uploads those ads directly to, e.g., Microsoft adCenter. Instead, advertisers must download data manually, reformat it to match adCenter’s requirements, and upload it to Microsoft — just as Microsoft’s lengthy procedure specifies.

For many advertisers, Google’s restrictions on data export impose an ongoing burden even beyond the advertiser’s initial signup with a competing PPC provider. Consider an advertiser that changes its ads or keywords often — perhaps selling seasonal merchandise, or experimenting with alternative advertising strategies. Such an advertiser would typically prefer to make changes in one place, and have the changes automatically propagate to all the advertiser’s PPC platforms. If Google remains the advertiser’s primary PPC provider, the advertiser would probably want to make changes in Google’s interface, then have other PPC platforms optionally automatically copy those changes. But Google’s "may not … cop[y]" restriction applies equally to ongoing resynchronizations. If an advertiser made daily changes to its Google campaigns, it would have to daily repeat the manual export/import process — a task that would be both time-consuming and prone to error.

In short, the net effect of the quoted restriction is to reinforce the tendency of small to medium-sized advertisers to "single-home" — to use only Google AdWords, to the exclusion of competing platforms.

At their peril do advertisers rely solely on Google: If advertisers get stuck using only Google, for lack of any easy and efficient way to buy some traffic elsewhere, Google can charge prices higher than competing platforms. Of course Google can’t raise prices infinitely; at some point, advertisers would overcome the lock-in, accept manual export, and copy ads to competitors. But Google’s "may not copy" restriction increases the costs of multi-homing — letting Google charge that much more than competitors, without advertisers facing compelling incentives to look elsewhere.

The "May Not … Cop[y] Data" Prohibition: Effect on Competing Ad Platforms, on Publishers, and on Users

By encouraging small to medium-sized advertisers to advertise only with Google AdWords, Google’s API restriction reduces the number of advertisers using competing ad platforms. This harms competing platforms in two distinct ways. First, it reduces competitors’ coverage — preventing competitors from featuring relevant ads that pertain to obscure user searches. (Consider the power supply example from the first paragraph of this piece — better and more useful ads at Google.) With fewer relevant ads, the competing platform offers users an inferior service — inviting users to look elsewhere, and reducing the likelihood of a paid click that would earn the platform an advertising fee.

Second, by reducing the number of advertisers bidding for advertising positions at other platforms, the quoted provision dramatically reduces revenue at those platforms. My December 2006 Optimal Auction Design in a Multi-unit Environment estimates the revenue benefits of additional advertisers based on publicly-available data and estimates of market fundamentals. The intuition is straightforward: When many advertisers seek positions for a given search term, they must bid higher to avoid being outbid and receiving inferior listing position. Conversely, when only a few advertisers seek positions, prices can be strikingly low since even a low bid earns a prominent position.

Google’s API restriction also reduces the value of advertising inventory held by third-party publishers. Consider a publisher seeking to sell its sponsored search or other text ad inventory to a provider of sponsored search services. In general, Google can afford to pay more because Google’s revenue per search is higher than competitors’. But how much will Google offer? Google maximizes profits by narrowly outbidding competitors; anything higher is waste. So the weaker competitors become, the lower Google can bid — and the less revenue publishers receive for the traffic they sell. Google’s "may not copy" API restriction serves a role in weakening competing platforms — keeping advertisers using Google alone, and hence reducing competing ad platforms’ ability to pay for publishers’ inventory.

End users also suffer from Google’s restriction on copying ads. Were it not for Google’s restriction, more advertisers would sign up to use competing ad platforms — increasing the usefulness of Yahoo Search and Microsoft Live Search for the users who choose those services.

Google’s Response

I forwarded these concerns to Google in March, and I managed to get in touch with Doug Raymond, product manager for AdWords API. Doug offered three rationales for the restriction. The list below summarizes his arguments (black) and my responses (blue).

  • Google: The quoted provision only applies to third-party developers. Individual advertisers remain free to write software that exports their Google campaigns.
    • Small to medium-sized advertisers don’t want to be developers. Rather, they want to use code that others write. That’s exactly why the AdWords API offers a concept of developers, rather than requiring that every advertiser write its own code.
    • As a leading provider of centralized computing services, as distinguished from small programs individual users build themselves, Google well knows the benefits of rigorous design by competent professionals.
  • Google: Advertisers can extract their data in other ways, e.g. a comma-separated-value (CSV) file.
    • Manual export is convoluted, slow, and error-prone. API-based access would be faster, easier, and more reliable.
    • The existence of an inferior alternative does not justify imposing restrictions that prohibit superior implementations.
    • In other contexts (detailed below), Google has made strong requests for, and commitments to, data portability.
    • In other contexts, Google emphasizes the benefits of streamlined, automated data transfer — never viewing convoluted manual procedures as an acceptable alternative.
  • Google: Third-party developers ought not have free access to advertiser data.
    • Google’s AdWords API already offers an appropriate security model to limit developers to serving those AdWords advertisers that have specifically granted such permission. In short, a developer needs a password to access an advertiser’s account.

Google’s Position on Data Portability in Other Contexts

Google’s prohibition on AdWords API data export stands in sharp contrast to Google’s position on data portability in other contexts. Indeed, Google has previously taken a firm position in favor of data portability. Some specific examples: In a November 2006 interview at the Web 2.0 Summit, Schmidt specifically promised that "We [Google] would never trap user data." Schmidt added that "If users can switch it keeps us honest." Just last month, Google CEO Eric Schmidt called for open access to (and indexing of) social network data — telling IBM’s Business Partner Leadership Conference "People should be able to move from place to place, and their data is available everywhere" and "open is best for the consumer." Well-known Google blogger Matt Cutts summarized Google’s commitment to data openness with the catchy title "Not trapping users’ data = GOOD" and a long list of Google products that support data export.

I credit that Schmidt’s statements refer to other kinds of data — search engines’ records of users’ search history, and a wide assortment of data held by social networks. But the same principles plainly apply to access to search ads: Just as consumers benefit from being able to move their data as they see fit, so too do advertisers benefit from flexibility.

Moreover, it strains credibility for Google to ask social networks to share their data with Google, while Google simultaneously imposes contractual roadblocks preventing others from accessing Google data.

Next Steps and Google’s Other Restrictions

Google already faces antitrust scrutiny for its striking growth and market share. In that context, it’s particularly hard to defend the restriction at issue — a barrier to competition, without any apparent pro-competitive purpose. Regulators might reasonably require that Google remove the quoted provision — letting third-party developers export and synchronize AdWords data if advertisers so desire. This would be a trivially straightforward requirement — just a sentence to be stricken from Google’s AdWords API T&C’s. Because Google’s existing APIs already provide the required data, Google would not need to add any new code or any new API functions.

Other AdWords API restrictions also deserve scrutiny. For example, Google insists that advertising tools collect AdWords instructions through separate fields not used for other ad platforms — blocking simplification via a single interface to streamline advertisers’ decisions. Google prohibits advertising tools from storing Google data in a single relational database along with data for other ad platforms — increasing the complexity of designing a system to manage campaigns on multiple platforms. And Google prohibits reports that compare Google ad performance data (e.g. costs and profits from advertising at Google) with data from other ad platforms — hindering advertisers’ efforts to evaluate competitors’ offerings. I gather Google defends these restrictions on the grounds that they purportedly prevent advertiser confusion. Perhaps — but their more obvious effect is to increase the costs and complexity of using competing ad platforms. Perhaps I’ll consider these restrictions in greater detail in a future article.

Meanwhile, I’m struck by Google’s calls for data portability in other contexts. With Google’s ongoing request that other companies provide data to Google, perhaps Google will return the favor by abandoning its "may not copy" restriction — ideally promptly and unilaterally, without requiring that regulators force Google’s hand.