Category: trust

Trust and trustworthiness

Posted on

Adverse Selection in Online ‘Trust’ Certifications

Edelman, Benjamin. “Adverse Selection in Online ‘Trust’ Certifications.” Proceedings of the International Conference on Electronic Commerce (2009): 205-212. (ACM International Conference Proceeding Series.)

Widely used online “trust” authorities issue certifications without substantial verification of recipients’ actual trustworthiness. This lax approach gives rise to adverse selection: the sites that seek and obtain trust certifications are actually less trustworthy than others. Using a new dataset on web site safety, I demonstrate that sites certified by the best-known authority, TRUSTe, are more than twice as likely to be untrustworthy as uncertified sites. This difference remains statistically and economically significant when restricted to “complex” commercial sites. In contrast, competing certification system BBBOnline imposes somewhat stricter requirements and appears to provide a certification of positive, albeit limited, value.

Posted on

Coupons.com and TRUSTe: Lots of Talk, Too Little Action updated March 20, 2008

Six and a half months ago, I reported a variety of bad practices at Coupons.com. Key among my concerns: Coupons.com stored data in deceptive filenames and registry entries designed to look like part of Windows — with names like c:\WINDOWS\WindowsShellOld.Manifest.1 and HKEY_LOCAL_MACHINE\SOFTWARE\Microsoft\Windows\CurrentVersion\Controls Folder\Presentation Style. Furthermore, Coupons.com failed to remove these files upon a user’s specific request to uninstall.

Because Coupons.com was certified by TRUSTe Trusted Download, I reported these behaviors through TRUSTe’s Watchdog form. TRUSTe investigated and, it claimed, required Coupons.com to make changes. Last month, TRUSTe declared success: “Coupons, Inc. rolled out a number of significant changes …. To improve registry key and naming (s.i.c.), the new version of the software uses an improved security scheme that writes only one registry key placed in a typical location, named in an appropriate manner.” TRUSTe concluded by giving itself a pat on the back — calling this sequence “an excellent outcome” in that “[a] user found a problem, filed a complaint, and TRUSTe worked with the Participant to make necessary corrections.”

I wanted to see for myself whether TRUSTe’s oversight is as effective as TRUSTe claims. So I downloaded Coupons.com’s current software onto an ordinary computer in my lab. (I couldn’t use a VMware virtual machine because Coupons.com detects VMware and refuses to install.) To my dismay, Coupons.com’s software continued to create the same deceptively-named files and registry keys I reported in August:

c:\WINDOWS\uccspecc.sys
c:\WINDOWS\WindowsShellOld.Manifest.1
HKEY_CLASSES_ROOTManifest.Template.1shellex
HKEY_LOCAL_MACHINE\SOFTWARE\Microsoft\Windows\CurrentVersion\uccspecc
HKEY_LOCAL_MACHINE\SOFTWARE\Microsoft\Windows\CurrentVersion\Controls Folder\Presentation Style
HKEY_LOCAL_MACHINE\SOFTWARE\Microsoft\Windows\CurrentVersion\Explorer\EnableAutoTrayHistory
HKEY_LOCAL_MACHINE\SOFTWARE\Microsoft\Windows\CurrentVersion\Internet Settings\UrlDecoding

I prepared a screen-capture video to confirm and memorialize the deceptively-named files and registry keys. (My video begins by showing the New York Times front page, to demonstrate the date of testing.)

I then used Control Panel – Add/Remove Programs to attempt to uninstall Coupons.com’s software. I found that the specified files and registry keys all remained in place — even though TRUSTe further promised that “[t]he new version uninstaller removes the files.”

What’s going on? Maybe TRUSTe tested a different version of Coupons.com’s software than the version offered to the public. Maybe Coupons.com posted the wrong file. But whatever the reason, TRUSTe’s claims are inconsistent with my test results.

TRUSTe’s Oversight and What to Do Next

My testing indicates that Coupons.com has not made the changes TRUSTe specified. In particular, Coupons.com continues to use multiple registry keys and filenames with intentionally deceptive locations and names — exactly contrary to TRUSTe’s claim that “only one registry key” is used and that it is placed in a “typical location” with an “appropriate” name. Furthermore, Coupons.com leaves these files and registry keys after uninstall — exactly contrary to TRUSTe’s claim that the new uninstaller “removes the files left behind.”

Far from TRUSTe’s self-congratulatory rhetoric, Coupons.com’s practices reflect badly on TRUSTe: Despite clear violations widely reported 6+ months ago and a supposed investigation by TRUSTe, the problems continue to this day.

Worse, through two different channels, TRUSTe has falsely told users they can trust Coupons.com. First, Coupons.com has continuously remained on TRUSTe’s Trusted Download “whitelist” despite my initial report. That is, TRUSTe continued to certify Coupons.com even when TRUSTe knew of Coupons.com’s deceptive practices and even when there was no dispute that the practices were ongoing. A better strategy, per my September 2007 recommendation, would be to suspend violators until they have fully corrected their practices. Otherwise, a user looking at the “whitelist” cannot know which companies are truly in good standing, versus which have fallen short and are must make improvements.

Second, TRUSTe has posted announcements (1, 2) that falsely characterize the status of Coupons.com’s improvements: In September TRUSTe promised the changes would be “completed within 90 days” — but in fact, they’re still not in place 180 days later. In February TRUSTe proclaimed the changes complete — but in fact Coupons.com’s software still has the same problems I previously identified.

These failings go to the core of TRUSTe’s promise to “make privacy your choice.” TRUSTe claims to be giving users the information they need to make informed decisions. However, TRUSTe’s information is systematically in error — to the benefit of the companies that pay TRUSTe to get certified, but to the detriment of any users who mistakenly rely on TRUSTe’s investigations.

An Additional Violation: Executable Software Left Behind After Uninstall

My recent tests also revealed a new file I hadn’t noticed in prior tests: c:\WINDOWS\system32\cpnprt2.cid. How did I miss this file? It appears only after a user first prints a coupon — not when a user initially installs Coupons.com software. So this file wasn’t created in my prior testing.

Despite the file’s unusual .CID extension, the file is actually a DLL containing executable code. Although “cpnprt” bears some relationship to Coupons.com’s product name (“CouPoN PRinTer”), I can see no proper reason to place this file within c:\WINDOWS\ rather than in c:Program Files\Coupons with Coupons.com’s other files. So Coupons.com’s improper file locations include not only data files (like those listed above), but also executable code.

Moreover, I see no proper reason for calling the file a .CID rather than the DLL that it is. This misnaming serves to further obfuscate the file’s purpose and to prevent typical users from determining that the file contains executable software code.

In separate testing, I confirmed that this file remains on a user’s computer even after the user removes Coupons.com’s software. (This too is shown in my screen-capture video.) So Coupons.com leaves behind not just data, but also executable software. Leaving executable code stands starkly in contrast to Coupons.com’s license agreement which mentions only that “license keys wil not be removed when the Software is uninstalled” — but says nothing about software code left behind.

Coupons.com violates TRUSTe Trusted Download requirements when it leaves executable code after a user’s uninstall request. Trusted Download rule 7.(a)(ii) requires a complete uninstall and allows only limited exceptions — none of them applicable here. (The closest exception allows “properly disclosed anti-fraud … measures” — but this practice is not “properly disclosed,” nor is surviving executable code required to track whatever practices might conceivably be at issue.) Coupons.com’s cpnprt2.cid file therefore constitues another violation of applicable Trusted Download rules.

Coupons.com’s Ongoing DMCA Litigation with John Stottlemire

Last summer I mentioned Coupons.com’s misguided DMCA litigation against John Stottlemire. The case drags on: John’s blog reports ongoing events, including John’s motion to dismiss, the court’s granting of that motion, Coupons.com’s second amended complaint, and John’s second motion to dismiss.

My view remains that this litigation is ill-advised for Coupons.com: Coupons.com has too much work to do, improving its own software and its own business practices, to waste management time and attention on pursuing a user who merely helped others remove deceptively-named files and registry keys. Coupons.com has nothing to gain here: Even if Coupons.com can force John to stop telling users how to remove unwanted Coupons.com software, others will immediately pick up where John left off.

There’s plenty more to be said about the case — especially, concern at using the DMCA to stifle useful public-interest discussion of how to remove unwanted software from an ailing computer. But I’ll leave that to others: TechDirt, Wired, and various bloggers.

Update (March 20, 2008)

TRUSTe’s Response and My Hands-On Testing

In a March 19 posting, TRUSTe claims that the issues described above reflected Coupons.com software available only between March 15 and March 17. But TRUSTe stands behind its February report that Coupons.com had “addressed [the] concerns” TRUSTe previously raised based on my prior article. I emphatically disagree. In particular, my hands-on testing, memorialized in video records, clearly demonstrates that Coupons.com continues to violate TRUSTe’s prior instructions and applicable TRUSTe rules. Consider my March 19 video:

1. At 0:02, I demonstrate the current date and time. I then run an InCtrl scan to record existing files and registry keys.

2. At 1:15, I begin to browse the Coupons.com site, and at 1:25 I attempt to print a coupon. 

3. At 1:33, I begin to install the Coupon Printer program, including providing a name and email address when requested (2:20). 

4. At 2:55, I browse c:\WINDOWS\ to show the newly-created and deceptively-named CID file (as discussed above).  I then proceed to find a file by the same name placed in c:\WINDOWS\system32 also.

5. At 3:30, I rerun Inctrl to identify newly created files and registry keys.  The results are visible beginning at 5:35.  I notice the HKEY_CLASSES_ROOT\English.cpl registry key in the listing (5:45), and at 5:50 I use Regedit to confirm that the key is indeed present. 

6. At 6:30, I request an uninstall in the usual way (Control Panel – Add or Remove Programs).  I then show that deceptively named file remains in c:\WINDOWS\ (7:14) and c:\WINDOWS\system32 (7:08); despite my uninstall request, these files were not removed.  I show that the deceptively-named registry key remains also (7:02). 

The Violations Revealed by My Hands-On Testing

The preceding video presents three separate different violations of TRUSTe rules and of TRUSTe’s prior representations of Coupons.com’s supposed compliance:

A) Step 4 shows a deceptively-named file placed on a user’s computer. There is no proper reason to call this file a .CID rather than the DLL that it is. Nor is there any proper reason for Coupons.com to place the same file in both c:\WINDOWS\ and c:\WINDOWS\system32. Indeed, my tests indicate that Coupons.com sometimes uses one of those folders, sometimes the other, and sometimes both — a randomization procedure with no proper purpose, but with the natural effect of confusing users and hindering detection and removal.

These deceptive filenames are exactly contrary to TRUSTe’s claim that it has resolved the problem of Coupons.com’s “inappropriately-named files.” These deceptive filenames and randomized locations also violate TRUSTe rule 14(e)(v), which prohibits “using randomized or intentionally deceptive file names … for the purpose of avoiding detection and removal.”

B) Step 5 shows a deceptively-named registry key. Coupons is not, and is not commonly known as, “English.cpl.” Indeed, the file extension “CPL” indicates a Control Panel applet or extension — but Coupons.com offers no such extension. Neither does Coupons.com have any proper basis to place its configuration data in HKCR — a registry area reserved for file extensions and COM class registrations. Rather, Coupons.com clearly chooses this area to store its configuration data because users would never think to look here. Indeed, in repeated testing, I found that Coupons.com sometimes used other keys instead. For example, in a separate video early on March 19, I found that Coupons.com used HKCRWeb.Template.URL rather than HKCREnglish.cpl. Randomization of registry keys further confirms that Coupons.com uses these registry locations to avoid detection.

These randomized and intentionally-deceptive registry keys are exactly contrary to TRUSTe’s claim that all registry keys are “placed in a typical location [and] named in an appropriate manner.” These deceptive filenames and randomized locations also violate TRUSTe rule 14(e)(v), which prohibits “using randomized or intentionally deceptive … registry entries for the purpose of avoiding detection and removal.”

C) Step 6 shows that Coupons.com fails to remove all its files and registry keys upon a user’s specific request to uninstall.

The retention of these files is exactly contrary to TRUSTe’s claim that the “new version uninstaller removes the files left behind.” The retention of these files also violates TRUSTe rule 7.(a)(ii), requiring a complete uninstall and allows only limited exceptions — none of them applicable here.

The retention of these files also violates Coupons.com’s license agreement — which mentions only that “license keys will not be removed when the Software is uninstalled,” but says nothing about software code left behind. Although TRUSTe’s Trusted Download rules do not specifically require that a company comply with the provisions of its license agreement, I take such compliance to be so obvious that it does not require a specific mention. Coupons.com’s violation of representations in its own license agreement therefore constitutes yet another violation of TRUSTe requirements.

Additional Violations: Coupons.com Retrieving Windows CD key and system serial numbers

In testing using API and registry-monitoring tools, I have determined that Coupons.com retrieves a wide variety of sensitive Windows registry keys and computer configuration settings including Windows Product ID, Windows CD key, motherboard serial number, and hard drive serial number. These numbers serve to identify a specific individual computer, and these numbers persist over the lifetime of a computer. Coupons.com. These practices stand in sharp contrast to Coupons.com’s representations to users:

  • The Coupons.com “promo” promises that “The Coupon Printer does not gather or ask for any personal information about … your computer.” Yet my testing indicates that Coupons.com gathers detailed computer-specific information about each computer on which it is installed.
  • Coupons.com’s privacy policy similarly promises that “The Coupons, Inc. software … only collect[s] information about what coupons have been printed and redeemed from your computer” — again, directly at odds with my observation that Coupons.com collects far more information.
  • Coupons.com’s license agreement discloses this information collection only by admitting that the “software uses anonymous, assigned numbers and/or anonymous information about your computer or device.” But the numbers at issue are not anonymous: These numbers identify a specific individual user based on the user’s unique and unvarying Windows CD key, motherboard serial number, and hard drive serial number. TRUSTe rule 1.qq defines such information to be pseudonymous (“information that may correspond to a person [such as] machine ID”), while rule 1.i defines anonymous information to exclude all pseudonymous information. Coupons.com thus errs in characterizing these numbers as “anonymous.” Moreover, Coupons.com errs in disclosing this data collection practice only in its license agreement; because this practice speaks to user privacy, it belongs in Coupons.com’s privacy policy.

TRUSTe’s Ineffective Investigation and Response

TRUSTe staff could have identified each of these defects when they tested Coupons.com software in February. Instead, TRUSTe staff issued a boilerplate endorsement — failing to identify shortcomings that would have been apparent in any careful analysis.

Remarkably, even after my post above and even after John Stottlemire’s March 18 post detailing many of these issues in great detail, TRUSTe nonetheless described Coupons.com’s problems as “corrected.” TRUSTe even called this process “a good example of how the [Trusted Download] program should work.” I emphatically disagree: Coupons.com remains flagrantly in violation of TRUSTe’s instructions and rules, and TRUSTe has failed either to obtain suitable corrections or to eject Coupons.com from its whitelist.

To this day, Coupons.com is in breach of TRUSTe’s rules, and TRUSTe knows it. Yet Coupons.com remains listed on TRUSTe’s whitelist as if its practices are beyond reproach and as if the company is in good standing vis-a-vis TRUSTe’s rules. That’s outrageous, and users should demand better.

Posted on

Certifications and Site Trustworthiness

When a stranger promises “you can trust me,” most people know to be extra vigilant. What conclusion should users draw when a web site touts a seal proclaiming its trustworthiness? Some sites that are widely regarded as extremely trustworthy present such seals. But those same seals feature prominently on sites that seek to scam users — whether through spyware infections, spam, or other unsavory practices.

It’s no great surprise that bad actors seek to free-ride on sites users rightly trust. Suppose users have seen a seal on dozens of sites that turn out to be legitimate. Dubious sites can present that same seal to encourage more users to buy, register, or download.

But certification issuers don’t have to let this happen. They could develop and enforce tough rules, so that every site showing a seal is a site users aren’t likely to regret visiting. Unfortunately, certification don’t always live up to this ideal. Writing tough rules isn’t easy, and enforcing them is even harder. Hard-hitting rules are particularly unlikely when certification authorities get paid for each certification they issue — but get nothing for rejecting an applicant.

Today I’m posting Adverse Selection in Online “Trust” Authorities, an empirical look at the best-known certification authority, TRUSTe. I cross-reference TRUSTe’s ratings with the findings of SiteAdvisor — where robots check web site downloads for spyware, and submit single-use addresses into email forms to check for spam, among other automated and manual tests. Of course SiteAdvisor data isn’t perfect either, but if SiteAdvisor says a site is bad news, while TRUSTe gives it a seal, most users are likely to side with SiteAdvisor. (Full disclosure: I’m on SiteAdvisor’s advisory board. But SiteAdvisor’s methodology speaks for itself.)

(update, July 2009: I have posted a revised version of Adverse Selection in Online “Trust” Authorities, as published in the Proceedings of ICEC’09)

What do I find? In short, nothing good. I examine a sampling of 500,000+ top web sites, as reported by a major ISP. Of the sites certified by TRUSTe, 5.4% are untrustworthy according to SiteAdvisor’s data, compared with just 2.5% untrustworthy sites in the rest of the ISP’s list. So TRUSTe-certified sites are more than twice as likely to be untrustworthy. This result also holds in a regression framework controlling for site popularity (traffic rank) and even a basic notion of site type.

Particularly persuasive are some specific sites TRUSTe has certified as trustworthy, although in my experience typical users would disagree. I specifically call out four sites certified by TRUSTe as of January 2006:

  • Direct-revenue.com – Makes advertising software known to become installed without consent. Tracks what web sites users visit, and shows pop-up ads. Historically, blocks many attempts at removal, automatically reinstalls itself, and deletes certain other programs from users’ PCs. Faces litigation by the New York Attorney General plus consumer class actions.
  • Funwebproducts.com – This site, among other Ask.com toolbar distribution points, installs a toolbar into users’ web browsers when users install smileys, screensavers, cursors, or other trinkets. Moves a user’s Address Bar to the right side of the browser, such that typing an address into the standard top-left box performs a search rather than a direct navigation. Promotes its toolbar in ads shown by other vendors’ spyware.
  • Maxmoolah.com – Offers users “free” gifts if they complete numerous sequential partner offers. Privacy policy allows sharing of user’ email addresses and other information with third parties. In testing, providing an email address to Maxmoolah.com yielded a total of 485 distinct e-mails per week, from a wide variety of senders.
  • Webhancer.com – Makes online tracking software, which I have personally observed is often installed without consent. Monitors what web sites users visit, and sends this information to Webhancer’s servers.

This is an academic article — ultimately likely to be a portion of my Ph.D. dissertation. So it’s mathematical in places where that’s likely to be helpful (to some readers, at least), and it’s not as accessible as most of my work. But for those who are concerned about online safety, it may be worth a read. Feedback welcomed.

In its response to my article, TRUSTe points out that Direct Revenue and Maxmoolah no longer hold TRUSTe certifications. True. But Maxmoolah was certified for 13+ months (from February 2005 through at least March 2006), and Direct Revenue was certified for at least 8 months (from April 2005 or earlier, through at least January 2006). These companies’ practices were bad all along. TRUSTe need not have certified them in the first place.

TRUSTe then claims that its own web site made an “error” in listing FunWebProducts as a member. TRUSTe does not elaborate as to how it made so fundamental a mistake — reporting that a site has been certified when it has not. TRUSTe’s FunWebProducts error was compounded by the apparent additional inclusion of numerous other near-identical Ask.com properties (Cursormania, Funbuddyicons, Historyswatter, Mymailstationery, Smileycentral, Popularscreensavers). TRUSTe’s error is particularly troubling because at least some of the erroneously-listed sites were listed as certified for 17 months or longer (from May 2005 or earlier, through at least September 12, when Google last crawled TRUSTe’s member list).

As to Webhancer, TRUSTe claims further tests (part of TRUSTe’s Trusted Download program) will confirm the company’s practices. But that’s little benefit to consumers who currently see Webhancer’s seal and mistakenly conclude TRUSTe has already conducted an appropriate review of Webhancer’s products, when in fact it has not. Meanwhile, I have personally repeatedly observed Webhancer’s bad installation practices day in and day out — including widespread nonconsensual installations by the notorious Dollar Revenue, among others. These observations are trivial to reproduce, yet Webhancer remains a TRUSTe certificate holder to this day.

Consumers deserve certifications that are correctly issued in the first place — not merely revoked after months or years of notorious misbehavior, and not mistakenly listed as having been issued when in fact they were not. TRUSTe is wrong to focus on the few specific examples I chose to highlight. The problem with TRUSTe’s approach is more systemic, as indicated by the many other dubious TRUSTe-certified sites analyzed in my dataset but not called out by name in my paper or appendix.

Consider some of the other unsavory sites TRUSTe has certified:

  • TRUSTe certifies numerous sites that most users would call spammers — like focalex.com (which sends users 320+ emails per week, in SiteAdvisor’s tests), yourgiftcards.com (147 emails per week), and everyfreegift.com (86). All three of these sites remain TRUSTe members listed on TRUSTe’s current member list.
  • TRUSTe continues to certify freecreditreport.com, which offers a “free” credit report that actually costs users $12.95/month if they don’t remember to cancel — a practice so misleading it prompted FTC litigation.
  • TRUSTe has certified Hotbar (now owned by 180solutions) and Hotbar’s Wowpapers.com site — advertising software that tracks users’ browsing and shows extra pop-ups.
  • In January 2005, mere days after I reported eZula’s advertising software becoming installed without consent, TRUSTe’s newsletter specifically touted its certification of eZula.
  • TRUSTe even certified Gratis Internet, which was revealed to have sold 7.2 million users’ names, email addresses, home phone numbers, and street addresses, in specific violation of its privacy policy.

TRUSTe’s response claims that my conclusions somehow reflect SiteAdvisor idiosyncrasies. I disagree. I can’t imagine any reasonable, informed consumer wanting to do business with sites like these. TRUSTe can do better, and in the future, I hope it will.

I’m sometimes asked where I’m headed, personally and professionally. Posting a new academic article offers an appropriate occasion to explain. I’m still working on my economics Ph.D., having drafted several papers about pay-per-click advertising (bidding strategies, efficiency, revenue comparisons), with more in the pipeline. After that? An academic job might be a good fit, though that’s not the only option. Here too, I’d welcome suggestions.

Posted on

Search Engine Safety, Revisited

This article uses data from SiteAdvisor, a company to which I serve as an advisor.

In January I bemoaned the sorry state of search engine results for "screensavers." I pointed out that most "screensavers" ads lead to sites I can’t recommend, and I criticized search engines for their failure to enforce higher standards. But this problem goes well beyond that single keyword and that single genre of sites.

Today SiteAdvisor’s Hannah Rosenbaum and I released The Safety of Internet Search Engines. We obtain top search engine keywords from authoritative sources like Google Zeitgeist. We extract top organic and sponsored search engine results for those keywords. Then we evaluate site safety, using SiteAdvisor’s assessments of spyware, spam, scams, and other Internet menaces.

A representative Google ad -- asking users to pay for software widely available elsewhere for free.SiteAdvisor markup of search results, flagging a representative Google ad — asking users to pay for software widely available elsewhere for free.

Our most notable result? Search engine ads are a risky business. Overall, across all keywords and search engines, 8.5% of sponsored results were "red" or "yellow" by SiteAdvisor’s standards, versus only 3.1% of organic results. It’s not unusual to see ads for notorious spyware vendors like Direct Revenue (as documented in my January piece); for sites that charge for software available elsewhere for free (like the ad shown at right, trying to charge $29 for Skype’s free phone); and for spammers that send hundreds of mesages per week, if a user enters a single email address. These scams deceive and harm search engine users, and I’d like to see Google update its advertising editorial guidelines to prohibit such practices — then enforce these rules with appropriate diligence.

Our article includes an abundance of data. I particularly enjoy this chart of Google site safety by individual keyword — showing "free screensavers" as our single most dangerous search, with other notorious searches including "bearshare," "free music downloads," "winzip," and "kazaa." See also our charts of specific red and yellow sites found within search results.

The full article:

The Safety of Internet Search Engines

Posted on

Deciding Who To Trust

This article is a bit different from most of my site: My other articles generally discuss specific vendors, their practices, and how they cause harm. This article offers a possible solution — from a company that, let me say at the outset, has invited me to join its advisory board. They didn’t ask me to write this; I’m writing on my own. And they don’t control me or what I write. But for those not interested in a commercial service that may help protect users from spyware, please read no further.

Much of the spyware problem results from users visiting sites that turn out to be untrustworthy or simply malevolent. I’m certainly not inclined to blame the victimized users — it’s hardly their fault that sites run security exploits, offer undisclosed advertising software, or show tricky EULAs that are dozens of pages long. But the resulting software ultimately ends up on users’ computers because users browsed to sites that didn’t pan out.

How to fix this problem? In theory, it seems easy enough. First, someone needs to examine popular web sites, to figure out which are untrustworthy. Then users’ computers need to automatically notify them — warn them! — before users reach untrustworthy sites. These aren’t new ideas. Indeed, half a dozen vendors have tried such strategies in the past. But for various reasons, their efforts never solved the problem. (Details below).

This month, a new company is announcing a system to protect users from untrustworthy web sites: SiteAdvisor. They’ve designed a set of robots — automated web crawlers, virtual machines, and databases — that have browsed hundreds of thousands of web sites. They’ve tracked which sites install spyware — what files installed, what registry changes, what network traffic. And they’ve built a browser plug-in that provides automated notification of worrisome sites — handy red balloons when users stray into risky areas, along with annotations on search result pages at leading search engines.

The SiteAdvisor Idea

I’ve long known that the best way to assess a web site’s trustworthiness is to examine and test the site. In general that’s remarkably time-consuming — requiring at least a few minutes of time, of a high-skill human researcher. But a tester is inevitably looking for a few basic characteristics. Does the site offer programs for download? If it does, do those programs come with bundled adware or spyware? In principle this is work better suited to a robot — a system that can perform tests around the clock, with full automation, in massive parallel, at far lower cost than a human staff person. SiteAdvisor has built such robots, and they’re running even as I write this. The results are impressive. See an example report.

Of course automated testing of web sites can find more than just spyware. What about spam? Whenever I see a web form that requests my email address, I always worry: Will the web site send me spam? Or sell my name to spammers? As with spyware, it’s a problem of trust. And it’s a problem SiteAdvisor can investigate. Fill out hundreds of thousands of forms, putting a different email address into each. Wait a few months and see which addresses get spam. Case closed.

To provide users with timely information about who to trust, SiteAdvisor has to put a plug-in into users’ browsers. In general I’m no fan of browser plug-ins; most plug-ins serve marketing companies’ interests (i.e. by showing ads) rather than actually helping users. But at just 92 pixels in width, SiteAdvisor’s plug-in is remarkably unobtrusive. I run it on my main PC, and it shares space otherwise left vacant by the Google Toolbar (the only other browser plug-in I accept). See first screenshot below, showing SiteAdvisor in action.

SiteAdvisor in action, evaluating zango.com.   SiteAdvisor's detailed "dossier" report of entertainmentwallpaper.com -- reporting what downloads it offers (and what software they bundle), as well as links, emails, and other areas of possible concern.

Of course there’s more to SiteAdvisor than just these pop-up balloons. If a user clicks “More” in a warning balloon, or otherwise searches the SiteAdvisor site, SiteAdvisor gives detailed information about the risks it found. These detailed “dossiers” report what downloads a site offers (and what software they bundle), as well as links to other sites (potentially hostile or tricky), emails (potential spam), and other areas of possible concern. See right image above, and additional screenshots.

My Role in SiteAdvisor – and How Others Can Help

I’ve been excited about SiteAdvisor — about their product, their technology, and (most importantly) their ability to help users with a serious problem — ever since I learned about the company. I’m so impressed that I agreed to join the company’s advisory board. I’m not involved in day-to-day operations, so specific suggestions are best sent to SiteAdvisor staff, not to me. That said, my relationship with SiteAdvisor is likely to be longer and deeper than my typical consulting gigs, reflecting the seriousness of my commitment to SiteAdvisor.

It’s not easy to design robots that automatically rate the web, and despite SiteAdvisor’s best efforts, their initial ratings aren’t quite perfect. With that in mind, they’re running a preview program. Interested readers can browse SiteAdvisor’s ratings and flag anything that seems wrong or incomplete. SiteAdvisor’s system anticipates its own fallibility — it offers numerous areas for users to contribute comments. There’s even space for reviewed web sites to comment on their ratings — for example, to explain why they think they’ve been unfairly criticized.

Why get involved? If you think, as I do, that SiteAdvisor will attract a large group of passionate users, then it’s sensible to help improve the reviews these users receive. Also, SiteAdvisor has produced an incredible dataset, which they’ll be sharing under a Creative Commons license. In the coming months, I’ll be using this data for research; I’m anticipating some exciting articles analyzing how and where users get infected with spyware. Meanwhile, preview participants get access to SiteAdvisor’s fascinating dossiers (example) — a great way to track which programs install which spyware.

SiteAdvisor in Context

As I mentioned above, SiteAdvisor isn’t the first group seeking to improve the web by rating web sites. But SiteAdvisor makes major advances over previous efforts.


An ActiveX installer with a misleading company name, purportedly "click yes to continue."An ActiveX installer with a misleading company name, purportedly “click yes to continue.”

Consider, for example, the code-signing system associated with ActiveX controls. (See example at right.) Anticipating security problems with ActiveX, Microsoft designed IE so that it only shows an ActiveX installation prompt if the ActiveX package is properly signed by an accredited code-signer like (in this example) VeriSign. VeriSign in turn sets criteria on who can receive these certificates. But despite these checks, the system turns out to be woefully insecure. For one, VeriSign wasn’t always tough in limiting who can get its certs. (The cert at right was issued a company calling itself “click yes to continue,” a highly misleading company name. Additional examples.) In addition, VeriSign’s main requirement is that a company provide a verifiable name. A company’s software may be highly objectionable — pop-up ads, privacy violations, spam zombies, you name it — but if the company gives its true name and pays VeriSign $200 to $600, then they’re likely to receive a certificate. After I criticized VeriSign’s cert-issuing practices this spring, VeriSign tightened its processes somewhat, but its Thawte subsidiary continues to issue certificates to companies that users rightly dislike. And other cert-issuers are even worse.

The ActiveX debacle shows at least three problems that can plague a certification system.

1) Certifying the wrong thing. ActiveX code-signing certifies characteristics of lesser concern to typical users. In particular, ActiveX code-signing it certifies that a vendor is who it says it is, and code-signing certifies that the specified vendor really did develop the program being offered. That’s a nice start, but it’s not what most users are most worried about other. Instead, users reasonably want to know: Is this program safe? Will it hurt my computer? As it turns out, a code-signing certificate says nothing about trustworthiness of the underlying code. But seeing the “verified” statement and VeriSign’s well-respected name, users mistakenly think code-signing means a program is sure to be safe.

2) Dependent on payment. I worry about certification businesses that receive payment from the companies being certified. If VeriSign issues a code-signing certificate, it gets paid $200 to $600. If it denies a cert, it gets $0. So it’s no surprise that lots of certificates get issued. I credit VeriSign’s good intentions, on the whole. But VeriSign staff face some odd and troubling incentives as they try to meet their code-signing financial objectives.

3) Complaints. There’s often no clear procedure for users to complain of improperly-issued certificates. I previously noted that VeriSign lacked a formal complaint and investigation process. After my article, VeriSign established a complaint form. But there are no public records of complaints received, of pending complaints, or of complaint dispositions. VeriSign may be doing a great job of handling complaints and of correcting any errors, but the public has no way to know.

Remarkably, these same problems plague other self-styled trust authorities. TRUSTe‘s main seal, its Web Privacy Seal, largely certifies that a web site has a privacy policy and that the site has agreed to resolve disputes in the way that TRUSTe requires. The policy might be highly objectionable and one-sided, but TRUSTe will still issue its seal. From the perspective of typical users, this is a “certifying the wrong thing” problem: Users expect TRUSTe to tell them that a site’s privacy policy is fair and that users can confidently provide personal information to the site, but in fact the certificate implies no such thing. (Indeed, six months after I revealed Direct Revenue, eZula, Hotbar, and Webhancer as TRUSTe certificate-holders, TRUSTe’s member list says all but eZula are all still members in good standing. In addition, these companies are known not for their web sites but for their advertising software — products TRUSTe’s certificate doesn’t cover at all. So TRUSTe’s certification is especially likely to mislead users seeking to evaluate these vendors.) Furthermore, TRUSTe receives much of its funding from the vendors it certifies, raising the worry of financial incentives to issue undeserved certificates. Finally, when I’ve sent complaints to TRUSTe, I haven’t always felt I received a prompt or appropriate response. So in my view TRUSTe suffers the same three problems I flag for the VeriSign/code-signing system.

TrustWatch certifies 180solutions as a "verified secure" siteTrustWatch certifies 180solutions as a “verified secure” site


TrustWatch‘s search engine and toolbar are superficially similar to SiteAdvisor: Both companies offer toolbars that claim to help users stay safe online. But TrustWatch suffers from the same kinds of mistakes described above. TrustWatch generally endorses a site if it has a certificate from GeoTrust, Entrust, TRUSTe, or HackerSafe. These groups vary in their respective policies, but none of them affirmatively checks for the privacy violations, spyware, spam, or other ill effects that users reasonably worry about. Instead, their focus is on SSL certificates — important for some purposes, but peripheral to today’s biggest security problems. Meanwhile, the TrustWatch endorsers charge for their certs — raising the payment problems flagged above. Predictably, TrustWatch’s system yields poor results. For example, TrustWatch certifies 180solutions and Direct Revenue with its highest “verified secure” rating. That’s an endorsement few security experts would share.

At least one certification system (besides SiteAdvisor) seems immune from the problems described above: Stan James‘ Outfoxed provides a non-profit self-organizing assessment of web site trustworthiness, based on recommendations from a web of trusted experts. Because individual users can decide which recommenders to trust, Outfoxed offers the prospect of ratings based on characteristics users actually care about — solving the “wrong thing” problem. Outfoxed doesn’t charge web sites for ratings, and Outfoxed’s relationship-based trust assessments can distribute meaningful feedback to assure rating accuracy. So Outfoxed addresses the problems described above, and I think it reflects a major step forward. That said, as a self-organizing system, Outfoxed needs a critical mass of experts in order to take off. I worry that it might not get there.

Separately, a few security firms have designed automated systems to seek out spyware. See Microsoft’s HoneyMonkeys and Webroot’s Phileas. But these projects only detect exploits. In particular, they don’t identify the social engineering and misleading installations that web users face with increasing regularity.

SiteAdvisor won’t suffer from the three major problems described above. SiteAdvisor tests the specific behaviors most objectionable to typical users — extra pop-up ads, privacy violations, gummed up PCs, and of course spam — and SiteAdvisor doesn’t give a site a green light just because it has an SSL cert or a posted privacy policy. SiteAdvisor won’t issue certifications upon payment of a fee. And in addition to soliciting an abundance of comments, SiteAdvisor promptly and automatically publishes comments for public review. So, though I’ve been critical of other certification systems, I’m truly excited about SiteAdvisor.

Posted on

The News, at My Site and Elsewhere

I’ve recently written about increasingly controversial online schemes — from installations through security holes, to spyware companies deleting each other, to programs that set affiliate cookies to claim commissions they haven’t fairly earned.

These aren’t nice practices, so I suppose it comes as no surprise that someone — perhaps some group or company that doesn’t like what I’m writing — has sought to knock my site offline. For much of Monday and Tuesday, as well as several hours last week, all of benedelman.org was unreachable. My prior web host, Globat, tells me I was the target of the biggest DDoS attack they’ve ever suffered — some 600MB+/second.

The Operations, Analysis, and Research Center at the Internet Systems ConsortiumDDoS attacks continue, but I’m fortunate to be back online — entirely thanks to incredible assistance from Paul Vixie of the Internet Systems Consortium. You may know Paul as the author of Bind or as co-founded of MAPS. (Or just see his Wikipedia entry.) But he’s also just an all-around nice guy and, apparently, a glutton for punishment. Huge DDoS attack? Paul is an expert at tracking online attackers, and he’s not scared. A special thanks to his Operations, Analysis, and Research Center (OARC) for hosting me. In any case, I apologize for my site’s inaccessibility yesterday. I think and hope I’ve now taken steps sufficient to keep the site operational.

Meanwhile, there’s lots of spyware news to share. I now know of fourteen different states contemplating anti-spyware legislation — a near-overwhelming list that is partiucularly worrisome since so many bills are silent on the bad practices used by the companies harming the most computer users. (Indeed, seven of the bills are near-perfect copies of the California bill I and others have criticized as exceptionally ineffective.) At the same time, federal anti-spyware legislation continues moving forward — but in a weak form that I fear does more harm than good.

Then there’s COAST’s dissolution — to my eye, the predictable result of attempting to certify providers of unwanted software when their practices remain deceptive. It’s reassuring to see Webroot standing up for consumers’ control of their PCs, though surprising to see Computer Associates defend COAST’s certification procedure as “valuable.” Now that Webroot and CA have withdrawn from COAST, COAST seems bound to disappear — probably better for users than a COAST that continues certifying programs that sneak onto users’ PCs.

The final surprise of last week’s news: Technology Crossover Ventures joined in a $108 million round of VC funding for Webroot. Wanting to own a piece of Webroot is perfectly understandable. But TCV is also an investor in Claria, a provider of advertising software that Webroot removes. (See also other investors supporting spyware.) How can TCV fund both Claria (making unwanted software) and Webroot (helping users remove such software)? TCV seems aware of the issue: They’ve recently removed Claria from their Companies page. But other sources — Yahoo! Finance, Private Equity Week, Archive.org, and even the Google cache — all confirm that the investment occurred.

Posted on

180 Talks a Big Talk, but Doesn’t Deliver updated February 4, 2005

The anti-spyware community has been abuzz all weekend with the news of spyware company 180solutions joining the Consortium of Anti-Spyware Technology (COAST). From the 180solutions press release:

“180solutions, a provider of search marketing solutions, today announced it has become a developer member of … COAST. … By working with COAST and complying with its strict Code of Ethics, standards and guidelines, 180solutions aligns itself with the organization’s governing companies, … PestPatrol, … Webroot. … “180solutions has passed a lengthy and rigorous review process demonstrating their commitment to develop and distribute spyware-free applications,” said Trey Barnes, executive director of COAST.”

Some specific worries:

Substantive conflict of commitment

COAST members PestPatrol and Webroot currently detect and remove 180 software. So these companies are (rightly!) telling their users that 180solutions software should be removed from users’ computers.

At the same time, according to 180’s press release, 180solutions is “releasing versions of its applications that have been reviewed and evaluated by COAST.” This press release, COAST’s “review” of 180 software, and COAST’s acceptance of 180 into its consortium can only be taken to constitute a COAST endorsement of 180. That’s a clear conflict with COAST members simultaneously recommending that users remove 180 software.

Then there’s the conflict of interest that inevitably arises whenever an anti-spyware company declares an alleged spyware provider to be legitimate. Users buying a vendor’s anti-spyware software think they’re buying that vendor’s best efforts to identify and remove software users don’t want. When the vendor instead accepts funds from a software provider, one making the kind of software that the vendor is supposed to be removing, users can’t help but wonder whose interests the vendor has in mind. To my mind, the better strategy is for anti-spyware vendors to refuse partnerships with any company making software that might colorably be claimed to be spyware. (See Xblock’s statement of policy.)

I don’t want to overstate the problem. So far, PestPatrol and Webroot still detect and remove 180 software. 180 isn’t listed on COAST’s Members page. And COAST members don’t directly receive the money 180 pays COAST.

But the latent problems remains: For a fee, COAST is certifying controversial providers of allegedly-unwanted software, dramatically complicating the role and duties of COAST and its members. COAST staff are providing favorable quotes in 180 press releases. Who can users trust?

180solutions installation practices are outrageous and unethical

180’s endorsement by COAST is particularly puzzling and particularly worrisome due to 180’s many bad business practices. Indeed, in my testing, 180’s installation practices remain among the worst in the industry. The details:

I have personally observed (and preserved in video recordings) more than two dozen instances of 180 software installed through security holes. (Example video.) Just yesterday, I browsed the Innovations of Wrestling site (iowrestling.com, proceed at your own risk), where viewing the site’s privacy policy invoked a security exploit installing more than a dozen unwanted programs, 180solutions software included. (Note that iowrestling’s installations are at least partially random, so it’s hard to replicate this result. But I kept a video and packet log of my findings.)

Even when 180 installers do request consent to install, the disclosure is often quite misleading. For example, I previously documented Kiwi Alpha installing 180, first mentioning 180 at page 16 of a 54-page license agreement. With 180’s installation warning buried in such a long text, ordinary users are unlikely to learn that Kiwi gives them 180. Certainly users don’t grant knowing consent to the installation.

180’s web site claims “no hiding,” but 180 uses a variety of tricks to make its software harder to find and remove. 180 sometimes uses randomized filenames which make its files unusually difficult to locate. 180 also installs itself into multiple directories — sometimes c:Program Files180solutions (or similar), but sometimes into the root of c:Program Files and sometimes directly into a user’s Windows directory. If uses do manage to find and delete some 180 files, another 180 program often pops up to request reinstallation. If these tricks don’t constitute hiding, I don’t know what does.

180’s controversial installation practices are not mere anomalies. I’ve observed these, and others like them, for months on end. Even 180solutions’ director of marketing sees the problem. See Seattle Post-Intelligencer article, reporting his admission that “n-Case could get bundled with other free software programs without the company’s knowledge [which] could lead to the n-Case software fastening to individual’s computers without their knowledge.”

How did 180 get into this mess? It seems 180 hasn’t been careful in choosing who they partner with. In fact, they recruit distributors (as well as advertisers) by unsolicited commercial email. See 20+ examples.

Interestingly, in its recent press release, 180 does not claim to have stopped these controversial practices. If 180 did make such a claim, I’d be able to disprove it easily — there are so many sources of 180 software installed without notice and consent. Instead, 180 claims only that they are working on a “transition” to improved business practices.

But this isn’t the first time 180 has promised to clean up its act. In March 2004, 180’s CEO claimed 180’s “Zango” product — then the new replacement for the older n-CASE — would give users more information before installation. In an April interview, he attributed to the old n-CASE product “certain users … who are not sure where or how they got our software,” but said “the Zango product … is a means to improve that.” On at least these two occasions, 180 has pledged to improve its practices. Nearly a year later, 180 software often still gets installed without notice or consent. So we’re still waiting for the promised improvements. Meanwhile, 180 continues to benefit profit from its millions of ill-gotten installations.

180solutions advertising practices are outrageous and unethical

Beyond controversial installation methods, 180 also deserves criticism for its intrusive and allegedly-anticompetitive advertising practices.

180 covering Delta.com with Hawaiian Airlines web site180 covering Delta.com with Hawaiian Airlines web site

When 180 covers a web site with one of its competitors, 180 doesn’t just show a small popup ad (like, say, Claria — not that Claria’s practices deserve praise). Instead, 180 opens a new web browser showing the competitor’s site, generally covering substantially all of the targeted web site. A user who wants to stick with the site he had previously requested must affirmatively close the new window — taking an extra step due to 180’s intervention. What would we think of a telephone company that connects a user to Gateway when the user dials 1-800-Dell-4-Me, unless the user then presses some extra key to return to what he had requested initially? The real-world analogy makes it almost too easy to assess 180’s legitimacy: No telephone company could get away with such a scam, yet 180’s advertising practices have gone largely unchallenged.

Even more problematic are 180 ads targeted at competitors’ check-out pages. Sometimes 180 lets a user browse a merchant’s web site uninterrupted, but when the user reaches the page requesting order confirmation, 180 then covers the merchant’s site with a competitor — interrupting the user’s purchase. Again, the real-world analogy is straightforward. Suppose one retailer sent its sales employees into a competitor’s store, to invite users to take their business elsewhere as they waited in line to reach the checkout counter. The intruding employees would be arrested as trespassers.

Then there are the thousands of 180 ads that include affiliate codes. Some of 180’s ads cover a web site with a competitor reached through an affiliate link. Via these ads, companies find themselves promoted by 180, and find themselves directly or indirectly paying commissions to 180 — all despite never requesting that 180 advertise or promote them.

Even worse are the 180 ads that target a merchant with its own affiliate links. Here, merchants end up paying affiliate commissions where they’re not otherwise due. For example, when users reach merchants’ sites by clicking through non-affiliate links or by typing merchants’ domain names, 180 nonetheless intercedes by opening affiliate links to merchants’ sites. Whether shown in double windows, hidden windows, or on-screen decoys, 180’s affiliate links make merchants’ commission-tracking systems think resulting purchases resulted from 180’s promotional efforts. Unless merchants figure out that they’re being cheated — being asked to pay commissions not fairly earned — 180 and its advertisers receive commission payments for users’ purchases. (Details; example.)

There’s plenty more to criticize about 180. To this day, installations on zango.com let users install 180 software without so much as seeing 180’s license agreement. Even 180’s current uninstall procedures give far more information than 180 provides prior to installation. And Andrew Clover reported 180 code that deletes competitors’ programs from users’ disks.

COAST’s credibility on the line

180’s claims of planned improvement are essentially unverifiable. Since 180 admits to a mix of permissible and impermissible installations, its claims of improvement cannot be falsified by critiquing current behavior. Instead, whenever I or others show 180 software installed without proper notice and consent, 180 can say this is just a remnant of prior practices not yet cleaned up in “transition.” By the plain text of 180’s press release, we’ll have to wait at least 90 days to prove that 180 isn’t living up to its promises to COAST and to users.

Why would COAST sign onto this bargain? MediaPost reports 180 paying COST a membership fee as large as $10,000 per year, so that gives one clear explanation. Also, notwithstanding participation by PestPatrol and Webroot, COAST’s past is hardly uncontroversial. In 2003, Lavasoft (makers of Ad-Aware) decided to leave COAST, complaining that COAST’s focus on “revenue generation … reflect[s] badly on the entire anti-trackware industry.” Similarly, Spybot refused to join COAST due to participation by companies that were, in Spybot’s view, unethical.

COAST’s credibility is on the line. I don’t see endorsement of software providers as an appropriate part of COAST’s mission. But even if such work were appropriate, 180 deserves no such praise — its history of outrageous practices and its continued use of such practices mean it should be criticized, not granted an award or endorsement.

Update (February 4): Reporting “concern” at COAST’s certification program, Webroot resigned from COAST.

Update (February 7): Computer Associates (makers of PestPatrol) also resigned from COAST. However, a CA spokesperson defended COAST’s endorsement procedure, calling such endorsements “valuable.”

Disclosure: I serve as a consultant to certain merchants concerned about fraudulent activities by 180solutions and its advertisers. I have advised certain attorneys and merchants concerned about 180solutions activities and practices.