CFP Presentation on Search Engine Omissions; Spyware Workshop Comments updated June 3, 2004

Today I presented Empirical Research on Search Engine Omissions at Computers, Freedom, and Privacy (CFP) in Berkeley, CA. My presentation focused on two prior empirical projects in which I documented sites missing from Google search results: Localized Google Search Result Exclusions (documenting 100+ controversial sites missing from google.de, .fr, and .ch) and Empirical Analysis of Google SafeSearch (documenting thousands of unobjectionable and non-sexually-explicit sites missing from google.com when users enable Google’s SafeSearch feature to attempt to omit sexually-explicit content).

On Monday I was in DC for the FTC‘s Spyware Workshop. I thought the final panel, Governmental Responses to Spyware, did a fine job of explaining the legislative options on the table, and of noting the pressure to address the problem of spyware for the large and growing number of affected users. But I was dismayed that the first panel (Defining Spyware) classified as fine and unobjectionable certain programs that, in my experience, users rarely want, yet often find installed on their computers. Key among these undesired programs are software from Claria (formerly Gator) and WhenU. The technical experts on the second and third panels agreed that these programs pose major problems and costs for users and tech support staff. Yet the first panel seemed to think them perfectly honorable.

Also puzzling was a new position paper from the Consumer Software Working Group recently convened by CDT. Examples of Unfair, Deceptive or Devious Practices Involving Software (PDF) purports to offer a listing of bad behaviors that software ought not perform. It certainly lists plenty of behaviors that so outrageous as to be beyond dispute. But what it misses — indeed, ignores — are the harder cases, i.e. the programs that make spyware a more complicated issue, and the programs that affect the most users. For example, the Examples document condemns software installed without any notice to the user. It is silent about — and thereby is taken to endorse — the far more typical practice of showing a user a license agreement and/or disclosure that describes the software in euphemisms, but admittedly does provide at least some notice of the software’s purpose.

What to make of the document’s failure to consider the methods actually used by the controversial software with highest installation rates? Perhaps one explanation is that Claria and WhenU helped draft the report! (See the signators listed on page five.) That said, the document doesn’t purport to be comprehensive. Perhaps a future version will address the problems of drive-bys and euphemistic, lengthy, or poorly-presented licenses.

For more on the workshop, and another critical reaction, see other attendees’ notes on dslreports.com forums (especially a recent post by Eric Howes). See also impressive studies from PC Pitstop showing that more than 75% of Gator users don’t even know they have Gator (PDF) (not to mention consenting to Gator’s license agreements) and more than 85% for WhenU (PDF).

See also a transcript of the workshop (PDF).

 

DNS as a Search Engine: A Quantitative Evaluation

DNS as a Search Engine: A Quantitative Evaluation. (June – July 2002)

In the course of the Internet’s growing popularity, many Internet users have come to use the domain name system (DNS) as a directory and search engine: When trying to reach the web site of a new or unknown company, users often request the web page at the address http://www.companyname.com, replacing “companyname” with a guess as to a site’s likely domain name. However, this DNS-based method is imperfect in that users may fail to correctly guess or remember a given company’s domain name, instead typically receiving errors or sites operated by other entities.

The research described in this article suggests that alternative search mechanisms, such as leading search engine Google, provide the content of interest with greater accuracy and reliability than does the DNS. This finding supports the claim offered by, among others, DNS software designer Paul Vixie, that DNS “is not a directory service and was never intended to be used as one.” This finding also quantifies Dan Gillmor‘s “Google effect” whereby Google replaces DNS as the preferred mechanism of locating content online.

Research further suggests that, while DNS offers what some might consider relatively high accuracy when conducting searches for top brands, companies, and organizations, DNS is substantially less accurate in searches for smaller brands, companies, and organizations.