FC: Filtering advocate responds to Consumer Reports article

[Declan McCullagh <declan@well.com>]

[David Burt is a renegade librarian who used to operate filteringfacts.org 
and is an ardent defender of the software. Now he works for a censorware 
vendor. I'm copying Consumer Reports in case they choose to reply to the 
earlier message (http://www.politechbot.com/p-01733.html). ---Declan]

**********

From: "David Burt" <dburt@n2h2.com>
To: "Declan McCullagh" <declan@well.com>
Subject: re:FC: Consumer Reports gives thumbs-down to smut-blocking software
Date: Wed, 14 Feb 2001 20:49:16 -0800
Message-ID: <PGEOKGPKMBNGGDNJLCMFCEBICJAA.dburt@n2h2.com>
MIME-Version: 1.0
Content-Type: multipart/alternative;
         boundary="----=_NextPart_000_0000_01C096C7.9860C170"
X-Priority: 3 (Normal)
X-MSMail-Priority: Normal

Unfortunately, the Consumer Reports test suffers from the same deficiencies 
as most other filter tests:

1) An absurdly small sample.
A filter is trying to block no more than 10% of the web at most.  With 
millions of websites, obviously you need a large sample, at a bare minimum 
10,000 unique URLs, to get a decent idea of its effectiveness.  The 
Consumer Reports test used only 86 sites.  That's simply not enough, the 
possibility of error is large.  When a tester is trying to measure over 
inclusiveness in multiple categories, "sexually explicit content or 
violently graphic images, or that promote drugs, tobacco, crime, or 
bigotry", with 86 sites, absurd doesn't even begin to accurately describe 
this "test."

2) Purposefully selected sample.
The problem of a small sample is compounded by the fact that the sample is 
not random.  The author of the report obviously had a bias against filters, 
and using a purposefully selected sample under these conditions is a 
serious invitation to abuse.

4) Testing the wrong thing
The claims about wrongly blocked sites are made against "AOL Young Teen", 
which is a white list.  That means it is a list of pre-approved sites, 
rather than a list of excluded sites.  The "AOL Young Teen" setting blocks 
probably 99% of the Internet.  It's not something that a school would use 
as a filter.

5) Drawing the wrong conclusions:
The article discusses filtering requirements for certain schools and 
libraries.  Yet it doesn't test the filters used in schools and 
libraries.  The most popular products in ed space, CyberPatrol Server 
version, N2H2, I-Gear, WebSense, Smart Filter, and X-Stop were not included 
in the testing.

This badly conducted research that didn't even test institutional grade 
filters cannot be said to have any relevance to the appropriateness of 
filters in schools and libraries.

---------------------------------------------------------------------------------
David Burt, Market Research Manager
dburt@n2h2.com  http://www.n2h2.com/
Intelligent Technologies For A Safe and Productive Internet
Phone 206 892-1130  Fax: 509 271-4226
--------------------------------------------------------------------------------




-------------------------------------------------------------------------
POLITECH -- Declan McCullagh's politics and technology mailing list
You may redistribute this message freely if it remains intact.
To subscribe, visit http://www.politechbot.com/info/subscribe.html
This message is archived at http://www.politechbot.com/
-------------------------------------------------------------------------