Incredibly, despite widespread public opposition, reasoned technical and policy arguments, and international condemnation by the Electronic Frontiers Foundation and others, the Australian government looks set to proceed with its misguided plan to censor prohibited internet content. In December, the government released details of how the censorship (euphemism of choice: ‘filtering’) scheme will work. It’s not pretty, but there are a couple of consolations tucked away in this announcement:

The Government will introduce legislative amendments to the Broadcasting Services Act to require all ISPs to block RC-rated material hosted on overseas servers.

This is good news for industry (given that there’s to be a filter at all). Because the scheme will be implemented by way of legislative change, ISPs will probably be able to rely on ‘compliance with law’ clauses in customer contracts to restrict their services, and also save face in comparison to a self-regulated industry scheme. This is also good news for democracy: subjecting the bill to proper parliamentary scrutiny is appropriate for a measure of this magnitude (it would arguably be improper — and would probably breach convention — to enact the measure through delegated legislation), making it more likely that the inefficacy and fundamental problems with this policy will be seen and corrected. Of course, given the ALP’s numbers in the Senate and House of Representatives, this may be of purely symbolic value.

Under the National Classification Scheme and related enforcement legislation it is already illegal to distribute, sell or make available for hire RC-rated films, computer games and publications. … This material is currently subject to take-down notices by the Australian Communications and Media Authority (ACMA) if it is hosted online in Australia. However, ACMA is unable to directly regulate content hosted overseas. [The censorship scheme] is an additional measure to the existing take-down regime for Australia-hosted content. …

This is interesting, but largely unsurprising: ACMA has always lacked the power to deal with foreign-hosted content, which made its takedown powers effectively useless against the types of materials the government is seeking to filter. This statement also suggests that the government intends to maintain the current take-down regime for local content, and to apply the filter only to foreign content — bypassing some potential constitutional issues concerning the implied freedom of communication in respect of political and governmental matters.

“The report into the pilot trial of ISP-level filtering demonstrates that blocking RC-rated material can be done with 100% accuracy and negligible impact on internet speed,” Senator Conroy said.

Well, this is just laughable. Of course a URL-based blacklist can be blocked with 100% accuracy — that’s what a blacklist is. But that tells us nothing about false negatives (RC material that isn’t included on the blacklist) and false positives (non-RC material that is). It is, on its own, an irrelevant statistic — it was the very first condition for the trial to take place. Check out the report for yourself here. Other notable conclusions from the report include:

• ‘Telstra did not test circumvention, because it considers that filtering can be circumvented by a technically competent user. … Circumvention testing was not conducted by New Zealand authorities when performing their filtering trial.’ (now there’s a diplomatically worded statement! Enex’s testing found that only 8-16.2% of circumvention attempts of the ACMA blacklist could be thwarted);
• ‘Telstra found its filtering solution was not effective in the case of non-web based protocols such as instant messaging, peer-to-peer or chat rooms. Enex confirms that this is also the case for all filters presented in the pilot’ (in other words, this won’t prevent access to RC material using the very tools that are most commonly used to access it);
• ‘Telstra reported that heavy traffic sites could overload its trial filtering solution if included in the filtering blacklist.
  This is also the case for all filters presented in the pilot.’ (in other words, the blacklist proxy can be deliberately or accidentally bypassed using a DDoS attack or simply by a large number of ordinary users trying to connect to RC content at the same time — paradoxically, meaning that more the popular an illegal piece of content is, the less likely it is to be blocked); and
• ‘Equipment and customer computer system(s), and the number of subscribers connected to a service, versus the capacity of the upstream connection that the ISP maintains … will all affect performance, and can do so at 40 percent performance degradation over theoretical maximum linerate, or more in some cases. The performance results in the pilot are assessed in the context of these variables.’ (in other words, the performance cost of filtering may be ‘negligible’ for some, but is potentially as high as 40 per cent for disadvantaged users — for example, those with worse equipment or cheaper subscriptions).

But here’s where the report really gets interesting:

All filters participating in additional content filtering in the pilot blocked between 78.80 percent and 84.65 percent of inappropriate material.
All filters participating in additional content filtering in the pilot blocked less than 3.37 percent of innocuous content.

This suggests that the filters aren’t nearly as effective at picking up content that isn’t on a predefined blacklist. The implications are, of course, broader than just ‘inappropriate material’ — in practice, to block most RC material the government would also need to rely on fuzzy detection methods to supplement the URL blacklist, or risk missing all the RC content that is shifted to another server as soon as it becomes blocked. This sort of detection is not at all accurate: around 1 in 5 RC items get through, and there’s a 3.37 per cent false positive rate (which is unacceptably high, and would probably be much, much higher for certain kinds of ‘borderline’ content).

• ‘Enex considers it unlikely that any filter vendor would achieve 100 percent blocking of the URLs inappropriate for children without significant over-blocking of the innocuous URLs because the content on different commercial lists varies and there is a high rate at which new content is created on the internet. Enex has also noted, through previous testing, that the higher the accuracy the higher the over-blocking.’ (this captures nicely the inherent tension between accuracy (low false negatives) and correctness (low false positives) — it’s basically impossible for the government to achieve both goals using a blacklist system);
• ‘In terms of over-blocking the results of this trial show that, while an improvement on previous testing levels, this [3.37%] is still considered high.’
• ‘For one of the pass-through filters in the pilot, the increase in list size did not result in any additional performance impact. For another of these filters, however, noticeable performance impact was identified’
• ‘in situations where there is a potential for very high traffic sites, such as YouTube, to have pages on the filtering list, this could result in significantly higher traffic rates passing through the filter, even though the specific pages being accessed are not those on the blacklist. This could cause additional load on the filtering infrastructure and subsequent performance bottlenecks’ (in other words, with the blacklist in operation, it could be much slower to access legitimate websites like YouTube or Facebook);
• ‘To support peaks in traffic, vendors recommend allowing additional network capacity of approximately four times the estimated traffic at the filter.’ (this is both technically and economically unfeasible for most, if not all, Australian ISPs, even with NBN infrastructure)
• during performance testing, participant ISPs encountered performance degradations of 44.15%, 35.8%, 36.45% and the high twenties and teens;

Anyway, there are plenty of other problems identified in the report. Go read it for yourself.

A couple of other further points worth noting from the government’s press release:

The RC Content list will be compiled through a public complaints mechanism. …

“The Government will immediately undertake public consultation with the release today of a discussion paper on additional measures to improve the accountability and transparency of processes that lead to RC-rated material being placed on the RC Content list,” Senator Conroy said.

Some of the options raised include the use of block pages and appeal mechanisms, notification to website owners of RC content and the review by an independent expert and report to the Parliament.

The Government will allocate funds to ACMA to enhance the security of the RC Content list and to automate its transmission to ISPs.

While the whole notion is still ridiculous, it’s good to hear that the government is considering ‘crowdsourcing’ style methods of identifying sites for inclusion — a vaguely democratic, but still highly ambiguous process — and is alive to concerns about how the list will be obtained and updated by ISPs. Whatever method is taken, complete transparency to the public and website owners is crucial.

Legislation is expected in the autumn 2010 session of Parliament, with a 12 month implementation period — also good for industry, since we can expect huge teething problems, and a rushed implementation would be even worse for end users.