An Australian university has estimated that just 0.3% of files shared through BitTorrent are definitely legal. But torrent advocates have questioned the methodology of the study.

The report (PDF), from the University of Ballarat’s Internet Commerce Security Laboratory, involved a random selection of 1,000 torrent files. Of those:

• 890 were confirmed to breach copyright
• 91 were pornographic content which purported to be “amateur” material but where it was not possible to confirm the copyright owner had given permission for them to be shared
• 16 were of uncertain legality
• 3 were confirmed as being distributed with permission.

The report also noted that the most common categories were movies, music and television shows, in which all the torrents breached copyright laws. Listed by the number of people seeding (that is, sharing the complete file or files), all but one of the ten most popular files were movies.

However, there has been some questioning of the report’s accuracy. TorrentFreak gives the best breakdown of the potential problems, including:

The selection was not entirely random, but rather taken from the most-seeded files. That means categories such as books and software, which could have more legal torrents, are under-represented.

The figures for the number of seeds for particular torrents appear massively overstated (some had a million or more seeds listed despite being two years old), suggesting bogus data has been used. In particular, the number of seeds listed for the top ten torrents bears no relation to any confirmed figures. It look suspiciously like the report includes a lot of torrents which are fake and don’t contain the files they claim to.

Other figures don’t tally up: for example, one site which only distributes torrents of legal music files currently makes up 1.5% of all torrents on a major site.

Of course, nobody is seriously disputing that the vast majority of torrent files do contain copyright-breaching material, but if those calling for tighter copyright controls are going to use statistics, it is important their methodology isn’t open to question.