About a week ago, the Election Science Institute released a report analyzing the performance of a DRE with a VVPAT in Cuyahoga County, Ohio. The report appears to me to be well written and the study well thought out. It has also generated a lot of chatter on the Internet. I have found on some "pro paper trail" mailing lists that I am on that people have used this report to show that DREs are error prone, and that the paper is more important than ever. Groups such as Voters Unite produced reports to that effect (e.g. this one). Likewise, people who might be categorized as "anti paper trail", such as Dan Tokajo at Ohio State, have used this report to criticize VVPAT (see Tokajo's blog entry).
I find it interesting that different people on different sides of the issue have used this report to back up the claims they've been making all along. One thing that is absolutely clear to me, and something I believe pretty much everybody would agree on is that such studies are extremely valuable, and we need more of them.
I will take this opportunity, as I have in the past, to respectfully disagree with Dan Tokaji, although not entirely. I will concede that the machines used in this study clearly did not implement an ideal paper audit trail. In fact, if you read the study, it is pretty clear that there were many faults with the paper audit trail. Where I part ways with Tokaji's is in his conclusions. I do not believe that the concept of a voter verified paper audit trail should be thrown out just because there was a poor implementation of it. In fact, if you consider a ballot marking system, where there is no electronic tally, such a system qualifies as a VVPAT, and would by its nature avoid many of the problems that arose in Cuyahoga County, Ohio.
As the Voters Unite article that I reference above mentions, there were discrepancies in the electronic tallies between the machines and the memory cards. My feeling is that until we get to the point where we are guaranteed to have no discrepancies (and some day cryptographic solutions may get us there), we need to have a paper trail. We don't necessarily need to have a bad implementation of a paper trail as they did in Ohio, but we cannot afford not to have paper in the process because ultimately, if there are discrepancies, we have to resolve them somehow, and the best way that I can think of is to have pieces of paper that the voters have seen their votes recorded on, avaialable for counts and recounts.
I view the Cuyahoga County report as a very positive development. The more we study new systems and find their warts, the more we can discuss how to develop better systems to reach the goal that everybody is after.
