Krugman way off base on Alec Yasinsac

This morning, Paul Krugman has an Op-Ed in the New York Times titled When Votes Disappear. Normally I would be very pleased to read such an op-ed, and I was today as well, until I got two thirds of the way down and saw this:

"Although state officials have certified Mr. Buchanan as the victor, they’ve promised an audit of the voting machines. But don’t get your hopes up: as in 2000, state election officials aren’t even trying to look impartial. To oversee the audit, the state has chosen as its “independent” expert Prof. Alec Yasinsac of Florida State University — a Republican partisan who made an appearance on the steps of the Florida Supreme Court during the 2000 recount battle wearing a 'Bush Won' sign."

I almost fell out of my chair when I read that. Now, I was one of the first people to criticize the use partisan officials to administer elections, such as Ken Blackwell who while he was the secretary of state of Ohio was also co-chair of President Bush's reelection campaign in that state. But, what a different perspective it gives when you know the full story, as I do with Alec Yasinsac.

The Security and Assurance in Information Technology Laboratory (SAIT) at Florida State is the best security research group in the state of Florida if not the Southeast. I'm quite familiar with their research. The professors there include Breno de Medeiros, a recent Ph.D. alumnus of our program at Johns Hopkins, Mike Bermester, a famous Cryptographer, and of course Alec Yasinsac. I have known Alec for about 12 years. He is an extremely talented researcher and well respected security expert. The state of Florida contacted SAIT because they are the top computer security research group in the state. As soon as they were contacted, Alec Yasinsac called me with several other members of their lab on the phone because he was concerned that his Republican affiliation was being blown out of proportion by the local press. I understood his concern, but also noted that he is part of a whole group there, and that I believed they should perform this security audit. I also know that this group has recruited outside help from notables such as David Jefferson and Princeton Professor Ed Felten, who I believe are both involved in the audit, and are completely nonpartisan in their work.

I know very well that the SAIT group, including Alec, are only interested in finding out the truth and discovering what happened with the voting machines, if it is at all possible to do so. Hearing a high profile columnist such as Krugman refer to my friend Alec Yasinsac as a partisan hack really stings, and it causes me to now question every time I see someone painted with such a brush in the media. Furthermore, Krugman writes his pieces as though Alec would be performing the audit alone. What a difference it makes to actually know the people involved very well. Krugman would have done well to interview some computer scientists about Alec and SAIT before dismissing this audit out of hand. Sadly, I think this incident illustrates that this columnist is willing to embrace whatever circumstances and appearances serve his message with no regard for whether they are legitimate.

A Worst Case Scenario for a midterm election?

In several recent elections, the eyes of the country fell on one particular jurisdiction that came under the microscope and affected the entire nation. In 2000, it was Florida and hanging chads. In 2004 it was Ohio and long lines, and in 2006 it is shaping up to be Virginia and a single race that will determine which party controls the senate. Every article I have read today states that the race is going to come down to a recount.

Uh oh.

Virginia uses a plethora of different voting technologies. Just about every major vendor is represented. Most of votes in that state were cast on paperless DREs. There are no ballots to recount. A meaningful recount in Virginia is not possible.

The DRE vendors like to pretend that they can perform recounts. They take the vote totals on the machines and print corresponding ballots, and then count them by hand. Let me give an analogy to demonstrate how silly that is. It would be comical if vendors weren't actually doing it and convincing people that they were performing a recount.

Imagine if you had a word document on your computer, and the document stated some fact. You were not sure if the fact was true. So, to verify the fact, you print the word document, and then you read it out loud and say, "Ah, if that's what it says, then it must be true because I'm looking at a printout." What the vendors are doing is printing out the questionable results and then counting them. Of course they are going to match what was on the machine, but they do not provide an independent count. The so-called recounts of DREs are really just print and count, not RE-count. It is a waste of time.

Now, we hear that in Sarasota County, there were 18,000 undervotes in the race for the 13th congressional seat. The race is expected to be decided by fewer than 400 votes. If paper ballots had been used, the huge number of undervotes could be investigated. Without them, there is no recourse - no way to figure out why this happened. I have several theories. Perhaps that many people just did not care about that race. Unlikely in my opinion. Most likely is that the human interface, that is, candidate placement on the ballot caused many people to miss that race. The next possibility is that a software glitch caused votes in that race not to be counted. Finally, it is possible that someone actually did something to cause this. The problem with paperless voting is that we'll never know, and there will never be any way to find out.

It is unbelievable that the control of the US senate is coming down to a close race that cannot be recounted, and for which there are no physical ballots. The vendors may come out with their "emperor's clothes" recounts, but the public should understand that these are not really recounts, they are just print and count.

My Day at the Polls - Maryland General Election 2006

I woke up at 4:30 this morning, although the alarm was set for 5:15. I guess I had a lot of adrenalin pumping about the election. Would it be a total meltdown? Would the e-poll books work? Would the voting machines boot up playing cartoon videos on the screen, or would they appear to work fine? Last night, I went into the precinct after work to help set up the voting machines. We spent about an hour and a half figuring out the best way to configure them and the best way to process the voters, assuming we had long lines like we did in the primary. That saved us a lot of time, and I believe is the only reason we were able to open the polls on time this morning. But, I think that getting started the night before is what triggered the adrenalin rush that caused me to get so little sleep. I am pretty certain that the machines stayed in the synagogue great room unattended overnight.

For the most part things went fine in our precinct. Turnout was extremely high, and we had long lines at times, but I don't believe anybody left without voting due to that. We averted several problems that could have been serious due to the diligence and foresight of our excellent chief judges and the rest of the poll workers. For example, one of our chief judges discovered during the day that we were short two tamper tape seals, which would have caused us to be unable to properly seal two of the voting machines when we closed the polls. She discovered this because she was checking and double checking everything throughout the day. She placed a call to the board of elections, and they sent us the missing tamper seals. Here's another example: When voters check in, there is a voter authority card printed that has the voter's name and party affiliation on it, and which the voters sign. This paper is then put in an envelope on the machine that the voter uses. We we running out of paper, and when one of the printers could not print anymore, we shut down that poll book temporarily, and one of our judges rushed off to another precinct to get some paper for the e-poll book printer. Then, we were able to reopen that e-poll book. We laughed when we discovered 4 extra rolls of paper in one of our boxes at the end of the election. We had just missed them earlier.

The judge who went to the other precinct to get the paper reported that the other precinct had only two voting machines there, and that one of them had died after around 75 votes had been cast on it. That machine was taken out of service and sent to the nearby town of Towson, where presumably the internal flash of the machine would be used to recover those votes. Meanwhile, another voting machine was supposed to be delivered to replace it. I never found out if that happened.

In the early goings, about the most dramatic thing that happened in our precinct was that a woman thought she dropped her hearing aid into one of the machines and insisted that we take it apart to try to recover it. Luckily, it was found nearby on the floor. As to our technical support, once again, as in the primary, our technician was a representative from Diebold who had been hired the day before, and who was servicing three precincts. I saw her from time to time during the day, but as far as I could tell, she really did not have much to do. She was not allowed to touch the machines.

I was impressed with the performance of the e-poll books that failed so miserably in our primary. In our precinct, they worked flawlessly. I observed them very carefully. One test I did was when a couple split up and the husband checked in on my e-poll book, while his wife checked in on the one at another table. The instant that the wife was checked in, she appeared as having voted on my e-poll book. I repeated this test several times. We ran three e-poll books, and I watched them in many different situations throughout the day, and I did not find a single problem. In fact, as a poll worker, I can say that they were quite handy, especially when people came in who were in the wrong precinct, and we were able to tell them where to go because we had the whole state's database on each e-poll book. I still feel that I would prefer a paper card check in system because of fear of how stuck we would be if the power went out, or if the machines failed in an unexpected way. But, with a simple augmentation to our procedures, I would be happy to use these poll books. The modification I would make would be for the check in judges to also have a printed booklet of all the registered voters, sorted in alphabetical order. It would only have to have names, and say, birth dates (to make duplicates unlikely). The judges would have to place a check mark next to each voter's name as they voted. Thus, if the e-poll books worked fine, the burden would be rather small. If they failed in the middle of the election, we would have 3 booklets with sorted lists of who had voted, and we could continue checking in voters with the booklets, making sure nobody checked in more than once. It would be enough of a backup system to make me happy, and under those circumstances, I would support using the e-poll books. (The privacy issues about whether it is a good idea to have so many electronic copies of this database out there is another story. It's important, but I will not address it here because I'm exhausted, and I have a lot of other things to say about our election today.)

I was on a media black out while at the polls, and I just returned home a few minutes ago, so I have no idea what happened in the rest of the country, or in the rest of Maryland today. I can say that in my precinct we only had one serious event with the Diebold voting machines. It happened after we had already closed the polls, and the last few voters who were in line when we closed the doors at 8 pm were voting. This occurrence underscored my biggest concerns and fears about these machines. Before I describe this problem, let me talk about one aspect of my day at the polls. After the primary in September, I wrote a blog entry like this one about my day at the polls. Many of my fellow judges that day eventually read that blog entry, and between that day and today, I have been in the local media in Baltimore quite a bit, appearing on radio shows almost daily, and several times on many days, and appearing on local television a few times a week. By the time our election came around today, my position on e-voting was pretty well known to my fellow judges and to many of the voters who came into the precinct today. As a result, several of the other judges, and quite a few voters commented to me that they were going to read my blog entry tonight; it was a given that I would blog about it. Knowing that I was going to write this, and that many people were going to read it, made people pretty careful to include me in every discussion about issues that came up, and to make sure every single aspect of our election was by the book, which I don't think is the way the majority of precincts are run, based on emails I've received from many election judges in other precincts after the last several elections.

So, while we were watching the last handful of voters cast their ballots (oops, I should say "touch their candidates names on a screen" because we don't use ballots in Maryland, except for absentee and provisional), one of the chief judges came up to me and said that there was a "situation". I was called over where a voter was explaining to one of the judges what had happened, and he repeated his story to me. The voter had made his selections and pressed the "cast ballot" button on the machine. The machine spit out his smartcard, as it is supposed to do, but his summary screen remained, and it did not appear that his vote had been cast. So, he pushed the smartcard back in, and it came out saying that he had already voted. But, he was still in the screen that showed he was in the process of voting. The voter then pressed the "cast ballot" again, and an error message appeared on the screen that said that he needs to call a judge for assistance. The voter was very patient, but was clearly taking this very seriously, as one would expect. After discussing the details about what happened with him very carefully, I believed that there was a glitch with his machine, and that it was in an unexpected state after it spit out the smartcard. The question we had to figure out was whether or not his vote had been recorded. The machine said that there had been 145 votes cast. So, I suggested that we count the voter authority cards in the envelope attached to the machine. Since we were grouping them into bundles of 25 throughout the day, that was pretty easy, and we found that there were 146 authority cards. So, this meant that either his vote had not been counted, or that the count was off for some other reason. Considering that the count on that machine had been perfect all day, I thought that the most likely thing is that this glitch had caused his vote not to count. Unfortunately, because while this was going on, all the other voters had left, other election judges had taken down and put away the e-poll books, and we had no way to encode a smartcard for him. We were left with the possibility of having the voter vote on a provisional ballot, which is what he did. He was gracious, and understood our predicament.

The thing is, that I don't know for sure now if this voter's vote will be counted once or twice (or not at all if the board of election rejects his provisional ballot). In fact, the purpose of counting the voter authority cards is to check the counts on the machines hourly. What we had done was to use the number of cards to conclude something about whether a particular voter had voted, and that is not information that these cards can provide. Unfortunately, I believe there are an unimaginable number of problems that could crop up with these machines where we would not know for sure if a voter's vote had been recorded, and the machines provide no way to check on such questions. If we had paper ballots that were counted by optical scanners, this kind of situation could never occur.

Some conclusions now before I go off to bed. I believe that with proper care, diligent following of procedures, and no unexpected computer or power glitches, there is the possibility that an election in Maryland can run smoothly in a given precinct. We will never know if the results produced by the machines are an accurate tally of the votes that were cast. Did we get it right today in my precinct? It's very possible. The results were consistent with the expected outcome based on our demographics. The only surprise was that Republican Governor Ehrlich beat out Democratic Mayor O'Malley in the governor's race by about 14% of the vote. This was surprising because our precinct voted 2-1 or more for Democrats in all other races, and the precinct is known for having that ratio. Still, I think that the governor's race results are not unrealistic given conversations I've had with democrats who were going to vote for him. But here's the rub. We cannot audit our election. We cannot perform a recount. We cannot see how the votes were really counted. We had election observers in our precinct, and they had nothing to observe, except to write down the final tallies when the outcome was computed.

So, the election is finally over. In the morning, we'll probably have many results across the country and some places where the races are too close to call. In Maryland there are still over 180,000 absentee ballots that need to be counted. All around America, poll workers such as myself are going to sleep now, exhausted after working at least 16 hours as volunteers, putting in this day so that we can continue to enjoy the benefits of democracy. Now its time for partisans to put aside their differences and to figure out how to design better voting systems that can be independently audited, that are not too vulnerable to failures and human error, and that are completely transparent to voters in every way. In Maryland, the pendulum has swung far away from such systems, and I am hopeful and optimistic that we will switch to a precinct-count optical scan paper ballot system with random spot audits before the elections in 2008.

Advice to Voters on November 7

Well, tomorrow is Election Day, and 39% of voters will be casting their votes on electronic voting machines, and the vast majority of votes in the US will be counted by electronic equipment. While I do not believe that there is any reason to have confidence in the fully electronic paperless voting machines used in Maryland and in many other places, I still think that the only way to make sure your vote is not counted is not to vote. So, I suggest that everybody who is registered to vote, get out and vote! Here are my suggestions to voters:

1. Check your voter registration card and sample ballot that you hopefully received in the mail to make sure you know where your polling place is. You would be surprised at how many people go to the wrong precinct. Show up during the non-rush hours if you can. The slowest times are probably between 10 a.m. and 3 p.m.
   
2. Check your summary screens carefully. There have been reports in Florida and Texas of summary screens presenting different candidates from the ones chosen by the voters. Furthermore, there have been reports of certain races not appearing at all in the summary screens, despite voters casting votes. Finally, there are reports of e-voting machines in Virginia truncating the names of candidates on the summary screen. If you find any discrepancy, report it immediately to the poll workers and don't leave the polls without getting to a summary screen that represents exactly how you want to vote.
   
3. Consider yourself to be a poll watcher during you time at the polls. Be vigilant of the behavior of other voters and the poll workers. Make sure nobody is loitering around any of the equipment. Feel free to ask the poll workers about security procedures. If you see any suspicious activity, report it immediately to the chief judge in the precinct and call the local board of elections.
   
4. Sign up for Verified Voting's Election Transparency Project. They provide a toolkit for election observation.
   
5. Read up on the equipment used in your precinct before you vote. There is an excellent resource for that on the EFF web site.
   
6. If you experience any problem at the polls, call the Election Protection Hotline at (866) OUR-VOTE.
   

Let's hope that this election runs as smoothly as possible. Hopefully, in 2008, the momentum will shift away from paperless voting, and we'll be able to verify the outcomes of our future elections.

UConn VoTeR center report: Diebold AV-OS is vulnerable to serious attacks

A powerful new report was released yesterday about the Diebold AccuVote Optical Scan voting terminal (AV-OS). This is a thorough and independent security analysis of the machines that will be used in Connecticut to count votes on November 7. It is based on hands-on experimentation with the system, and is thus more like the Princeton study of the Accuvote TS than my team's earlier source code analysis. Like the Princeton team, the UConn researchers had no access to any internal documentation from the vendor, no source code, or any other information that would have given them an advantage over a random attacker who happened to get access to the machine. Everything they needed to know to perform the attacks was done by reverse engineering the system and observing its behavior. The evaluation was done as part of an evaluation on behalf of the state of Connecticut. They should be commended for not only allowing, but for requesting this study. The report published on their web site explains the attacks in enough detail to be convincing, but some low level details are reserved for another copy of the paper that is only available from the authors by request.

The authors show that "even if the memory card is sealed and pre-election testing is performed, one can carry out a devastating array of attacks against an election using only off-the-shelf equipment and without having ever to access the card physically or opening the AV-OS system box." The attacks presented in the paper include manipulating the count so that no votes for a particular candidate are counted, swapping votes for two candidates, and reporting the results incorrectly based on biases that are triggered under certain conditions.

The attacks in this paper are cleverly designed to make a compromised machine appear to work correctly when the system's audit reports are evaluated or when the machine is subjected to pre-election testing. Besides manipulation of the voting machine totals and reports, the authors explain how any voter can vote an arbitrary number of times using (get this), Post-it notes, if the voter is left unattended.

The attacks are possible because of serious security vulnerabilities that could have been prevented with proper security design. For example, if a serial cable is connected to the AV-OS, an attacker with a laptop can easily obtain a dump of the memory card contents. The dump is obtained in cleartext because the system performs no authentication of any computer that is connected on that port. The dump can be very useful for an attacker, for example, to reconstruct the password and audit records associated with the memory card. The communication between the voting machine and the GEMS tabulation system is unencrypted and unauthenticated. Instead, they use a CRC as a checksum. In our 2003 report, we identified this as a weakness in the Diebold Accuvote TS because CRCs are easily broken. The authors of the new report show how to spoof the GEMS server to the AV-OS, which forms the basis of many of their attacks.

The authors also validate some of the attacks presented earlier by Harri Hursti. They report that the executable code on the memory cards (!!) can be changed so that the counter values change.

Reading this report was a hair raising experience for me. Diebold has clearly not learned any of the lessons from our 2003 report, and it is startling to see that their optical scan ballot counter is as vulnerable to tampering, vote rigging, and incorrect tabulation as the DRE. The big difference, of course, is that optical scanners can be audited. Ballots counted by hand can be compared to the totals of the AV-OS, and machines tabulating incorrectly can be identified. This report highlights the dangers of trusting any component of a voting system that is software based, and the importance of widespread random audits. With optical scan technologies, we can have a secure election even if the systems cheat, due to the opportunity to audit and perform recounts. With DREs, we are left with whatever results the machines compute.

I strongly urge everyone to read this new report out of UConn.

A preview of Florida 2006

There is a story in today's Miami Herald about glitches in the voting machines during early voting. You can only imagine what Election Day will be like if these problems were encountered with a relatively small number of voters at the polls. While most of my comments about e-voting have to do with security threats that are invisible, I am also discouraged by the widespread technical problems that are not just noticeable, but screaming for attention.

Quoting from the article,

"He touched the screen for gubernatorial candidate Jim Davis, a Democrat, but the review screen repeatedly registered the Republican, Charlie Crist. That's exactly the kind of problem that sends conspiracy theorists into high gear -- especially in South Florida, where a history of problems at the polls have made voters particularly skittish."

The article contains other specific examples of the voting machines getting the wrong information on the summary screen. Who knows if the votes that are recorded correspond to the actual choices or to the summary screen. The fact that they don't match is enough reason to conclude that this is an unacceptable way to vote.

Our EAC chairman stated, as I quoted in my previous blog entry, "The bottom line is that our nation's voting equipment, election results and election officials can and should be trusted." I don't see how such a statement can be made in light of the problems with the equipment that early voters in Florida are reporting.

I'm often wondering what it will take to get rid of electronic voting in this country. I used to think that it might take a computer glitch or malicious hacker to cause a ridiculous result, but now I'm thinking that maybe these machines will just fail so miserably that the public will not tolerate them.

A response to EAC Chairman op-ed

In an opinion piece yesterday, EAC chairman Paul DeGregorio argues that academics who are criticizing electronic voting machines are running experiments "in the sterile environment of a laboratory" and that the "hype over hacking [can] discourage voters from participating in elections." He also states that the academic, computer scientists who demonstrate that we can "hack a voting machine" with "unlimited time and resources" are proving nothing. I believe that these comments are aimed more at Ed Felten than at me, but I feel compelled to respond, or at least, to blog about this here.

In my book, Brave New Ballot, I use an analogy about the way the FDA tests drugs to demonstrate how broken the voting system testing process is. This comment by Mr. DeGregorio brings that analogy to mind again. Say that a drug is released to the public and that several well regarded doctors test the drug in their labs and determine that for some reason, this drug is dangerous. Can you imagine someone in the government reacting to that by encouraging people to use the drug and stating that these academic scientists are testing the drug in an unrealistic setting?

But, by responding that way, in a sense, I'm taking the bait because Mr. DeGregorio has actually mischaracterized our position with respect to electronic voting. His op-ed article is based on the flawed assumption that we oppose DRE voting machines because they can be hacked in the lab. While I believe that these machines are indeed vulnerable to undetectable viruses, and while I believe that the demonstrations put forward at Princeton are realistic and frightening, the truth is that focusing a debate on that question is a distraction from our real reasons for opposing these voting machines.

These machines are software based. They require trust in the people who wrote the software. They require that the software be free of bugs, and they provide no means for auditing or checking the vote count. The system is the least transparent voting apparatus I can imagine. Why should we use voting systems that require trust in the manufacturer, trust in their software, and trust that there will never be physical access to the machines by an attacker when there are simple, and available voting technologies (e.g. machine or hand marked paper ballots with precinct optical scan and random audits) that do not require that level of trust?

Paul DeGregorio states in his article:

"The bottom line is that our nation's voting equipment, election results and election officials can and should be trusted. Election officials ... deserve constructive criticism and solutions, not baseless attacks and unfounded accusations about the equipment they use. Attacking their integrity and the system in broad strokes is even less productive."

I have not seen any reason to trust our nation's voting equipment. Trusting it just because an election official says we should is not good enough for me. I want to trust a system because I don't believe it can be compromised, not because someone implies that not trusting it is not patriotic.

This time, Internet voting is being deployed

In 2004, I served on an external peer review panel member for SERVE. Working with David Jefferson, Barbara Simons, and David Wagner, who were also on the panel, we published a report entitled A Security Analysis of the Secure Electronic Registration and Voting Experiment (SERVE). This report led to the cancellation of that risky project.

Well, the Federal Voting Assistance Program (FVAP) is at it again, this time in the form of the Interim Voting Assistance System (IVAS), which is being deployed for this election. I reunited with my co-authors of the SERVE report to publish a new report titled Internet Voting Revisited: Security and Identity Theft Risks of the DoD’s Interim Voting Assistance System. At the end of the report, we summarize the risks of the new system, and I'll repeat them here:

1. Tool One exposes soldiers to risks of identity theft. Sending personally identifiable information via unencrypted email is considered poor practice. No bank would ask their customers to send SSNs over unencrypted email, yet Tool One does exactthat. This problem is exacerbated by potential phishing attacks.
   
2. Returning voted ballots by email or fax creates an opportunity for hackers, foreign governments, or other parties to tamper with those ballots while they are in transit. FVAP's system does not include any meaningful protection against the risk of ballot modification.
   
3. Ballots returned by email or fax may be handled by the DoD in some cases. Those overseas voters using the system sign a waiver of their right to a secret ballot. However, it is one thing for a voter's ballot to be sent directly to their local election official; it is another for a soldier's ballot to be sent to and handled by the DoD – who is, after all, the soldier's employer.
   

Diebold voting machine malfunctions during chief judge retraining

Maryland is providing additional training to chief judges for the November 7 election. Here is an excerpt from an email I received yesterday from one of the two chief judges from my precinct during the primary, and who will serve as chief judge again in the general election (posting this with her permission).

"I wish you could have been with [us] on Saturday when we 'retrained'. There was a Diebold representative there demonstrating the machine and guess what. It malfunctioned! Nothing too bad though. She was trying to cancel the ballot and the machine said it had been inactive and started to shut down."

I think this is worrisome, and I believe that what happened here is that in the chief judge retraining, they carried out scenarios that are not the most expected ones during an election. While the testing that is conducted on the machines during certification and as part of logic and accuracy testing probably covers most of the likely, expected cases, I doubt that something like canceling a ballot in administrator mode gets much test coverage. I think we'll see problems on election day with voting system features that are on the fringes. In Computer Science, boundary conditions are notorious for containing unexpected bugs, and it's scary to think that these can result in the voting machines malfunctioning and shutting down.

This is one of the reasons that I'm nervous about the e-poll books as well as the voting machines. The state says that the e-poll books were tested after the recent bug fixes, but there is no way that any amount of testing can simulate the stress on the system that a real election with hundreds of busy precincts will put on the system. Whenever there is an unusual case, then the system will be running in a state and executing code that was probably not subjected to testing. I don't think that the state elections administrators understand this. It is a bad idea to deploy a buggy system that is patched so close to an actual election. Even if the paper check-in cards are provided as a backup, there will be no way to switch to them if the e-poll books fail during the day because it would take hours to figure out who already voted.

Man of the Year

It started last Thursday. Somebody asked me if I have seen the movie Man of the Year yet. He suggested that I see it right away. On Friday four people asked me if I had seen it yet. They all suggested that I not wait to see it. So, last night, Ann and I got a babysitter and went to the movies for the first time in a couple of years.

Wow.

*** Spoiler alert ***
If you don't want to know what happens in Man of the Year, go see the movie before you read the rest of this.
*** Spoiler alert ***

In the movie, the United States has adopted an electronic touch screen voting system with no paper trail. The manufacturer is a large company whose name begins with a 'D', in this case the name is Delacroy. As the presidential election approaches, a software engineer named Eleanor Green at Delacroy discovers a glitch in the system that might cause an incorrect outcome in the Presidential election. Indeed, an independent candidate, a comedian played by Robin Williams who is only on the ballot in a handful of states, ends up winning the election due to the glitch.

Eleanor Green figures out the cause of the bug in the voting machine. It has to do with the alphabetical ordering of the double letters in the candidates names. The candidates are named Dobbs, Kellogg and Mils. BB comes before GG which comes before LL. While I found the basic premise believable, I think they would have been more convincing, from a technical perspective if the bug had been based on the length of the names. A candidate with a really long name accidentally wins the election because the name overflowed the buffer that held the candidate name in the program. Of course, I'm sure that detail would have been over the head of most viewers.

What I really like about the way this movie portrays the e-voting issue is that it is an unintentional bug in the system that causes the wrong result. In fact, the bug in the Delacroy system is one where it is likely that all manner of testing the machine would not uncover the problem. Unless the testing was done with candidates who had double letters in their names, the problem would remain hidden. When debating with supporters of DREs, I've often been asked how one could rig the voting machines in advance if the candidates names are not known at the time that the software is written. And, while I have an answer to that (described in my book), this movie gives a realistic scenario that also answers the question. An unintentional bug in the system could throw the election in an arbitrary way, while still passing all of the logic and accuracy tests. I've seen enough software bugs to believe this is possible.

For some reason, the reviews of this movie that I found on the Net are negative, but I think it is a must see for anybody interesting in the electronic voting issue.

Another Diebold source code leak

This week, three disks containing Diebold source code, that appear to have come from Wyle Labs and Ciber Inc, the independent testing authorities that certify voting machines for federal qualificaiton, were delivered anonymously to a former Maryland state delegate. The story was covered this morning in the Washington Post and the Baltimore Sun. I was asked by a reporter to inspect the disks to verify their contents, and I enlisted Adam Stubblefield and my Ph.D. student Sam Small, and together we examined them.

The disks contained source code for the BallotStation software, which is the software on the voting machine, and what was labeled as GEMS, which is the back end tabulation system. The GEMS disks were password protected, and while I'm certain we could have cracked them, we chose not to. The BallotStation source code was not protected at all. It was the 2004 version, which is newer than the source code we analyzed in 2003, and appears to be slightly later than the version analyzed by the Princeton team. I would love the opportunity to perform a similar analysis on this code, but yesterday, we were only given the opportunity to inspect to the code to determine whether it was genuine. As a condition to inspecting the disks, we agreed not to make copies or to perform any other activity with the software. An analysis of this source code would answer many questions that I've been asked about whether Diebold fixed the problems we encountered in our previous analysis. Of course, I don't believe that all of the problems we found back then are even fixable, but some of them are.

I've been getting calls all day asking exactly what the significance is of the new software leak. I'm not really sure. If the software leaked out of Diebold, then they obviously have not learned any lessons about securing their proprietary information. If, as I suspect (due to the labels on the disks), the software leaked out of the testing labs, then that is a serious problem that has to be addressed. Don't get me wrong - I think that voting system software should be available to the public, but that is a different issue from whether or not testing labs are competent at protecting things that they are trusted with and that they believe they are supposed to protect.

Dealing with failure

An important sub-area of Computer Science is fault tolerance. In a nutshell, fault tolerance is the ability of a system to continue to function in spite of a failure of one or more of its components. A system that can continue to work even if many parts fail in unexpected ways is said to be more fault tolerant than one that does not.

It seems to me that one of the unheralded problems with the Diebold system, and with DREs in general is that it is extremely fault in-tolerant. Consider a few simple examples from the September 12 Maryland primary:

• In Prince George's County memory cards were accidentally left in the voting machines, causing votes not to be counted initially, and at the very least losing track of the chain of custody of those votes.
  
• In Montgomery county, and in at least one precinct in Baltimore county, smartcards were not delivered to the precincts, causing long lines and people leaving the polls without voting
  
• The removal and reinsertion of a memory card in a Montgomery County precinct caused the voting machine not to tally votes on the memory card. The votes had to be recovered by Diebold off the internal flash memory in the machines, once again losing track of the chain of custody of those votes.
  
• A dead power jack in my own precinct almost caused all the voting machines to run out of power and fail

DREs are highly vulnerable to power outages, software bugs, poll worker errors, hardware failures, and the list goes on. It is very difficult to anticipate how/when the system will experience a small failure, and the system is not fault tolerant, as previous experience has shown.

The reason that I advocate paper ballots is that while a paper based system is not going to be perfect, it will be much more fault tolerant than a fully automated DRE-based system. And on Election Day, we only have one chance to get it right. We need fault tolerance.

A "security feature" from Diebold

The following is an excerpt from an email I received from a gentleman named Walter Mancuso who was a Republican Chief Judge in the September 12 primary in Montgomery County, Maryland.

Approximately a week after the primary I received a telephone call from the Montgomery County Board of Elections inquiring as to why one of the eight touch screen voting machines that we used on election day had recorded no votes, even though 55 voters were logged onto the machine. Neither I nor the Democratic chief judge had any explanation. The person at the [Board of Elections (B of E)] told me that they would investigate, talk to Diebold, and get back to me. After a week I called the Board and asked what they had discovered. I was told that at 6:50 AM (prior to opening the polls) that particular touch screen machine had been rebooted, and the memory card had been removed and reinserted into the machine. I was told that removing the memory card activated a security feature of the touch screen unit, and thus nothing was recorded on the memory card. By the way, no error message was displayed to indicate that that machine had been tampered with and thus should not be used. When we accumulated the votes on the zero machine after the polls closed, that particular machine reported than no voters had used the machine during the election. Thus, prior to talking to Diebold we assumed that the 55 votes were lost.

Walter goes on to explain the the board of elections used the hard drive of the voting machine to recover the 55 missing votes.

So, according to this, there is a security feature that causes a machine with a memory card that is ejected and then reinserted to not record any votes on that memory card. I don't understand what security threat this is designed to counter. Even if a rogue memory card is inserted, how does not recording votes on that card protect anything? Furthermore, this introduces a new risk. A malicious poll worker (i.e. a malicious person who decides to become a poll worker to disrupt the election) could insert and remove each memory card at the precinct during setup. If you believe the message from Walter Mancuso, that would cause none of the votes in that precinct to be recorded anywhere except on the hard drives of the voting machines. But, these votes could only be recovered after the election by the Board of Elections in conjunction with Diebold, as Walter explained later in this message:

However Diebold had good news for us. That good news was that each touch screen unit contained a hard drive, so the B of E, with the help of Diebold was able to recover the “missing” votes from the hard drive.

I worry about the chain of custody of these votes. The chief judges and the other judges have no way to monitor or audit these votes before they are produced and counted by the board of elections, working with the vendor.

NYC book signing 10/4 7 pm

For those of you in New York, I'll be leading a discussion about e-voting and signing copies of BRAVE NEW BALLOT at the McNally Robinson bookstore in New York City at 50 Prince St (between Lafayette and Mulberry) on October 4, 2006. That's this Wednesday evening at 7 pm.

Michigan 5-0; Ravens 4-0

While this blog has been devoted almost entirely to e-voting, I'm going to take a quick break with this post because my favorite football teams are off to an incredible start. Michigan is 5-0, ranked 6th, and appears to be headed for an incredible showdown with Ohio State. I think it is reasonably likely that Michigan and Ohio State will both be undefeated and ranked 2 and 1 respectively when they play.

Meanwhile, the Baltimore Ravens are 4-0 after two unbelievable endings in their last two games. Those two games were pretty painful to watch, but the Ravens somehow pulled them out, due in no small part to their new quarterback, Steve McNair. I still prefer college ball, but when the local NFL team is 4-0 after some pretty frustrating seasons, you gotta love it.

Go Blue!!!
Go Ravens!!!

An octopus of paper

In between various meetings yesterday, I watched some of the House Administration hearing as it was video streamed. I noticed that some of the Representatives made the same mistake that I hear others make and one that I hear repeated often in questions from reporters. "How can we possibly be advocating paper given the problems in Cuyehoga County?" In the ESI study, it was shown that some of the papers did not print, and that the papers did not match the electronic tally. It seems to me that this is like saying, "How can you possibly drive a car given the problems that existed in the Ford Pinto?" or "How can you ever trust a US company given the fraud at Enron?"

Many of us in the Computer Science community have been writing and speaking about the risks of totally electronic voting for several years (Peter Neumann has been doing it for decades). While Diebold has given us a specific instance of an extremely poor electronic voting system, the Accuvote, our criticisms have been leveled at the concept of DREs as much as at the Accuvote. On the other hand, I think that the criticisms of paper ballots apply to the specific "DRE with a paper trail" system that was deployed in Cuyahoga County, and that they do not apply to the concept of paper ballots or paper trails.

More and more, I believe that the best solution to the e-voting dilemma is to use computer-marked or hand-marked paper ballots that are optically scanned, and to randomly audit the scanners. Audio modules can be used for sight-impaired access. I do not like the idea of retrofitting DREs with long rolls of paper trails, or as one election official referred to it, an octopus of paper. I think such weak designs give ammunition to the supporters of DREs and confuse some members of Congress about whether the problem is with the concept of using paper in voting systems or with a particular VVPAT system.

Rivest on audit size estimation

Ron Rivest has a draft of an excellent paper on estimating the number of items (e.g. voting machines) that need to be audited to discover whether or not the machines are cheating. The paper assumes that there is a reliable way to manually check whether a machine is cheating. For example, if every machine had a corresponding paper trail that had been verified by voters, then one could count the papers by hand and check them against the machine.

Rivest has once again dazzled us with his creativity. He presents a simple rule of thumb that can be calculated with a calculator or in one's head for determining how many machines to audit, using what he terms the "rule of 3". Appendix A is especially useful for people who do not follow the technical details. It shows the number of machines to audit based on the number of bad ones that exist and based on the confidence level one wants to achieve. So, for example, in appendix A, you can see that if you have 1,000 machines, and there are 50 "bad" ones, then to have 95% confidence that you have discovered at least one of the bad ones, you must audit at least 57 machines. This, as compared to the rule of thumb which produces the number 59. Amazingly, the rule of thumb is so elegant, and yet it always comes close, and always errs on the side of being a little conservative, meaning that it will never recommend auditing too few.

Rivest has not published this draft, and he is still seeking comments, so if you have any suggestions after reading his paper, he would appreciate it if you could send them.

I now quote from the last section of the paper, where I think this work can have tremendous impact:

"We hope that the rules presented here will provide useful guidance for those designing sampling procedures for audits...it would probably be best to merely mandate a sample size sufficient to detect, with a specified level of confidence, any election fraud sufficient to have changed the outcome."

I often meet with legislators at the state and federal level to discuss voting issues, and I will be pointing them to this work from now on. Thank you Ron Rivest for once again contributing something elegant, practical and long needed!

"free" DREs are expensive

I've read that when drug dealers want to hook a new victim, they often provide free samples, and once the person is addicted, then the price goes up. By providing HAVA funds to the states and requiring them to purchase electronic voting machines, Congress may have inadvertently jump started an expensive addiction.

Let's look at Maryland. My state has spent $106 million on Diebold electronic voting machines. I am not certain, but I would have to guess that much of that money came from our HAVA funds. I have read the HAVA act, and I did not see anything in there about continued financial support to maintain these machines. However, the shelf life of commercial commodity hardware is quite low. Anyone who owns a laptop knows that at some point, the hard drive will fail and the battery will need to be replaced. Batteries are particularly short lived, regardless of whether or not they are used. Despite the fact that voting machines will mostly sit idle and are only used during testing, primaries and general elections, their parts continue to age, and some parts, such as the batteries will need to be replaced every couple of election cycles. And, these parts tend to fail in unpredictable ways. If we continue to use the Diebold DREs, we can expect that several years down the line, a significant fraction of the machines will start to fail arbitrarily in the middle of an election, when the equipment is stressed all at once. The only way to prevent this is to regularly upgrade all of the major parts, the way oil is changed in a car every 3,000 miles. This is very expensive, and there are likely to be no more freebies from the HAVA dealer.

Consider the mechanical lever machines. I am very critical of many aspects of these in terms of transparency, recountability and audit. However, in terms of maintenance cost, these voting machines were relatively cheap. Oil the gears and they lasted for decades. How many people have had the same computer for 10 years? Clearly the primary reason the answer is "very few" is that software and hardware become obsolete as technology advances. But as a result, manufacturers know that they only need computer parts to last 4-5 years at most, and thus there is no need for them to spend extra money producing parts that last longer.

Unless Maryland has a special fund put aside to regularly replace or upgrade many of the hardware components of the electronic voting machines we use, then in future elections, we will find that voting machine hardware failure rates at the polls will rise dramatically. If this past primary is any indication, such Election Day hardware failures will greatly disrupt our ability to hold fair elections.

I'm sure Congress did not set out to thrust an expensive habit upon the states, but those who yielded to the temptation of the HAVA windfall may have to find a way to fund the maintenance and upkeep of this equipment, and it is not going to be cheap.

Felten on voting machine keys

Ed Felten posted a very interesting comment on his blog today. It appears that the same key that Diebold and some Maryland officials tout as securing the memory cards in the Accuvote machine is found in hotel room minibars. Felten points out that while many of the problems that have been discovered in the Diebold machines are technical in nature and difficult to explain, this one is understandable to anyone. This example serves to illustrate to even non-technical people the public relations tactics of the vendor and its supporters. Was it really too difficult for them to design a more secure key? Who do they think they are fooling when they say that someone would have to pick the lock to access the memory cards? I can assure you that the cryptography that was saw in our 2003 analysis displayed weaknesses that were equally stunning, albeit more technical in nature.

Diebold should let us analyze the "new" system

As most of you who read this blog know, Ed Felten and his students at Princeton have implemented the attacks that we described in our paper back in 2003 and some new attacks as well. I was aware of this work and was an early peer reviewer of their paper. I have watched Diebold's reaction to the Princeton paper, and they are reacting as expected based on their track record. Doug Jones has posted a very good commentary about the Diebold response.

Here's what I recommend. Diebold's defense against our paper and against Princeton's paper is that we looked at an old version of the system. Well, my response to that is, let us look at the new one! Every election administrator in the country who uses the Diebold machines should want Ed's team and mine to perform a security assessment of their voting technology. If Diebold's system is not vulnerable to Princeton's virus, then wouldn't they welcome such a public analysis? If they fear that the new version is vulnerable, then isn't that a question that needs to be answered publicly?

Diebold maintains that they now use AES for encryption, and that for this reason, they are immune to the Princeton "hack". Their responses always demonstrate to me how little they understand about security. The Princeton malicious code is running on the machine with access to all of the data and memory. There is nothing in this system preventing malicious code from accessing the AES keys on the machine, and Diebold has never used any kind of key management. They always use the same key in all of the machines.

Why is it that in this country, vendors can get away with this? Shouldn't it be part of the process to have competent and independent security reviews? I think that vendors should have to post their security mechanisms, algorithms and protocols publicly, and let the security community evaluate them. The computer security community eschews security by obscurity. If the voting machines have good security, we'll say so, and if they don't, we'll point out the flaws so that they can be fixed. Secrecy creates legitimates doubts and suspicion. Diebold is not acting like a company that wants to get it right. They act instead like a company that is afraid to have their weaknesses exposed.

Princeton report on Diebold

Finally, someone has had a chance to study an actual Diebold system. I believe this report is going to shake things up and hopefully be the end of the Accuvote DRE.

My day at the polls - Maryland primary '06

I don't know where to start. This primary today is the third election that I have worked as an election judge. The last two elections were in 2004, and I was in a small precinct in Timonium, MD. This time, I was in my home precinct about 1/2 a mile from my house. We had 12 machines, over 1,000 voters and 16 judges. I woke up at 5:30 in the morning and was at the precinct before 6:00. It is now 10:18 pm, and I just got home a few minutes ago. As I have made it my custom, I sat down right away to write about my experience while everything was still fresh. In anticipation of this, I took some careful notes throughout the day.

The biggest change over the 2004 election was the introduction of electronic poll books that we used to check in voters. I was introduced to these in election judge training a few weeks ago. These are basically little touchscreen computers that are connected to an Ethernet hub. They each contain a full database of the registered voters in the county, and information about whether or not each voter has already voted, in addition to all of the voter registration information. The system is designed so that the machines constantly sync with each other so that if a voter signs in on one of them and then goes to another one, that voter will already be flagged as having voted. That was the theory anyway. These poll books turned out to be a disaster, but more on that later.

Around 7:15, when we had been open for business for 15 minutes already, a gentlemen shows up saying that he is a judge from another precinct nearby and that they did not receive any smartcards, so that they could not operate their election. We had 60 smartcards, and the chief judge suggested that we give them 20 so that they could at least get their election started. As she was handing them over, I suggested that we had to somehow verify his claim. After all, anyone could walk in off the street and claim this guy's story, and we would give them 20 access cards. The chief judge agreed with me. The guy pulled out his driver's license to prove who he was, but I told him that we were not doubting who he was, we just wanted to verify that we should give him the cards. He seemed to understand that. After calling the board of elections, we were told to give him the cards and we did. A little later, several voters who came in informed us that news reports were saying that in Montgomery county, there was a widespread problem of missing smatcards. I could only imagine what a nightmare that was for those poll workers because as it was, our precinct did not have this problem, and as you'll see, it was still tough going.

My precinct uses Diebold Accuvote TS, the same one that we analyzed in our study 3 years ago. The first problem we encountered was that two of the voting machine's security tag numbers did not match our records. After a call to the board of elections, we were told to set those aside and not use them. So, we were down to 10. We set up those machines in a daisy chain fashion, as described in the judge manual, and as we learned in our training. We plugged the first one into the wall and taped the wire to the floor with electric tape so nobody would trip over it. About two hours into the voting, I noticed that the little power readout on the machines was red, and I thought that this meant that the machines were on battery power. I pointed this out to one of the chief judges, but she said this was normal. An hour later, I checked again, and this time, the machines were on extremely low power. This time, I took the plug out to of the wall and tried another outlet nearby. The power icon turned green. I showed several of the judges, and we confirmed that the original outlet was indeed dead. Had I not checked this twice, those machines would have died in the middle of the election, most likely in the middle of people voting. I hate to think about how we would have handled that. A couple of hours later, the board of elections informed us that we should use the two voting machines with the mismatched tags, so we added them and used them the rest of the day (!).

When we were setting up the electronic poll books, I took over because I was more comfortable with the technology, and the others quickly deferred to me. So, a couple of hours into the election, when one of the poll books seemed to be out of sync with the others, the judges came and brought me to have a look. It appeared that this poll book was not getting synced with the others. I tested it by waiting for someone to sign in with a different poll book, and then a few minutes later trying to sign in that voter on the one in question. The voter was shown as having not voted yet. I repeated this test for about 20 minutes, but it never registered that voter as having voted, and the poll book was falling behind - about 30 by then - the other poll book machines. I suggested rebooting that machine, and we tried that, but it did not change anything. I pointed out to the chief judges who were huddled around me as I experimented, that as time went by, this poll book was going to fall further and further behind the others, and that if someone signed in on the others, they would be able sign in again on this one and vote again. After a call to the board of elections, we decided to take this one out of commission. This was very unfortunate, because our waiting lines were starting to get very long, and the check-in was the bottleneck. The last few hours of the day, we had a 45 minute to an hour wait, and we had enough machines in service to handle the load, but it was taking people too long to sign in.

The electronic poll books presented an even bigger problem, however. Every so often, about once every 15-25 minutes, after a voter signed in, and while that voter's smartcard was being programmed with the ballot, the poll book would suddenly crash and reboot. Unfortunately, the smartcard would not be programmed at the end of this, so the poll worker would have to try again. However, the second time, the machine said that the voter had already voted. The first few times this happened, we had some very irate voters, and we had to call over the chief judge. Soon, however, we realized what was happening, and as soon as the poll book crashed, we warned the voter that it would come up saying that they had already voted, but that we knew they hadn't. Then, the chief judge would have to come over, enter a password, and authorize that person to vote anyway. Then we had to make a log entry of the event and quarantine the offending smartcard. Unfortunately, the poll books take about 3 minutes to reboot, and the chief judges are very scarce resources, so this caused further delays and caused the long line we had for most of the afternoon and evening while many of the machines were idle. Another problem was that the poll book would not subtract a voter from its total count when this happened, so every time we had an incident, the poll book voter count was further off the mark. We had to keep track of this by hand, so we could reconcile it at the end of the day.

At times, the remaining two poll books were way out of synch, but after a while, they caught up with each other. When the lines got really long, we considered the idea of trying to use the third one that had caused problems, but we all agreed that we would feel very stupid if all of them started crashing more. I was worried that synching three of these on an Ethernet hub was more complex than 2, and in fact, they were crashing a bit less often when we had only 2. The whole time I was worried about what we would do if these thing really died or crashed so badly and so often that we couldn't really use them. We had no backup voter cards, so the best we could have done would have been to start letting everybody vote by provisional ballots. However, we had two small pads of those ballots, and we would have run out quickly. I can't imagine basing the success of an election on something so fragile as these terrible, buggy machines.

Throughout the early part of the day, there was a Diebold representative at our precinct. When I was setting up the poll books, he came over to "help", and I ended up explaining to him why I had to hook the ethernet cables into a hub instead of directly into all the machines (not to mention the fact that there were not enough ports on the machines to do it that way). The next few times we had problems, the judges would call him over, and then he called me over to help. After a while, I asked him how long he had been working for Diebold because he didn't seem to know anything about the equipment, and he said, "one day." I said, "You mean they hired you yesterday?" And he replied, "yes, I had 6 hours of training yesterday. It was 80 people and 2 instructors, and none of us really knew what was going on." I asked him how this was possible, and he replied, "I shouldn't be telling you this, but it's all money. They are too cheap to do this right. They should have a real tech person in each precinct, but that costs too much, so they go out and hire a bunch of contractors the day before the election, and they think that they can train us, but it's too compressed." Around 4 pm, he came and told me that he wasn't doing any good there, and that he was too frustrated, and that he was going home. We didn't see him again.

I haven't written at all about the Accuvote machines. I guess I've made my opinions about that known in the past, and my new book deals primarily with them. Nothing happened today to change my opinion about the security of these systems, but I did have some eye opening experiences about the weaknesses of some of the physical security measures that are touted as providing the missing security. For example, I carefully studied the tamper tape that is used to guard the memory cards. In light of Hursti's report, the security of the memory cards is critical. Well, I am 100% convinced that if the tamper tape had been peeled off and put back on, nobody except a very well trained professional would notice it. The tamper tape has a tiny version of the word "void" appear inside it after it has been removed and replaced, but it is very subtle. In fact, a couple of times, due to issues we had with the machines, the chief judge removed the tamper tape and then put it back. One time, it was to reboot a machine that was hanging when a voter was trying to vote. I looked at the tamper tape that was replaced and couldn't tell the difference, and then it occurred to me that instead of rebooting, someone could mess with the memory card and replace the tape, and we wouldn't have noticed. I asked if I could play with the tamper tape a bit, and they let me handle it. I believe I can now, with great effort and concentration, tell the difference between one that has been peeled off and one that has not. But, I did not see the judges using that kind of care every time they opened and closed them. As far as I'm concerned, the tamper tape does very little in the way of actual security, and that will be the case as long as it is used by lay poll workers, as opposed to CIA agents.

As we were computing the final tallies towards the end of the evening, one of the Diebold machines froze. We had not yet printed the report that is used to post the results. One of the judges went to call the board of elections. She said she was transfered and then disconnected. We decided to do a hard reboot of it after we closed down the other machines. When we finished the other machines, we noticed that the problem one had somehow recovered, and we were able to finish. Strange because it was frozen for about 10 minutes.

So, this day at the polls was different from my two experiences in 2004. I felt more like an experienced veteran than a wide eyed newbie. The novelty that I felt in 2002 was gone, and I felt seasoned. Even the chief judges often came to me asking advice on how to handle various crises that arose. Several other suggested that I should apply to be a chief judge in the next election cycle, and I will probably do that. The least pleasant part of the day was a nagging concern that something would go terribly wrong, and that we would have no way to recover. I believe that fully electronic systems, such as the precinct we had today, are too fragile. The smallest thing can lead to a disaster. We had a long line of "customers" who were mostly patient, but somewhat irritated, and I felt like we were not always in a position to offer them decent customer service. When our poll books crashed, and the lines grew, I had a sense of dread that we might end up finishing the day without a completed election. As an election judge I put aside my personal beliefs that these machines are easy to rig in an undetectable way, and become more worried that the election process would completely fail. I don't think it would have taken much for that to have happened.

One other thing struck me. In 2004, most voters seemed happy with the machines. This time around, many of them complained about a lack of a paper trail. Some of them clearly knew who I was and my position on this, but others clearly did not. I did not hear one voter say they were happy with the machines, and a dozen or so expressed strong feelings against them.

I am way too tired now (it's past 11 pm) to write any kind of philosophical ending to this already too long blog entry. I hope that we got it right in my precinct, but I know that there is no way to know for sure. We cannot do recounts. Finally, I have to say a few words about my fellow poll workers. We all worked from 6 a.m. to past 10 p.m. These volunteers were cheerful, pleasant, and diligent. They were there to serve the public, and they acted like it. I greatly admire them, and while the election technology selection and testing processes in this country make me sick, I take great hope and inspiration from a day in the trenches with these people.

Wagner responds to House committee

David Wagner has posted a response to some follow-up questions from the House Administration and Science committees after his testimony in Congerss recently. This is one of the best and well thought out summaries of why VVPR is critical and of the security problems of DREs. I strongly urge you to read this and to circulate his responses widely.

Absentee is not the answer

In response to the Baltimore Jewish Times Q & A article published about BRAVE NEW BALLOT, a reader wrote in:

A transparently countable paper ballot may be secured and used by voting an absentee ballot. While requesting and submitting an absentee ballot requires a little forethought, the benefit of knowing one's vote will indeed be counted as intended is worth the effort. While it is now too late for the Primary election, there is still plenty of time to request and vote absentee for the General election.

I disagree with this, although I can appreciate the sentiment. Here are the reasons why I disagree with the idea of using the absentee ballot mechanism because of the security problems with electronic voting:

• I beieve that In Maryland, you are required to sign an affidavit that you are unable to be at the polls on election day. I don't think people should be encouraged to lie in an affidavit.
  
• Absentee voting by mail opens up the opportunity for vote selling, and voter coersion. It should not be used as a replacement for precinct voting.
  
• Even if you vote by absentee, you are only guaranteeing that your vote will be recorded correctly. The general problem of the vulnerability of the machines to rigging, tampering, and unintentional faults still exists, and the election outcomes will still be in question.
  
• Absentee voting is a band-aid solution to the problem of people who cannot come to the polls. Increased absentee voting would transfer the trust in the system to the postal service, and I don't think that is appropriate, nor secure.
  

So, while I think that the lack of a paper record of votes in Maryland is an absolute disaster, I do not believe that encouraging absentee voting is a good response.

Areas of expertise

There is a story in today's Baltimore Sun about electronic voting and the upcoming primary in Maryland on Tuesday. In the story, Donald Norris, director of the National Center for the Study of Elections at the University of Maryland, Baltimore County, is quoted as saying:

"Computer science guys are able to get away with what I consider to be shameless scare tactics that don't take into account everything else that goes on in an election."

So, I looked up Dr. Norris on google. He has bachelors degree in History, and an MA and Ph.D. in Government. I would never take it upon myself to critize his understanding of government or history, and I find it surprising that he's willing to criticize computer scientists' understanding of the security issues in electronic voting systems. Sure, there are procedures for handling the voting machines and auditing them in Maryland, but I don't think Donald Norris appreciates the extent to which DRE voting machines are vulnerable, independent of whatever else goes into securing the election.

It is a fact that every single study by security professionals, including my research team, RABA, SAIC and Compuware and Inofsentry, have uncovered serious security vulnerabilities. It is a fact that there has never been a study conducted by computer security professionals that has concluded otherwise. None of the safeguards mentioned by Norris in the article, locks and tamper tape on the machines, accuracy tests before and during the election, and vigilance by poll workers, address the three primary concerns that I have, namely transparancy, auditability, and recovery.

Say that a bug in the voting system software that has never been triggered before causes thousands of voting machines to fail halfway through the election during the primary. What would we do? Seriously, what would we do? Nobody, not Dr. Norris, not myself, and not any comptuer scientists that I know can guarantee that a Windows-based system, running a 50,000 line application will not fail in a new and unexpected way when subjected to a load not possible during testing. So, it's 2 p.m., and half of the people have voted, and suddently all the machines start to crash. I've seen systems where this happens before. What do we do?

I know what we would do. I am a poll worker, and the instruction manual would tell me to notify the chief judge, who would call the board of elections. What would they do if they started receiving calls that the machines were failing and couldn't be rebooted? They would not know what to do.

This kind of failure is not that uncommon, due to bugs and accidents. What if somebody, say a foreign government, wanted this to happen? Would it be hard for them to cause such failures? I don't think so. It's hard enough to get big computer systems running reliably without such adversaries.

My request of Donald Norris is to stick to what he is an expert in, and to leave it to computer scientists to give opinions about computer systems. I would not argue Physics with a Physicist, and I would not correct a German professor's German. I don't think Donald Norris should be criticising the computer experts' opinion. But it's worse. He's criticising us for giving our opinions and not just criticing our opinions.

Bernie Galler, 1928 - 2006

One of the best, kindest, smartest and nicest people I have ever met died this past week. Bernie Galler, my friend, advisor, and mentor since I was 17 years old left this world unexpectedly and far too soon at age 77. I am not going to talk that much about his diverse set of accomplishments. They already appear in his obituary. What I can say is that besides my parents, nobody has had as profound an impact on my life, both professionally and personally than Bernie did. When I was at the University of Michigan, I used to call him my "father away from home." And he truly treated me like family.

I met Bernie Galler my first week of college in an undergraduate advising meeting. From the very first moment I met him, I felt that I had known him all of my life, and I realize now that it was because he treated me almost exactly the same as my own father. He took a great and genuine interest in me, asking me questions about my background and my personal life. It wasn't probing, it was just a true curiosity, and his advice came out of friendship and a desire to make my world a better place. Bernie created an atmoshphere of instant comfort in the room. Having just separated from my parents for the first time, there was a tremendous amount of comfort in meeting Bernie. In that first meeting, he offered me to call him anytime I had questions, and he gave me several phone numbers. He really meant it. Only now, as a busy professor myself, can I appreciate how unusual that was.

I continued meeting Bernie every semester. He convinced me to join the Honors program, and he set me up with summer jobs. He seemed personally thrilled with every accomplishment of mine, as though he had accomplished everything I did himself. Only my parents had ever shown me that kind of encouragement. I vividly remember one meeting with Bernie that stands out. I had recently broken up with a girlfriend, and I was very upset. As soon as I walked in, Bernie put everything down and walked over and put his hand on my shoulder. He asked me what was wrong, and we spent an entire hour talking about my lost relationship. He wasn't 40 years older than me then; he wasn't my professor; he was just my best friend.

Bernie encouraged me to apply to Michigan for graduate school, and he ended up as my Ph.D. advisor for a while, and then as co-chair of my doctoral committee. I also remember Bernie's support as I made it through the tough qualifying process for the Ph.D., which wasn't the smoothest sailing for me, and I remember his pride when I graduated. Again, you would think that he himself had graduated that day. At Michigan, I was a teaching assistant for Bernie several times, and by the end of my time in Ann Arbor, I was used to meeting him for lunch regularly and having dinner at his house.

Over the years, we stayed very close, emailing and visiting. Last year, I visited with Ann and the kids, and we had dinner at the Galler house with Bernie and Enid. Bernie got on the floor with the children and played with them - quite a sight! He also pulled out a box of toys that looked like they must have been old when I was 3 years old. We felt right at home.

Yesterday, at the funeral, I saw that Bernie's affection and friendship that I had always felt was the rule for him, not the exception. I listened to his beautiful family speak about him, and I felt as though I was speaking. I was at the same time jealous that these people had been able to spend their whole lives that close to him, and at the same time fortunate that he played such a big role in my life. I will miss him dearly, but I will try to use his life as a model for my behavior, as I often have in the past.

The world has lost one of the truly great and remarkable people. Exceptionally talented, and infinitely giving. Such people come along once in a lifetime.

BNB is official

Today is the official release of BRAVE NEW BALLOT. I'm heading to DC this afternoon to start the book tour, with the first stop the Diane Rehm show at 10:00 a.m. tomorrow morning, for a live interview about the book on NPR. The Maryland primary is a week from today, and I will post a summary of my day as an election judge as soon as I can. In the meantime, in the next few days, I expect to blog about several new and pressing issues related to the upcoming election.

Brave New Ballot sighting

Brave New Ballot is not officially released until this Tuesday, September 5, in two days. But, Ann and I were in Barnes and Nobles in the inner harber in downtown Baltimore last night, and we were looking in the Current Affairs section, and Ann spotted the book. There were 10 copies on the shelf, facing front. Luckily, my Treo 700p has a half decent camera on it, so here's the proof:

On the importance of paper ballots

A lot has been said about paper ballots and paper trails in the last few years. There are many good arguments for having the paper, including the ability to audit the machines, transparency of the vote counting, recount capability, and voter confidence. But there is another reason why it is important to have paper ballots (which I prefer over VVPAT on a DRE), and that is simply that electronic ballots are more fragile than paper. A power glitch can cause a magnetic memory card to lose its data. So can a magnet. There are multiple ways that electronic data can be come corrupted or lost. Paper is not immune to corruption or loss, but there are two big differences. The loss of many paper ballots is more likely to be noticed immediately, and a loss event is likely to effect fewer paper ballots than electronic votes. One memory card can hold thousands of votes, and such a card is significantly smaller than a deck of cards.

Look at the New Mexico election in 2004. The Washington Post published a story about how 678 votes were completely lost due to a programming error of the electronic voting machines by election staff that was not properly trained. Arguments were made by different people on different sides of the issue about why these votes were lost, or whether they in fact were really lost. The bottom line is that if people had voted with paper ballots (even if they were marked using a ballot marking electronic touchscreen machine), then the election workers would not have even been in a position to cause the votes to be lost.

Another example is Carteret County, NC where, as many news stories reported, 4,532 votes were lost due to faulty electronic equipment.

As we approach another election this fall, we have to consider the possibility of close races and lost votes. With so much at stake, it is a shame that we have to worry about whether or not computers will crash, memory cards will die, or election workers will make mistakes that could cause the wrong results to be tallied. Equally frighting is the possibility that the election will simply fail due to an unecoverable problem.

I don't think enough emphasis has been placed on the problem of recovery in the discussion of e-voting. It is easier to recover from election problems if we have paper ballots to count and machines to audit against paper than if all we have are electronic tallies.

Technology & Tennis

I was watching the US Open last night, and believe it or not, something made me think of the e-voting controversy. It all boils down to the use of new technology. For years now, audiences watching tennis on television have been able to see the "Mac" cam (named for John McEnroe who was famous for arguing line calls in his day and is now an announcer at the US Open) showing the precise landing of the ball next to or on the line. I, and many others, have advocated for the institution of instant replay where the players could challenge the call on a close line call, and instant replay using very high speed cameras could definitively answer the question of whether the ball was in or out. This year, the tennis association finally adopted instant replay at the US Open, and it was on display for the first time yesterday.

Late into the second set, Andre Agassi was having a tough time in his first round match against Andrei Pavel from Romania. Agassi had lost the first set, and it was 4-4 in the second - a very tight match. Pavel hit a ball down the line that looked like it clipped the line, but Agassi was sure it was out. John McEnroe in the announcers booth stated that it was clearly out, and that Agassi should challenge. So, Agassi challenged the call, and the close up slow motion replay showed that in fact it was good. Then something happened that I thought was amazing. McEnroe stated that "you have to question the technology." I couldn't believe it. McEnroe has been preaching for years that we need to add a camera on every line and have instant replay. Now he has what he wants, and the replay (which is actually a graphical simulation of the trajectory of the ball) shows that the ball was in, and so rather than accept it, McEnroe questions the technology.

I find that people in general are perfectly happy with technology, unless it disagrees with their inherent notion about something. When the results of something technologically enabled appear counter-intuitive, the first instinct is to challenge the correctness of the technology. This does not bode well for trusting elections to technological systems that require trust in the computers. As long as the election results are as expected, or as someone wants, that person will be happy. But, the minute there is a controversial election, a close race, or a disgruntled loser (and disgruntled supporters), the technology will come into question.

John McEnroe, long-time advocate of instant replay, had an immediate instinct to question the results of the technology when it disagreed with his observation. That comment by him undermines the entire efficacy of the solution. Similarly, if we continue to move towards election systems that require trust in software and computer systems, the public will justifiably lose confidence in the results as soon as they are unhappy with them.

Voting system reviews are needed

I have been receiving requests for help analyzing voting systems, and the number of requests are increasing now that primary season is heating up. Some of these requests are from parties representing losing candidates and some are from concerned citizens. I don't have the bandwidth to analyze these systems, and I'm already maxed out on travel. It's very frustrating to hear how many people feel they need their voting systems analyzed by experts, without being able to help them all. I know that many people, such as Doug Jones, Dan Wallach, and David Wagner are already assisting several jurisdictions with such analyses, but there are many places out there that are not getting any help. Here is the request I received today, for example:

Dear Mr Rubin,

Please find enclosed information regarding our election challenge in Memphis. We have sought help from others because of the diebold usage and the magnitude of improprieties that occurred during early voting and on election day including voter fraud. Your help is urgently needed to either recommend someone or to come yourself to review information on the machines. We were given your name by Mr. Dill at verifiedvoting.com. Please contact me at once either via email or by calling me at 901.550-1306.

Thanks.
Shep Wilbun

What is really needed is a resource such as a pool of technical people who can quickly descend upon any location, and who have the expertise to do computer forensics, as well as an understanding of elections and election law. Unfortunately such people are (to quote Fred Brooks) as rare as hen's teeth. If anybody has any ideas of what can be done to respond to this and other such requests, please post them in the comments below. My graduate students are all working at full capacity and cannot drop everything to help with these events, and in general, it seems that finding technical help, especially in remote locations such as Memphis, is virtually impossible on short notice. This is a real problem.

Election Judge Training

Our primary in Maryland is coming up on September 12. Once again, I've volunteered to be an election judge in Baltimore County, and this time, I was assigned my home precinct at temple Har Sinai, which is right near my house. Despite having already worked an election, I was required to attend poll worker training again for this election, a three hours session that I attended today with about 60 other judges. I'd say that three quarters of us had been judges before, from a show of hands. We went over all of the procedures, and then we had hands-on training with all of the equipment.

The process is mostly the same as in 2004. The one big difference is that we will be using Diebold's poll books for checking in voters in stead of paper access cards. The poll books are touchscreen computers that contain databases of all of the registered voters in the state of Maryland and their real-time status, meaning, whether they had registered absentee, already voted, were required to show photo ID (for first time voters), etc. The electronic poll books are all networked together via ethernet so that someone can't check in on one machine and then check in later on another. It was not made clear how, if at all, the machines were networked with other poll books at other precincts. I imagine that after the election, they are all synchronized somehow. So, if that's the case, then it would be possible to vote in one preceinct and then go vote in another one, but that would be caught after the fact. However, since there cannot be a record of how someone voted, it's not clear to me what could be done after catching people who did this (besides punishing them) to undo the extra votes. There is no network connection at the polls, so I'm not really sure how cross-precinct synchronization is possible until after the election. There is a modem used with the voting machines, but that is to report preliminary results to the board of elections, and we were not told that it would be used for the poll books.

I found several differences from the last time I did poll worker training too, mostly in the attitude of the other judges. Several of them told me that they did not trust these machines and that they don't see why we have to vote on these when the old way worked fine. I'm pretty sure none of them knew who I was. One gentleman went so far as to tell me that he was sure that "someone on the board of elections must have been in tight with the vendor and received kickbacks, otherwise why would they go to all the trouble to get these unnecessary machines." I shrugged and kind of nodded uncomfortably. There seemed to be several issues that upset some of the judges. One of them was the fact that in Maryland election judges are not allowed to ask people who come in to vote for a photo ID, unless the poll book comes up with a photo ID requirement (MVA registered voters voting for the first time) or if a registered election challenger demands it. When I suggested to a few people that there might be poor people who do not have driver's licenses, they got it, but overall, there was a strong sentiment that security was so important that this was a crazy law.

Another issue that caused a stir was when we were shown how the memory cards are used to accumulate the votes. We were told that these were the definitive ballots, and that we had to guard them with our lives. The man next to me turned to me, again, not knowing my background (I believe), and said that he was outraged that the memory cards would be used to hold votes. He said that a magnet could erase them, and that these fragile things should not be trusted with votes.

Overall, I felt that the awareness that many of us have raised about the DREs in Maryland and the security problems, the lack of audit, and the inability to perform meaningful recounts, had made a difference in this group. One thing that I found a bit funny was a flowchart of how to handle requests for provisional ballots. This appeared in the election judge manual, and we went over it. At the end of the flowchart it says, a person requests a provisional ballot because they are protesting electronic voting, and the instructions say that such a person may not receive a provisional ballot. It was interesting that in the election judge manual in Maryland, there is an acknowledgement that some people might protest the use of DREs at the polls, and there are instructions for dealing with that situation.

Out of the approximately 60 people there, I would estimate that most were older than 75 years old. There was one other person there of my generation, and she told me that her oldest is starting college next year. So, even she is probably a bit older than me, as my oldest is 7 years old. The people seemed enthusiastic, attentive, and despite the 3 hour duration of this training, which did not end early, everybody paid attention the whole time, and nobody hesitated to express themselves about their thoughts.

One final comment. When the admin smartcard was demonstrated to us, the PIN value was 1111. The trainers joked about that, and then said not to worry because there would be real passwords used in the actual elections. (of course in 2004, 1111 was the real value)

Election Science Institute report

About a week ago, the Election Science Institute released a report analyzing the performance of a DRE with a VVPAT in Cuyahoga County, Ohio. The report appears to me to be well written and the study well thought out. It has also generated a lot of chatter on the Internet. I have found on some "pro paper trail" mailing lists that I am on that people have used this report to show that DREs are error prone, and that the paper is more important than ever. Groups such as Voters Unite produced reports to that effect (e.g. this one). Likewise, people who might be categorized as "anti paper trail", such as Dan Tokajo at Ohio State, have used this report to criticize VVPAT (see Tokajo's blog entry).

I find it interesting that different people on different sides of the issue have used this report to back up the claims they've been making all along. One thing that is absolutely clear to me, and something I believe pretty much everybody would agree on is that such studies are extremely valuable, and we need more of them.

I will take this opportunity, as I have in the past, to respectfully disagree with Dan Tokaji, although not entirely. I will concede that the machines used in this study clearly did not implement an ideal paper audit trail. In fact, if you read the study, it is pretty clear that there were many faults with the paper audit trail. Where I part ways with Tokaji's is in his conclusions. I do not believe that the concept of a voter verified paper audit trail should be thrown out just because there was a poor implementation of it. In fact, if you consider a ballot marking system, where there is no electronic tally, such a system qualifies as a VVPAT, and would by its nature avoid many of the problems that arose in Cuyahoga County, Ohio.

As the Voters Unite article that I reference above mentions, there were discrepancies in the electronic tallies between the machines and the memory cards. My feeling is that until we get to the point where we are guaranteed to have no discrepancies (and some day cryptographic solutions may get us there), we need to have a paper trail. We don't necessarily need to have a bad implementation of a paper trail as they did in Ohio, but we cannot afford not to have paper in the process because ultimately, if there are discrepancies, we have to resolve them somehow, and the best way that I can think of is to have pieces of paper that the voters have seen their votes recorded on, avaialable for counts and recounts.

I view the Cuyahoga County report as a very positive development. The more we study new systems and find their warts, the more we can discuss how to develop better systems to reach the goal that everybody is after.

Interview with Kitty Pilgrim on CNN

Today, I took the train down to Washington, DC and did an interview with Kitty Pilgrim on "Lou Dobbs Tonight" on CNN about e-voting and Brave New Ballot.


(double click on the image to view streaming video)


The interview lasted 5 minutes and 37 seconds - a record for me on CNN, I think. I had the opportunity to make a few critical points about the importance of voter verification and the security problems inherent in DREs. I'm hoping that the release of the book will expose a good portion of the public to the issues on a much deeper level than the sound bites in the media have over the last couple of years.