Earlier this week, I took a train up to NYC to give a talk to some potential ISE customers on Wall St. A collection of Chief Information Security Officers and other executives from financial firms. I was asked to speak about software security, and two things happened on this trip that put to rest any doubt that the current state of software security and network security is dismal. I didn't doubt it, but I thought it was particularly humorous that these happened on a trip whose purpose was to give this particular talk.
I arrived at my hotel about an hour before I was scheduled to speak. Since the hotel was only a couple of blocks from Wall St., I figured that I had time to go online and read my email. I opened up my laptop in my room and saw that there was a WiFi base station whose SSID was "Exchange" (which was the name of my hotel) along with several other available base stations. So, I connected to my hotel's access point. I had full bars, so the connection was strong, but I was unable to reach my email server. I had a look at the IP address assigned to me by the network and noticed that it was a factory default address that was probably not what the hotel was using. So, I called the front desk, and I told the woman who had just checked me in that I was having a problem with the wireless network. It seemed that I was not getting a valid IP address. She said something about their street address, and I realized that while this nice lady was very good at checking me into my room, she was not going to be the best tech support person I had ever had.
I explained to the woman that I was able to connect to the wireless network, but that I was unable to read my email because the network was not working. She understood that and said, "Yes, this happens all the time. I will just reboot the thingy. Give it a few minutes and try again." That sounded like a reasonable solution. Meanwhile, I tried the other wireless networks, and none of them would allow a connection without a password. I chalked this up to progress.
Several minutes later, I reconnected to the Exchange network, and I was assigned what looked like a normal NATed IP address. But, I was still unable to connect anywhere. So, I opened up a browser window to see if I needed to log in. What I saw surprised me at first. It looked like some kind of menu console for managing an appliance. I clicked around and realized that I had the ability to configure routing and firewall rules. In fact, I was logged into the hotel's router - the "thingy" if you will. I smiled to myself at the thought of what I could do if I wanted to, but I quit out of that and was able to access the Internet. The connection was pretty slow, and I chuckled at the thought of getting back into the administration console to filter out the other users in the hotel. Of course, I decided against that.
Unbelievable!
But, it gets better.
When I arrived back at Baltimore Penn Station, I left the train and walked to my car. I drove up 2 levels in the parking garage, and I arrived at the exit gate. This parking garage installed an automated system where you use a credit card to get in when you arrive, and if you use the same credit card when you leave, you don't need to take a ticket, and it charges that card and lets you out. At least that's the theory. It didn't work that way on this trip. As I approached the exit, I saw that there were two lanes open for exiting, and that the car in front of me had pulled into one of them. So, went to the other one and inserted my credit card. On my mind was my daughter's school play, which started in about an hour. I had time to grab a quick sandwich and then head to her school. I had planned my trip so that I could be back in time to see her perform.
After about a minute, it seemed odd to me that my credit card had not come out yet. The machine said that it was validating ticket data. But, I had not inserted a ticket. So, I pressed the intercom button, and an attendant asked if she could help me. I told her that I put my credit card in a while ago, and that I wanted to pay and leave. The gentleman in the truck in the other lane yelled to me that he was in the same boat, so I told the woman that neither one of us could leave. She asked us to hold on a second, and in about another minute a woman in a parking attendant uniform appeared. She told me that it might be that the other gentleman and myself inserted our credit cards at the exact same time in the two different machines. I agreed that this was indeed possible. In the meantime, I rather long line of cars had formed behind us.
The parking attendant backed up all of the cars and suggested that I back up about one car length, and that the other gentleman do the same. Then, she suggested that I drive back up to the machine, which I did. My credit card came out, but she said I had to reinsert it. I did, and it said that it was validating ticket data. The attendant said, "oh no." That didn't sound good. I asked what the problem was. She said that every once in a while, when two people insert their credit cards at the exact same time, it crashes their whole system. We did the back up thing again to retrieve our cards. Since the other guy was first, she went and processed his payment manually. That took about 3 minutes. Then, she took my credit card and went to do mine. In the meantime, another car behind me drove into the other lane, which was now available and inserted his card. The system did not respond. It was hosed. A few minutes later, she came back and gave me my credit card and receipt and opened the gate so that I could exit. The line of cars was now very long, and she said she would have to do them all by hand until a technician could come. I have no idea where this technician was coming from, but I was glad to be on my way. I got that sandwich, but because of my delay, I had to eat it in the car on the way to my daughter's play.
What kind of software design results in this kind of crash? The answer is pretty clear to anyone who has worked with software. While they may have tested the system exhaustively, they probably did not test the possibility of putting credit cards in two different machines at the exact same time. Which brings me back (as usual on this blog) to voting machines. They may be tested and tested and certified and verified and validated. But, if on Election Day something unusual happens, a scenario that was not anticipated, something might go very wrong. And, if there is no tangible, physical record of the votes that were cast on the machine, then votes might be lost in an unrecoverable way.
Given what I've seen about voting system standards and voting system testing labs, I would bet money that the parking garage system at Baltimore Penn Station was tested more extensively before it was deployed than the Diebold voting machines that we use in Maryland.
