New York law requires each voting machine to be tested at least once a year by the county Board of Elections, including casting at least 800 votes. The purpose of testing is to determine if the machine accurately records votes.[1]
With DREs (“Direct Recording Electronic” voting systems), this requirement would not be fulfilled by AUTOMATED tests -- because they do not make use of the:
1. "user interface" on which votes are manually entered and the ballot cast (the touch screen or push buttons), and the programming that makes these parts work;
2. programming that captures manually-entered votes and records them electronically;
3. printer for the VVAR (voter verified audit record). This printer must print the votes before the ballot is cast; wait for the voter to inspect the user interface and printout to confirm that they display the same votes and then cast the ballot; and then roll up or otherwise move the printout into a secure, non-viewable storage area;
4. programming and hardware that enable voters to void the printout, change their votes, reprint the VVAR, and inspect it again before casting their ballot;
5. smart cards or other methods used to grant voters access to the DRE.
Because automated testing does not simulate votes cast by voters, it cannot determine if the machine accurately records votes. For example, only touches performed by people can determine whether the touch screen is out of alignment (or calibration) or the pushbuttons are faulty. Also, only testing by people can detect cases where the ballot image on the screen and VVAR appear accurate but the electronically-recorded ballot -- the legal ballot – is inaccurate.
Human testers must access the DRE using the same method required on election day, enter votes, verify the VVAR, and cast ballots. Tests must include all legal votes and vote combinations, overvotes, and undervotes or blank votes. Tests must be duplicated for each interface designed for voters with disabilities or minority languages. Tests must use at least the largest number of ballots and votes that could be cast on election day (some errors in DRE programming can show up only after a certain number of ballots had been entered; also, tampering can be designed to “kick in” after a given number of ballots or votes have been entered in order to remain undetected in small tests). Testers must keep running tallies of all votes cast, or count votes on the VVAR to confirm that electronic and VVAR tallies are the same.
Testing must use the ballot faces and ballot programming to be used on election day. It is easy for a vendor to create a "canned" test that uses its own ballot face and vote combinations, and always works because it merely self-verifies.
To estimate the time needed for a team of one tester and one observer from each major party to test one DRE with 800 votes is simple. If a ballot has 10 races and offers voters the chance to enter 10 votes, then 800 votes requires at least 80 ballots. (Proper testing would usually require more than 80 ballots, as discussed above.) If each ballot requires 3 minutes to cast (time to insert the smart card, enter the votes, inspect the VVPAT, and cast the ballot), casting 80 ballots with 800 votes requires 240 minutes or 4 hours. This assumes that no one stops for a break or lunch, and no one requests to stop for a minute because it looks like something is wrong or a tester made a mistake. Finally, the electronic and paper tallies must be compared.
Vendors supply automated test cartridges or scripts to enable Boards of Elections to avoid the time and expense of realistic, independent testing. However, comparable computer systems in business and industry are always thoroughly tested -- as well as continuously audited to find errors that escaped detection during testing.[2] That's why commercial systems tend to work.
The huge number of documented failures of electronic voting systems are partly the result of inadequate testing[3], which leaves errors to be discovered by voters on election day when machines refuse to accept votes for specific candidates, transfer votes from one candidate to another, etc. Because New York law does not require full auditing of elections run with DREs, testing must be thorough because it is our last opportunity to discover and correct errors.
With PBOS (“Paper Ballots and precinct-based Optical Scanners), the tests are simpler. Also, because fewer optical scanners than DREs are required to serve a given number of voters, fewer machines need to be tested.
Testing uses marked paper ballots of the same kind that will be used on election day. A "test deck" of ballots with all possible vote combinations is prepared. The test deck is fed through the scanner at a pace similar to that on election day -- one ballot every two to five seconds. The correct tallies can be developed as the ballots are marked, so that the testing team does not have to keep running tallies for each race every time a ballot is scanned.
If all vote combinations can be created with a small number of votes, the test deck can be shuffled and scanned again until the required number of votes has been scanned. Rescanning the same deck also makes it easy to simulate elections with a maximum number of voters.
Because ballots marked by voters with disabilities or minority languages would not differ from other ballots, no separate tests of the scanner would be required for them. Testing of accessible ballot-marking devices can be performed separately, simplifying the entire testing procedure.
To conclude, Boards of Elections must evaluate not only the up-front and recurring costs of DREs versus PBOS systems, but also the difference in staffing, time, training, and cost required to test the equipment.[4] Failure to perform meaningful tests can result in malfunctions on election day, causing chaos, disenfranchisement, voter dissatisfaction, corruption of elections, lawsuits, and demands to hold new elections.
----------
1.
State of New York Election Law, Article 7, Title 2, Section 7-206.
2. Testing
is important, but cannot replace auditing because it is impossible to control
what programming and data (votes, tallies) are in a computer that has
communication capability.
3.
120-page list of documented failures: www.votersunite.org/info/messupsbyvendor.asp
4.
More info, pre-election testing:
www.verifiedvotingfoundation.org/article.php?id=6153
www.verifiedvotingfoundation.org/downloads/vendors/training/testingvotingsystems.jones.php