FROM:

Teresa Hommel

Chair, Task Force on Election Integrity, Community Church of New York

TO:

New York State Board of Elections

40 Steuben Street

Albany, NY 12207-2108

Comment on Draft Voting Systems Standards

Section 6209.6 Examination Criteria

--------------------------------------------------------------------------------------------------------

HOMMEL SUGGESTION - THREE ADDITIONAL SUBSECTIONS

D. USABILITY TEST MOCK ELECTION

1. After a voting system passes all tests described above in this section, the State Board shall conduct a Usability Test Mock Election on a minimum of four voting systems of the type submitted for certification, which shall consist of the Escrow System and a minimum of three additional systems of the same type.

2. The vendor shall supply three or more additional voting systems without fees to the State Board for the purposes of this test.

3. A minimum of ten business days prior to the Usability Test Mock Election for any voting system, and five days prior to the selection of participants, the State Board shall post notification of such selection and test on their web site, and shall send notification by mail and email to each county, each political party recognized in the state of New York, and any member of the public who has previously requested in writing to receive such notification.

a. Each political party recognized in New York State shall be requested to provide two legal observers and two technical observers for such test, and such observers may participate in any part of such test and may attend any training provided by vendors as part of such test.

b. Each County Board shall be requested to submit the name of their county for the random selection of one county to participate in the Usability Test Mock Election. If no County Board submits the name of their county, the State Board shall use the names of all counties for the random selection procedure provided that no county shall be selected more than once for such purpose.

c. Members of the public shall be requested to submit their names for selection as Test Election Inspectors, Test Voters, and Test Observers. If an insufficient number of members of the public submit their names, Election Inspectors from several counties shall be invited to work as Test Election Inspectors, Test Voters, and Test Observers. Such several counties shall include the county where the test is to be held, the county providing the Test Staff, and all adjacent counties.

d. Notification of the Usability Test Mock Election shall include the following:

1). The day and time for random selection of participants.

2) The method by which members of the public may submit their names for selection as participants in the Usability Test Mock Election.

e. The State Board shall perform a random selection procedure conducted at the offices of the State Board at the date and time announced to select participants in the Usability Test Mock Election including the county to provide Test Staff, and persons to serve as Test Election Inspectors, Test Voters and Test Observers.

f. Any member of the public may attend and observe the random selection procedure.

g. Members of the public who are not selected to work as Test Voters or Test Observers may attend and observe the Usability Test Mock Election, and one member of the public per voting system may use a recording device such as a Cam Corder to record any activities at such test.

4. Test Staff

a. The staff of the County Board of the county selected to participate in the Usability Test Mock Election shall work as the Test Staff for such test.

b. Vendors shall provide the State Board with all training materials and documentation intended for use by County Board staff in counties that will use the vendor’s voting systems. The adequacy of vendor training, training materials, and documentation shall be tested to determine whether they are sufficient to enable county staff to achieve independent competence in the conduct of elections, and such test shall consist of evaluation of the competence of the Test Staff in conducting the Usability Test Mock Election.

1) The vendor shall provide the Test Staff with all training in system handling, maintenance, trouble-shooting, service, and repair; ballot programming, election day preparation and operation, and all other tasks required for the independent conduct of elections with the voting system.

2) After certification of the voting system, such training materials and documentation used by the vendor in training the Test Staff shall be retained by the State Board as part of the Escrow System and shall be listed in the Escrow Component List.

5. Test Election Inspectors

a. A sufficient number of persons shall be selected by the State Board to work as Test Election Inspectors so that each specific voting system in the test has two Test Inspectors assigned to it.

b. The State Board shall pay Test Election Inspectors the same fee for their training and work as is paid to Election Inspectors in the county where the Usability Test Mock Election is held.

c. The Test Staff shall provide the Test Election Inspectors with all training, training materials, and voting system documentation necessary to perform all tasks required to act as Test Election Inspectors at the Usability Test Mock Election.

d. The training materials and voting system documentation used by the vendor in training the Test Election Inspectors shall be retained by the State Board as part of the Escrow System.

6. Test Voters and Test Observers

a. A sufficient number of persons shall be selected by the State Board to work as Test Voters and Test Observers so that each specific voting system in the test has a minimum of two Test Voters and two Test Observers assigned to it.

b. The Test Staff shall provide the Test Voters and Test Observers with all appropriate training.

7. Procedure for conducting the Usability Test Mock Election

a. The ballot from the most recent federal election held in the Test Staff's county shall be used for the Usability Test Mock Election.

b. The Test Staff shall prepare the voting systems for the Usability Test Mock Election without assistance from any vendor staff or technicians, but may use the documentation they received from the vendor in the course of their training.

c. The State Board shall create a minimum of 200 ballots with votes in every possible combination, to be provided to the Test Voters. More ballots shall be created if more are needed to provide one ballot with each possible combination of votes.

1. Such ballots shall be printed on paper and numbered sequentially for ease of use by Test Voters. The ballots should be randomly arranged so that they are not be entered in their numbered sequence.

2. The State Board shall create a copy of such ballots in their numbered sequence, one ballot per page, to be called the Observer Ballot-Entry Log, for use by Test Observers to manually take notes confirming the correct entry of each such ballot and to make notes regarding any ballot entered differently than prepared and any irregularities encountered in entering it.

d. Voting procedure.

1) For Usability Test Mock Elections with Optical Scan voting systems, each Test Election Inspector shall provide each Test Voter with one paper ballot at a time in a manner similar to a real election.

2) For Usability Test Mock Elections with DRE voting systems, each Test Election Inspector shall provide each Test Voter with one smart card (or access code or other item as needed by the DRE voting system) at a time, to enable the voter to enter and cast one ballot in a manner similar to a real election.

3) Test Voter shall take turns enter ballots on the specific voting system to which they have been assigned.

e. If an Optical Scan voting system is being tested, a minimum of 20 ballots shall be created using each accessibility attachment, and a minimum of 20 ballots using each minority language required to be used in New York state in any county.

f. If a DRE voting system is being tested, a minimum of 20 ballots shall be entered using each accessibility attachment, and a minimum of 20 ballots using the language display for each minority language required in any county in New York state.

g. The actions of each Test Voter shall be witnessed by the three Test Observers assigned to the same specific voting system. The first Test Observer shall personally visually witness the entry of votes, printout and examination of the voter verified permanent paper record, and the casting of each ballot. The second Test Observer shall use a recording device such as a Cam Corder to make a permanent film of with the visual and auditory image of the Test Voter's work. The third Test Observer shall make a handwritten record on suitable forms created by the State Board of the ballots entered on suitable forms created by the State Board

h. The State Board shall provide light-weight and simple-to-use recording devices for use by Test Observers, and shall retain the recordings so made as long as the voting system is certified for sale and use in the State of New York. The State Board shall makes copies of such recording upon request for a fee not to exceed the actual cost of making such copy.

8. Procedure for End-Of--Day in the Usability Test Mock Election

a. An area in the same room where Test Voting is to take place shall be designated as the Central Reporting Location for the Usability Test Mock Election, and shall be supplied by the vendor with the appropriate equipment.

b. After all votes have been entered on all of the four voting systems, the State Board shall declare the end of voting.

c. After the end of voting, Test Election Inspectors shall use the procedures they learned from the Test Staff to extract end-of-election tallies and other information from the voting systems, and transmit them to the Central Reporting Location in a manner similar to a real election.

d. The Test Election Inspectors for each machine shall manually count the votes cast on the specific voting system to which they were assigned, and compare such tallies to the tallies produced by the voting system. During such counting procedure, the Test Voters and Test Observers assigned to the same voting system shall work as observers, and one of these persons shall use a recording device such as a Cam Corder to make a permanent film with the visual and auditory image of the counting procedure.

e. The Test Staff shall use a projector and screen to display the logs extracted from each voting system and shall compare the contents of each such log to the records provided by the Observer Ballot-Entry Log.

9. Criteria for passing the Usability Test

a. The voter-verified paper record for each ballot and the tallies for each voting system must be 100% accurate.

b. The voting system's election day activity log must be 100% accurate.

c. Every machine in the test must work with no malfunctions or failures.

d. The Test Staff must be able to perform all activities independently in order to conduct the election, including pre- and post-election activities.

e. The Test Election Inspectors must be able to perform all activities independently to facilitate voting.

f. The Test Voters must be able to vote without undue difficulty using every method provided by the voting system including all accessibility attachments and minority language interfaces.

E. PROFESSIONAL HACKING TEST

1. Prior to certification of any voting system, the State Board shall commission a professional hacking test to examine and attempt to compromise the Escrow System.

2. Criteria for passing the test shall be that the voting system withstands all attempts to compromise the Escrow System.

F. PUBLIC HACKING TEST

1. Prior to certification of any voting system and the Usability Test Mock Election, the State Board shall invite any member of the public to participate in the Public Hacking Test. All persons who wish to participate may observe the training conducted by vendors to the Test Staff for the Usability Test Mock Election, provided that prior to such observation the members of the public sign a non-disclosure agreement and are not be an employee of any vendor.

2. On the day following the Usability Test Mock Election, one voting system that was used in the test shall be available for members of the public to test the security of the voting system by attempting to compromise it.

a. County Board staff who worked as Test Staff at the Usability Test Mock Election may serve as Test Staff for this test. If an insufficient number of such Test Staff are available, members of the public who observed the training of Test Staff may act as Test Staff.

b. Members of the public who did not observe the training of Test Staff may act as Test Election Inspectors, Test Voters, and Test Observers.

c. Two Test Observers shall use a recording device such as a Cam Corder to make a permanent film with the visual and auditory image of all activities at this test.

--------------------------------------------------------------------------------------------------------

REASON FOR NEW SUBSECTIONS

SUBSECTION D. USABILITY TEST MOCK ELECTION

If systems are going to fail, or are unusable by the kind of people who will be county board of elections staff, election inspectors, or voters, it is better to discover this before certification. This is when vendors are most highly motivated to fix the systems or simplify their use. It is best to reveal and correct weaknesses before purchase and rollout of the equipment in an election, which would cause chaos and lawsuits.

Many members of the public distrust electronic voting systems. If the systems work, the public should see this before cynicism about the equipment lowers voter turnout.

To maintain our bipartisan administration of elections, vendor technicians must serve in an advisory capacity only. Vendor technicians must not perform tasks on behalf of bipartisan elections staff. Therefore vendors must fully train sufficient staff appropriate to fully manage the number of systems acquired. The Usability Test Mock Election enables the State Board to determine that the vendor is capable of providing such training.

E. PROFESSIONAL HACKING TEST and F. PUBLIC HACKING TEST

If a voting system is easily tampered, it is better to discover this before certification. This is when the vendor will be mostly highly motivated to fix it.