Comments on the New York State Board of Elections’ Draft Voting Systems Standards

by Teresa Hommel

http://www.wheresthepaper.org/RegsNov4Comment.htm

See also, Election Reform and Modernization Act (“ERMA”) with comments

http://www.wheresthepaper.org/ERMA_45Comments.htm

Draft regs:

http://www.elections.state.ny.us/hava/machine-cert-6209.pdf

Hearing schedule:

http://www.elections.state.ny.us/portal/page?_pageid=153,56962&_dad=portal&_schema=PORTAL

Comments embedded in the regs are in bold Times Roman.

The regs are in Ariel Plain.

Overall Comments

To date, the question "how do you know it's working?" when asked about electronic voting systems, has always been answered with some version of "I trust the computer." This draft of regulations for Voting Systems Standards, both in form and content, is one more version of that answer. This document is shoddy and unprofessional, and bodes ill for the quality of New York’s future election equipment, especially DREs.

1. The low starting fee of $5000 and the large amount of required paper documentation suggest that State qualification will consist primarily of one person spending a week familiarizing him or herself with the equipment as presented by the vendor.

2. The entire process is defined by information presented by the vendor. The starting assumption is that the machines work and the vendor will explain how. This is improper.

Arms-length evaluation is needed. This would include a mock election using a real ballot such as from the November 2004 election; a “stress test” of the maximum number of voters that the system will ever have to handle on one election day; entry of all possible vote combinations; use of all devices including the touch screen or pushbuttons, all accessible devices, minority language interfaces, and the printer; extraction of the end-of-day information from the system; and a complete audit of results and logs created by the DRE.

Arms-length evaluation would include examination of all internal and external memory of all kinds including all files, programming, operating system code, and any other memory contents, as well as a "red test" in which skilled and knowledgeable professionals and activists attempt to subvert the system.

Evaluation of voting systems can reduce the discovery of malfunctions on election day but cannot ensure election integrity for two reasons. First, because New York State statute allows communication capability in electronic voting systems, no evaluation of systems can ensure that tampering will not occur via remote access to systems during elections. Second, no evaluation of any large computer system can guarantee that it is error free (this has been stated frequently by computer scientists such as Dr. Avi Rubin).

For these reasons, regardless of the process of state approval of electronic voting systems, all electronic elections need to be fully audited by hand-to-eye count of 100% of the votes on the VVPAR (voter-verified permanent audit record) and reconciliation of discrepancies between computer and manual tallies.

3. The regs shut out voters and citizens, who are in fact the primary stakeholders in elections, from observation and participation in voting system evaluation, and do not even define them as "users" of the system.

4. The statute and regs appear not to require federal certification as a precondition of state certification. The language of the regs appears intentionally unclear.

5. The irregular outline form of this Draft appears to result from a hasty cut-and-paste effort to piece together parts of documents from other states (mentioning "wards" as political subdivisions) and other types of products including central-count paper-ballot vote tabulators.

=========================================================================

Subtitle V of Title 9 of the Official Compilation of Codes, Rules and Regulations of the

State of New York is hereby amended by adding thereto a new Part, to be Part 6209 and

to read as follows:

SUBTITLE V

Part 6209

Voting Systems Standards

Section 6209.1 Definitions. The terms used in this part shall have the significance

herein defined unless another meaning is clearly apparent in language or content.

Section Comment--Definitions can be political.

--Several definitions are misleading (see comments embedded in the regs below).

--The following 7 defined terms are not used in these regs, and appear to have been copied from other documents, such as for central-count optical scanners.

6. Election Management Software (EMS) ….

7. Encrypted copy ….

8. Escrow account ….

11. Operational manual ….

17. Resident vote tabulation programming ….

17a. Resident memory ….

25. VVPAT ….

1. Acceptance test means a test conducted by the county board of elections and the State

Board of Elections, to demonstrate that the voting system software as delivered and

installed in the user's environment, meets all of its functional requirements.

Comment 1--By definition, the "acceptance test" will require that the software appears to function. By definition "acceptance" will not require evaluation of security or consideration of voter confidence.

2. Auxiliary components means any device, materials or equipment which is used to give

assistance or aid to the actual voting device but is not a permanent or enclosed part of the

voting device.

3. Ballot layout means the positioning of all political party names and emblems, and names

and emblems of all independent bodies, office titles, ballot proposals, and candidate

names, in accordance with the requirements of the Election Law as to order and rotation.

4. DRE means a direct recording electronic voting system which records votes by means

of ballot display provided with mechanical or electro-optical components which are

activated by the voter. Styles include ballot overlay and touch-screen machines.

5. Precinct-based optical scan is a voting system which uses optical-scan technology and

enables voters to cast paper ballots at their respective polling places.

6. Election Management Software (EMS) means the software used by the system to

execute the layout of the ballots.

7. Encrypted copy means a scrambling of the programming code in which only the

manufacturer of the program may determine the sequence of such code.

Comment 2--Nothing in these regs deals with encryption.

8. Escrow account means a third party who shall be approved by the State Board for the

purpose of taking custody of all materials required to be put in escrow by statute.

Comment 3--Statute was unclear and delegated to State Board, but these regs do not deal with escrow.

9. Log of maintenance performance means a written and/or electronic record which

contains all information relating to performance of scheduled and non-scheduled

1

---------------------------------------------------------------------------------------------------------------------------

maintenance requirements recommended by the vendor or manufacturer of such

equipment and all service visits performed by vendor or manufacturer.

Comment 4--Definition should include reference to maintenance performed by BOE staff.

10. Modification means any change in either software, firmware or hardware that directly

affects the operation of the voting system that will require re-examination of certified

equipment by the State Board.

Comment 5--Modifications frequently cause "indirect" effects, especially unexpected effects, which is why ANY modification to software should require re-examination. This definition is circular, because if the modification is held by the SBOE to not require re-examination, then by this definition it is not a modification.

11. Operational manual means (1) a manual of all procedures used to prepare the

equipment and provide proper maintenance procedures including the unpacking and

storage procedures to be utilized by county boards of elections personnel and (2) a manual

of election day setup and election day operating procedures to be utilized by the

inspectors.

12. Pre-qualification test means a predetermined set of votes and vote totals prepared by

the State Board. Such votes shall be entered upon the voting equipment and the results

of the casting of said votes shall be compared to the predetermined results of the test.

Comment 6--Any test requiring votes to be entered needs to require the votes to be entered in the same manner as during an election. This means use of all devices to be used by voters with and without disabilities, and use of all languages intended for voters with minority languages.

13. Printout means the printed copy of (1) zero totals, candidate names and offices and

other information produced by the voting equipment prior to the official opening of the polls

and (2) the votes cast for each candidate and question, the names of candidates and the

offices for each candidate and other information provided after the official closing of the

polls.

14. Hardware means the actual voting or ballot counting device.

15. Software means any and all codes for the operation of the vote counting system.

Comment 7--Since most software operates in conjunction with numerous data files, the contents of which affect and control the operation of the software, all data files in DRE voting systems should be considered part of the software for purposes of inspection and evaluation. These regs ignore all data files, directory (folder) structures, system environmental variables, device drivers, and other non-hardware parts of the voting system which affect the operation of software of the voting system.

Comment 8--This definition seems to have been copied from a document for a central tabulator, since it limits "software" to that used in a "vote counting system." It is unclear whether “vote counting system” is intended to include DREs and all computerized types of equipment that must be examined by the State Board.

16. Firmware means computer program stored in read-only memory devices embedded

in the system and not capable of being altered during system operation.

Comment 9--This definition is misleading for non-technical persons. A computer program is only stored in firmware, where the form in which it is stored cannot be changed.

Before any computer program can be used during system operation, regardless of where or how the program is stored (software, firmware, hard disk, floppy, CD, etc.), the system makes a changeable copy of the program. The copy may be in RAM (Random Access Memory), the computer's CPU (Central Processing Unit), etc. The copy can be altered. The copy is used during system operation. Use of a changeable copy is normal because all programs must be altered during operation but it is not desirable to change the stored original of the program.

During system operation, what any computer program does is affected and controlled by data files, communication from remote sources, the votes entered by the voter, etc. Therefore it is important for these regs to clearly distinguish between the storage of a program in firmware, and the use of a changeable copy of the same program during system operation. Moreover, all votes, ballot images, and tallies will alterable.

17. Resident vote tabulation programming means the manufacturer's internal firmware

program which shall permanently reside on the voting system's central processing unit,

registering, accumulating, and storing votes and ballot images.

Comment 10--See comment 9. Although programming may be stored as firmware, the copy used during system operation is always changeable. Moreover, the regs should recognize that votes, ballot images, and any accumulations will be alterable.

Comment 11--“central processing unit” may be unnecessarily limiting.

17a. Resident memory means the internal memory of the voting system that stores

election results and ballot images.

18. Source code means the assembly language statements or high level language used

to program the electronic equipment or vote tabulating system.

Comment 12--This definition contains a reference to "vote tabulating system" rather than voting system.

19. Specific environmental conditions mean and shall include the effect of natural

environmental conditions such as: temperature, humidity, dust and induced environmental

conditions such as handling, storage or transportation which many affect the operation of

the equipment.

2

---------------------------------------------------------------------------------------------------------------------------

20. State Board means the New York State Board of Elections.

21. Testing laboratory means a certified private or public laboratory used to perform tests

on the voting systems and related equipment.

Comment 13--Neither the statute nor these regs clearly require voting systems that are approved by the State Board to be federally certified. This definition reveals only that testing laboratories may be private or public. Section 6209.6 below (on Examination Criteria) states in paragraph C. that "All laboratory testing shall be conducted or verified by independent testing authorities accredited by the EAC." However, using the federally accredited lab does not mean performing the federal certification tests. Indeed, paragraph C. continues "Testing shall be performed in conformity with written procedures adopted by the State Board" which may be minimal if this document is any indication. Moreover, paragraph B. of the same section specifies "The State Board may, at its discretion, waive any part of, or all of, the analysis and test requirements ..."

22. Vendor shall include any manufacturer, company or individual who seeks to sell voting

systems in New York State.

23. Voting position means an area or square on the voting equipment used to place the

candidate's name, office or political party or independent body designation, or the

placement of ballot amendments and propositions.

24. Voting system means any electronic or computerized voting equipment and any

ancillary equipment supporting such system.

Comment 14--ERMA specifically allows the use of DREs and Optical Scan voting systems, so the regs should name them also and not define "voting system" as computerized and ancillary equipment.

25. VVPAT means a voter verifiable paper audit trail.

Comment 15--ERMA uses the terms "voter verifiable audit record" and "voter verified permanent paper record.”

26. Tactile discernible controls means a voting feature which allows persons with limited

reach and/or hand dexterity, the ability to cast their vote.

Comment 16--such controls are also needed by voters who are blind or visually impaired.

27. Audio voting feature means a device that allows blind or visually-impaired persons, or

persons with limited reach and/or hand dexterity, the ability to cast their vote.

28. Sip and puff voting attachment means a device operated by pneumatic switch which

allows persons with certain disabilities the ability to cast their vote.

29. Election Assistance Commission is the commission established by Help America Vote

Act of 2002, which serves as a national clearinghouse of information and reviews of

procedures with respect to the administration of federal elections.

30. Paper-based Ballot Counting Equipment means any electronic or computerized ballot

counting system or equipment which tabulates and reports votes cast on all paper ballots.

Comment 17--there are other types of equipment for counting ballots, such as weighing scales, that should not be eliminated from consideration or use by these regs.

31. Certification Test Desk means a pre-audited group of ballots marked with a

predetermined number of votes cast for each candidate, write-in position and each voting

option which appears on the ballot.

Comment 18--Test decks should be able to be used with both PBOS and DRE systems, although the testing personnel would have to manually enter the votes from the test deck into the DRE touch-screen or pushbuttons, or into the accessible devices of the DRE.

Section 6209.2 Polling Place Voting System Requirements

Section Comment--This section specifies basic functions required for voting systems.

--Systems will have to comply with the EAC's Voluntary Voting System Guidelines which are scheduled for availability late in 2007 after New York's equipment will have been purchased.

--The full face ballot requirement is retained.

--5-year batteries are required.

--Automated testing capability is required, foreshadowing its use and the avoidance of testing by people (see Comment 26).

--Dust and moisture are mentioned twice.

--Some accessibility requirements of ERMA are explicitly listed.

--Noise level and curtains are discussed.

A. In order for a polling place voting system to be considered by the State Board for

certification, it must comply with the mandates of New York State Election Law, and the

Election Assistance Commission's Voting System Guidelines, and meet the following

requirements:

Comment 19--The EAC's Voting System Guidelines are voluntary in federal law, but this makes them requirements for New York State. However, these Guidelines are scheduled to be available in late 2007, after any new equipment acquired by New York will have been purchased.

(1) Provide a full ballot display on a single surface.

3

---------------------------------------------------------------------------------------------------------------------------

(2) Provide a device which produces and retains a voter-verifiable permanent paper

record, pursuant to statute, which the voter can review and/or correct prior to the casting

of their vote.

Comment 20--with PBOS, the ballot marked by the voter is also the voter-verifiable permanent paper record.

(3) Provide a device or means by which the votes cast on the machine can be

printed or recorded or visually reviewed after the polls are closed.

Comment 21--with PBOS, ballots marked by voters are the means by which votes cast on the machine can be visually reviewed after the polls are closed.

Comment 22--It is unclear what purpose is to be served by printing of “votes cast on the machine” after the polls are closed, since the VVPAR should serve any need to review the votes cast on DREs. ERMA requires locks that prevent recording votes after the polls are closed (Page 5, Lines 23-27)

23 G. BE PROVIDED WITH A LOCK OR LOCKS, OR OTHER DEVICE OR DEVICES, THE

24 USE OF WHICH, IMMEDIATELY AFTER THE POLLS ARE CLOSED OR THE OPERATION OF

25 THE MACHINE OR SYSTEM FOR SUCH ELECTION IS COMPLETED, WILL ABSOLUTELY

26 SECURE THE VOTING OR REGISTERING MECHANISM AND PREVENT THE RECORDING OF

27 ADDITIONAL VOTES;

Perhaps paragraph (3) intended to require the printing, recording or visually reviewing of TALLIES of votes cast?

(4) Provide a battery power source in the event that the electric supply used to make

the voting system equipment function if disrupted. Such batteries must be rechargeable

and have minimum five-year life when used under normal conditions.

Comment 23--typo, “if disrupted” should be “is disrupted”

Comment 24--typo “have a minimum”

Comment 25--More detail is needed here, because few batteries have a 5-year life. Will this requirement mean that all electronic voting systems must be connected to an electrical source when they are in storage, thus creating a huge expense for counties that use them? Or that counties will have to buy rechargers and regularly have staff go to the storage warehouse and remove, recharge, and reinsert the batteries?

(5) The system shall contain software and hardware required to perform a diagnostic

test of system status, and a means of simulating the random selection of candidates and

casting of ballots in quantities sufficient to demonstrate that the system is fully operational

and that all voting positions are operable.

Comment 26--This requirement is self-contradictory. Software cannot simulate votes entered by humans, which require touch pressure on the touch screen or pushbuttons, use of accessible devices, use of the printer, and viewing of the ballot in minority languages. “Fully operational” also requires a person to extract the memory cartridge which is supposed to contain the votes and tallies at the end of the election day, and confirm that it contains accurate tallies and an accurate record of the votes cast. The only way to "demonstrate that the system is fully operational and that all voting positions are operable" is to have humans interact with a DRE voting system or an accessible ballot marking device, feed marked ballots into an optical scanner, etc.

(6) The system shall be designed to protect against dust and moisture during

storage and transportation.

B. In addition to the requirements of subdivision (a) of this section, fully-accessible voting

equipment certified by the State Board shall meet the following requirements for usability

by voters who are disabled:

Comment 27--subdivision (a) probably means subdivision A. above.

(1) The equipment shall be equipped with a voting device with tactile discernable

controls, pursuant to statute.

(2) Equipment shall be equipped with an audio voting feature, pursuant to statute.

(3) Equipment must be capable of being equipped with voting device of a sip and

puff technology nature, pursuant to statute.

Comment 28--typo, “equipped with a”

Comment 29—Note, tactile and audio must be standard features, sip and puff is not.

C. Standards for noise level

(1) Voting equipment to be certified by the State Board shall be constructed in a

manner so that noise levels of the equipment during operation will not interfere with the

duties of the election inspectors or the voting public.

(2) The noise level of write-in components of the equipment shall be so minimal that

it will be virtually impossible under normal conditions for someone at the table used by the

inspectors of elections to determine that a write-in vote is being cast or has been cast.

Comment 30--This suggests a problem that has not been previously discussed in New York, that computerized voting equipment is noisy and makes a noise when someone is entering a write-in.

D. Standards for curtain design

(1) Voting equipment curtains shall be constructed so that no one within the polling

4

---------------------------------------------------------------------------------------------------------------------------

site will be able to see how a voter is casting a vote.

(2) Curtains shall be so designed as to allow any voter, either electronically or

manually, to open and close the curtain with ease when entering and exiting the equipment

without obstruction.

Comment 31--Curtains are not required, and should not be required unless needed.

E. Environmental Standards

Voting systems shall be capable of withstanding reasonable levels of exposure to

dust, rain and humidity during storage, transport and use.

Section 6209.3 Paper-based Voting Systems

Section Comment—This section’s functional requirements should be applied to DRE systems also.

--Paragraph A. (1) is of questionable usefulness and may be inappropriate for paper-based voting systems unless the Election District is noted on the ballot, and for primaries, the party of the voter.

--Paragraphs A. (2) through F, H, and I require prevention of over votes and improper votes, and require tallies by district. It is not clear why these requirements have been applied only to paper-based voting systems and not to DREs as well.

--The meaning of Paragraph G. is unclear.

--Paragraph J. allows standard PC and peripheral equipment to be used with only functional testing, which does not ensure that it does not include malicious code or errors.

--Ballot specifications for paper ballots are listed under paragraph K. If tallies by Election District are required, then the requirement that ballots contain machine and manually readable coding to identify ballot style must also require identification of the Election District.

A. In addition to voting system requirements provided for elsewhere in these rules and

regulations, paper-based systems must

(1) mechanically or electronically prevent a voter from voting for candidates or ballot

proposals for whom or which he or she is not entitled to vote.

Comment 32--ERMA, Page 5 lines 5-7, specifies that voting systems shall:

5 C. BE CONSTRUCTED SO THAT A VOTER CANNOT VOTE FOR A CANDIDATE OR ON A

6 BALLOT PROPOSAL FOR WHOM OR ON WHICH HE OR SHE IS NOT LAWFULLY ENTITLED

7 TO VOTE;

The reg modifies the law by requiring paper-based voting systems to do this "mechanically or electronically." Such requirement may not be appropriate for paper ballot systems because the poll worker or Board of Elections simply gives the voter the correct ballot. This requirement is appropriate for DREs where the voter needs a smart card to activate the DRE. For example, in primary elections, the poll worker provides a smart card encoded with the voter's party; the DRE then displays the ballot for that party. With paper ballot systems, however, the poll worker simply gives the voter the correct paper ballot with the races for the voter's party.

(2) be able to prevent a voter from

(a) Over-voting

(b) Voting for the same person more than once for the same office or position

(c) Voting for candidates of another party in a primary election

Comment 33--Repeat comment 32.

B. The system may not count any votes for an office or ballot proposal which has been

over-voted or otherwise improperly voted.

C. An over-vote in one or more office or ballot proposals shall not prevent the counting of

all other offices or ballot proposals contained on the ballot.

D. In the case of candidates who appear on one or more party lines, the system must be

capable of correctly counting the vote according to provisions of Election Law S 9-112. The

system may not count votes

Comment 34--Missing end of last sentence.

E. In vote counting, the system shall ignore any mark on a ballot unless that mark is in a:

(1) voting position for a candidate whose name is on the ballot;

(2) voting position designated for write-in voting for a write-in candidate; or

(3) voting position for a ballot proposal.

5

---------------------------------------------------------------------------------------------------------------------------

F. The system shall provide a method for write-in voting and shall report the number of

votes cast in each contest in write-in voting positions.

G. The system shall provide a means by which the software may be positively verified to

insure that it corresponds to the format of the ballot face.

Comment 35--"insure" should be "ensure"

Comment 36--Paragraph G. is unclear. Must the software be able to report the lot number of the ballot face? Or must the software be able to self-test that the ballot face is correctly programmed? If the latter, correctness must be tested by people, as described in comment 26.

H. The system shall be capable of accumulating and reporting a count of the number of

ballots tallied for an election district and shall be capable of separating and tabulating those

election district totals to produce a report of the total of ballots tallied by groups of election

districts such as legislative districts, wards, etc.

Comment 37--Apparently this was copied from another state that has wards.

Comment 38--It should be easily possible to produce tallies of ballots per election district and various legislative districts if the system knows these districts. The voter's election district would have to be noted on the ballot for OpScan systems, and on the smart card for DREs.

I. The system shall be capable of accumulating and reporting by election district the total

votes cast for each candidate and the total vote for or against each ballot proposal. The

system shall also be capable of tabulating and reporting the vote cast for each candidate

and the vote cast for or against each ballot question by groups of election districts such as

legislative districts, wards, etc.

Comment 39--Apparently this was copied from another state that has wards.

J. Qualification tests for paper-based voting systems shall not be required for the following

types of equipment, and their suitability for elections use shall be determined by functional

tests which integrate them with the remainder of the system:

(1) Standard production models of general purpose data processing equipment

(PC'S, printers, etc.) shown to be compatible with these requirements and with the paper

ballot voting system.

(2) Production models of special purpose data processing equipment (scanners, bar

code readers, etc.) having successfully performed in elections use and having been shown

to be compatible with the paper ballot voting system.

Comment 40--How would anyone know whether a PC, scanner, bar code reader, etc. is a standard production model or has been altered? How can anyone know whether such equipment is compatible, has performed successfully in elections use? All components of all electronic equipment that is part of an election system should be examined and tested.

K. Ballot specifications:

(1) All ballots shall meet the specifications as to form and content required under

section 7-122 of the Election Law.

(2) Ballots shall be printed in black ink on white paper or on paper stock of different

colors to identify different types of ballots (i.e., emergency, affidavit, etc) or in the case of

a primary, to identify ballots for each political party according to the color assigned to such

party pursuant to law.

(3) Coding which is both machine readable and manually readable shall be used to

identify different ballot styles.

Comment 41--All machine readable coding should ALSO be human-readable, to ensure that the machine readable coding does not contain improper information. Also, since the same ballot style may be used in many Election Districts, the Election District needs to be coded on the ballot.

(4) Ballots used in the system shall be able to be counted by hand as well as be

counted by machine. The system shall provide an audit trail of all ballots cast, making

possible the reconstruction of the election, starting with the individual votes of all eligible

6

---------------------------------------------------------------------------------------------------------------------------

voters, in the case of a recount.

Comment 42--For PBOS systems the "audit trail" consists of the original paper ballots marked by voters. It is unclear what is meant by "reconstruction of the election." It is unclear why this requirement is being made, since this is one of the primary advantages of PBOS systems.

(5) The types of ballots used and their form, type size and arrangement must be

approved by the State Board of Elections.

L. Where a paper-based system is used for the central counting of absentee, affidavit,

emergency and special ballots, the requirements of 6209.2 do not apply.

Section 6209.4 Application Process

Section Comment--Vendors must complete a "pre-qualification test" consisting of programming two ballot faces, one each for a general and primary election, and then using them to accurately count some votes. The application consists of this task, printouts of tallies, lots of documentation specified in later sections, and $5000.

A. The Election Operations Unit shall forward an application form, upon request, to any

vendor, together with a copy of applicable rules and regulations and a pre-qualification test

format for both a general and primary election ballot program.

Comment 43--The regs should require that the form, rules and regs, and test formats be forwarded to the vendor within a specified number of days.

Comment 44--In paragraph B. below the vendor then programs the design into their equipment.

B. Said vendor shall return completed ballot layouts based upon the pre-qualification test

format to the Election Operations Unit. Upon approval of the layouts, the vendor shall

program such equipment and complete the pre-qualification tests for both ballot programs

provided, and enter the simulated votes upon said equipment for each election program.

C. The completed application shall be returned, with a printout of tabulated votes from the

primary and general election pre-qualification tests as cast on the voting system equipment

which the applicant requests to have certified. The pre-qualification test programs shall be

retained by the applicant for use in the certification process.

D. The application and printouts shall be reviewed to determine if the voting system shall

be considered for certification and the applicant shall be notified of such determination.

Comment 45--The regs should require that the response to the vendor must occur within a specified maximum number of days.

E. No application shall be deemed to be filed until all documentation required by these

rules has been submitted to the State Board or its designee.

F. A certified or bank check in the amount of $5,000 shall accompany such application,

and be applied towards the actual cost of examination.

G. Fees for the examination of a voting system shall be assessed against the vendor by

the State Board based upon the cost to the State Board for examination of such voting

system by an outside contractor, laboratory or other authorized examiner, but the fees

assessed shall not exceed the amount permitted by statute.

Section 6209.5 Submission of Voting Systems Equipment.

Section Comment--Vendors must turn in a complete voting system which they must maintain, which the State Board will retain.

A. Voting systems considered for certification by the State Board shall be delivered to the

State Board or its designee. Such equipment shall include auxiliary components and

equipment used to program ballot layout, and any other additional equipment used in the

operation of said voting system.

7

---------------------------------------------------------------------------------------------------------------------------

B. If the voting systems equipment is certified by the State Board, the specific equipment

and components examined by the State Board shall become the property of the State

Board for as long as the equipment is in use in the State or for such shorter period as the

State Board shall so determine. Voting systems not certified shall be disposed of pursuant

to the vendor's direction.

C. The applicant shall provide service and normal maintenance of said equipment after

certification and shall supply to the State Board, at no cost, any modification to the

equipment for upgrading of any feature during the period that said equipment is offered for

sale and use in the State.

Section 6209.6 Examination Criteria

Section Comment--This long section lists documentation that vendors must supply, as well as manuals for maintenance and repair.

--The State Board may submit the system for lab analysis, or waive any tests if the vendor submits "certified test data and reports" which is not defined.

--EAC-accredited labs will be used, but federal requirements will not have to be met, only NY State procedures which shall be available for public inspection at an unspecified time.

--Qualification will consist of "tests, code analyses, and inspection tests" to verify that software and hardware meet the vendor's own design requirements for functionality and physical parts, and that vendor documentation is correct.

--The "Functional Configuration Audit" verifies that the software is accurately described by the vendor's documentation. The State Board will do additional tests to verify NOMINAL system performance and validate a SAMPLE of the vendor's tests.

--The "Physical Configuration Audit" examines the software and documentation associated with the hardware to establish a baseline. The State Board might decide that some future changes to this baseline of hardware or software require re-evaluation.

--Please see especially Comments 52, 53, 54, 55, and 61.

A. The State Board or its designee, as part of its examination, may at its discretion, submit

the voting system for a laboratory analysis.

B. The State Board may, at its discretion, waive any part of, or all of, the analysis and test

requirements contained in subdivision (e), upon submission by the vendor of certified test

data and reports which verify system performance in a manner equivalent to the Board's

examination requirements.

Comment 46--Designation of "subdivision (e)" appears to be a typo, so it is unclear what requirements may be waived. Due to vagueness, it is unclear what "certified test data and reports" means.

C. All laboratory testing shall be conducted or verified by independent testing authorities

accredited by the EAC. Testing shall be performed in conformity with written procedures

adopted by the State Board and such procedures shall be available for public inspection.

Comment 47--The regs should require the procedures to be posted on the State Board web site as soon as they are adopted or available to testing laboratories. The public should not get them earlier or later than any vendor, laboratory or other party, nor have to comply with any special process in order to inspect them.

1. Software and Hardware Qualification Tests

Qualification of voting system software and hardware shall consist of a series of tests,

code analyses, and inspection tests performed at the federal level, to verify that the

software and hardware meet design requirements and that characteristics are correctly

described in the documentation items. Qualification shall also include a Functional

Configuration Audit and a Physical Configuration Audit.

Comment 48--EAC-accredited labs will determine whether the vendor's "design requirements" are met, and system characteristics correctly documented. In other words, the State Board may certify systems that have not been independently tested. The state's "functional and physical configuration audits" are described below.

A. Functional Configuration Audit

A functional configuration audit shall be performed to verify that the software complies

with the Software Specification. Vendor test data may be used in partial fulfillment of this

requirement; however, the State Board or its designee shall perform or supervise the

performance of additional tests, or order additional laboratory testing, to verify nominal

system performance in all operating modes and to validate, on a sampling basis, the

vendor's test data reports. The Functional Configuration Audit shall be performed in a

facility selected by the State Board.

Comment 49--The Software Specification is the vendor's design standards and conventions, environment and interface specifications, functional specifications, programming architecture specifications, and test and verification specifications.

Comment 50--Only "nominal" system performance and a sampling of vendor tests will be validated.

(1) Vendor Support

8

---------------------------------------------------------------------------------------------------------------------------

The vendor shall provide a list of all documentation and data to be audited and vendor

technical personnel shall be available to assist in the performance of the Functional

Configuration Audit.

Comment 51--The vendor controls the entire audit by providing the list of documentation and data to be audited.

(2) Technical Data

The vendor shall provide the following technical data:

(a) copies of all procedures used for module or unit testing, integration testing

and system testing;

(b) copies of all test cases generated for each module and integration test

and sample ballot formats or other test cases used for system;

(c) records of all tests performed by the procedures listed above, including

error correction and retest.

(3) Audit Procedure

The State Board or its designee shall review the vendor's test procedures and test

results.

This review shall include an assessment of the adequacy of test cases and input data

to exercise all system functions and to detect program logic and data processing errors if

such be present.

The review shall also include an examination of all test data which is to be used as

a basis for qualification.

Comment 52--The work described here cannot prove that these machines are trustworthy for unaudited use. First, a functional test does not inspect the entire system for malicious or insecure parts, or relationships of parts. Second, the process relies upon vendor-supplied information rather than independent investigation of the entire system. Third, the "adequacy of test cases" cannot be determined by functional assessment, but must be based on knowledge of all programming logic. The enormous number of electronic voting system failures that occur on election days are in part due to the limitation of federal certification testing which is also a functionality test, much of it automated. Fourth, automated tests, as described here, do not test the parts of the system interacted with by voters on election day; this leaves voters to discover failures of the touch screens or pushbuttons, printers, the accessible devices, and the display of the ballot in minority languages. It leaves poll workers to discover failures of the tallies cartridges that cannot be read to "extract" the day's tallies.

B. Physical Configuration Audit

(1) The Physical Configuration Audit is an examination of the software configuration

against its technical documentation to establish a configuration baseline for approval. The

Physical Configuration Audit shall include an audit of all drawings, specifications, technical

data and test data associated with the system hardware and this audit shall establish the

system hardware baseline associated with the software baseline. All subsequent changes

to the software baseline configuration shall be subject to re-examination. All changes to

the system hardware which may result in a change in the operation of the software shall

also be subject to re-examination.

Comment 53--This audit establishes the "baseline" of functionality and documentation. Changes can be made without re-examination of the system as long as the changes aren't reflected in this superficial view of the system which is entirely under vendor control. In other words, re-examination is not needed if the software changes as long as its "configuration" doesn't change. Hardware can change as long as it doesn't change the operation of the software. Who says? The Vendor. The State Board will not look at parts of the voting system that the vendor does not present to them.

(2) Vendor Support

The vendor shall provide a list of all documentation and data to be audited and vendor

technical personnel shall be available to assist in the performance of the Physical

Configuration Audit.

9

---------------------------------------------------------------------------------------------------------------------------

(3) Technical Data

The vendor shall provide the following technical data:

(a) identification of all items which are to be a part of the software release;

(b) identification of all hardware which interfaces with the software;

(c) configuration baseline data for all hardware which is unique to the system;

(d) copies of all software documentation which is intended for distribution to

users, including program listings, specifications, operator manual, user manual and

software maintenance manual;

(e) user acceptance test procedure and acceptance criteria;

(f) an identification of any changes between the Physical Configuration Audit

and the configuration submitted for the Functional Configuration Audit (FCA) and a

certification that these differences do not degrade the functional characteristics.

Comment 54--Both "audits" are vendor-managed and based on trust of the vendor rather than arms-length evaluation of the system, and are entirely unprofessional and improper. The regs do not define "a certification" and specify from whom or what institution.

(4) Audit Procedure

Required data items include draft and formal documentation of the vendor's software

development program which are relevant to the design and conduct of Qualification Tests.

The vendor shall identify all documents, or portions of documents, which contain

proprietary information not approved for public release. The State Board or its designee

shall agree to use the information contained therein solely for the purpose of analyzing and

testing the software and shall refrain from disclosing proprietary information to any other

person or agency without the prior written consent of the vendor. At the conclusion of the

examination, the State Board or its designee shall return to the vendor all such

documentation and shall not retain any copies thereof. The State Board or its designee

shall review the vendor's source code and documentation to verify that the software

conforms to the documentation, and that the documentation is sufficient to enable the user

to install, validate, operate and maintain the voting system. The review shall also include

an inspection of all records of the baseline version against the vendor's release control

system to establish that the configuration, being qualified, conforms to the engineering and

test data.

Comment 55--The procedure is vendor-managed and based on trust of the vendor rather than arms-length evaluation of the system, and is entirely unprofessional and improper.

The vendor decides what is "relevant."

C. Functional Tests

(1) For all equipment, functional tests should consist of validation of equipment

functional performance by means of procedures under "Laboratory Environmental Test

Procedures for Hardware and Software".

(2) Functional tests of voting system software which runs on general purpose data

10

---------------------------------------------------------------------------------------------------------------------------

processing equipment shall include all tests similar to those in procedures which are

necessary to validate the proper functioning of the software and its ability to control the

hardware environment. The tests shall also validate the ability of the software to detect

and act correctly upon any error conditions which may result from hardware malfunctions.

Detection capability may be contained in the software, the hardware or the operating

system. It shall be validated by any convenient means up to and including the introduction

of a simulated failure (power off, disconnect a cable, etc.) in any equipment associated with

vote processing.

Comment 56--Again, functional tests do not test the entire system, and cannot discover all errors nor most security weaknesses. If the State Board were serious, they would open the system to public testing, enter a maximum number of ballots, and examine the results.

2. Software, Hardware, Operating and Support Documentation

(A) Software Qualification

The following system software and firmware vendor data items shall be submitted as

a precondition of certification of acceptability for elections use.

(B) Vendor Documentation

Complete product documentation shall be provided to the State Board for voting

systems, their components and all auxiliary devices. This documentation shall be sufficient

to serve the needs of the voter, the operator and the maintenance technician. It shall be

prepared and published in accordance with standard industrial practice for electronic and

mechanical equipment such documentation shall include:

Comment 57--Rather than "complete" the documentation must be merely “sufficient” for voters, operators, and maintenance technicians.

(1) Software Specification

The Software Specification shall contain and describe the vendor's design standards

and conventions, environment and interface specifications, functional specifications,

programming architecture specifications, and test and verification specifications. Pre-

factory material should include document identification, an abstract of the specification,

configuration control status and a table of contents. The body of the specification shall

contain the following material:

Comment 58--What is “pre-factory material”?

(a) System Overview

The vendor shall identify the system hardware and the environment in which the

software will operate and the general design and operational considerations and

constraints which have influenced the design of the software.

(b) Program Description

The vendor shall provide descriptions of the software system concept, the array of

hardware in which it operates, the intended operating environment, the specific software

design objectives and development methodology and the logical structure and algorithms

used to accomplish the objectives.

11

---------------------------------------------------------------------------------------------------------------------------

(c) Standards and Conventions

The vendor shall provide information which can be used as a partial basis for code

analysis and test design. It should include a description and discussion of the standards

and conventions used in the preparation of this specification and in the development of the

software.

(d) Specification Standards and Conventions

The vendor shall identify all published and private standards and conventions used

to document software development and testing. Vendor internal procedures shall be

provided as attachments to this Software Specification.

(e) Test and Verification Standards

The vendor shall identify any standards or other documents which are applicable

to determination of program correctness and acceptance criteria.

(f) Quality Assurance Standards

The vendor shall describe all standards or other documents which are applicable

to the examination and testing of the software, including standards for flowcharts, program

documentation, test planning and test data acquisition and reporting.

(g) Operating Environment

The vendor shall provide a description of the system and subsystem interfaces at

which inputs, outputs and data transformations occur. It shall contain or make reference

to all operating environment factors which influence the software design.

(h) Hardware Constraints

The vendor shall identify and describe the hardware characteristics which influence

the design of the software, such as:

(1) the logic and arithmetic capability of the processor,

(2) memory read/write characteristics,

(3) external memory device characteristics

(4) peripheral device interface hardware data I/O device protocols, and

(5) operator controls, indicators and displays.

12

---------------------------------------------------------------------------------------------------------------------------

(i) Software environment

The vendor shall identify the compiler or assembler to be used for the generation

of executable code and a description of the operating system or system monitor. This

section shall also contain an overview of the compile-time interaction of the voting system

software with library calls and linking.

(j) Interface Characteristics

The vendor shall describe the interfaces between executable code and system

input-output and control hardware.

(k) Software Functional Specification

The vendor shall provide a description of the overall functions which the software

performs in the context of its mode or modes of operation. The vendor shall also describe

the capabilities and methods for detecting and handling exceptional conditions, system

failure, data input/output errors, error logging and audit record generation and security

monitoring and control.

(l) Configurations and Operating Modes

The vendor shall describe the various software configurations and operating modes

of the system; such as preparation for opening of the polling place, vote recording and/or

vote processing, closing of the polling place and report generation. For each software

function or operating mode, a definition of the inputs (characteristics, tolerances or

acceptable ranges) to the function or mode, how the inputs are processed and what

outputs are produced (characteristics, tolerances or acceptable ranges) shall be provided.

(m) External files

In the event that external files are used for data input or output, the definition of

information context and record formats shall be provided. The vendor shall also describe

the procedures for file maintenance, access privileges and security.

(n) Security

Security requirements and security provisions of the software shall be identified for

each system function and operating mode.

(o) Programming Specifications

The vendor shall provide an overview of the software design, structure and

implementation algorithms. Whereas the Functional Specification of the preceding section

provides a description of what functions the software performs and the various modes in

13

------------------------------------------------------------------------------------------------------------------------------

which it operates, this section should be prepared so as to facilitate understanding of the

internal functioning of the individual software modules. Implementation of functions shall

be described in terms of software architecture, algorithms and data structures and all

procedures or procedure interfaces which are vulnerable to degradation in data quality or

security penetration shall be identified.

Comment 59--The regs require a vendor-managed rather than arms-length process.

(p) Test and Verification Specifications

The vendor shall describe the procedures used during software development to

verify logical correctness, data quality and security. This description shall include existing

standard test procedures, special purpose test procedures, test criteria and experimental

design and validation criteria. In the event that this documentation is not available, the

Qualification Test agency shall design test cases and procedures equivalent to those

ordinarily used as a basis for in-house verification (see below).

(q) Qualification Test Specification

The vendor shall provide a specification for verification and validation of overall

software performance, including acceptance criteria for control and data input/output,

processing accuracy, data quality assessment and maintenance, exceptional handling and

security. The specification shall identify specific procedures by means of which the general

suitability of the software for elections use can be assessed and demonstrated. The

vendor's specification and procedure shall be used to establish the detailed requirements

of the tests described in "Laboratory Environmental Test Procedures for Hardware and Software" of this Standard.

Comment 60--The regs require a vendor-managed rather than arms-length process.

(r) Acceptance Test Specification

The vendor shall provide a specification for installations, acceptance and readiness

verification. This specification shall identify specific procedures by means of which the

capability of the software to accommodate actual ballot formats and format logic, and pre-

election logic, accuracy and security test requirements of using jurisdictions may be

assessed and demonstrated. The vendor's specification shall be used to establish the

detailed requirements of the tests described in "Laboratory Environmental Test Procedures

for Hardware and Software" of this standard performed to evaluate the adequacy of the

vendor's procedures and it shall be suitable for inclusion in the regulations and procedures

of user counties when preparing for the conduct of actual elections.

Comment 61--This vendor-directed rather than arms-length process will now become the requirement for counties, instead of than human-entered votes and ballots, etc. Dangerous!

(s) Appendices

The vendor shall provide descriptive material and data supplementing the various

sections of the body of the Software Specification. The content and arrangement of

appendices shall be at the discretion of the vendor. Topics recommended for amplification

and treatment in appendix form include:

14

------------------------------------------------------------------------------------------------------------------------------

(1) Glossary: Provide a listing and brief definition of all software module

names and variable names with reference to their locations in the software structure.

Include abbreviations, acronyms and terms which are either not commonly used in data

processing and software development or which are used in an uncommon semantic

context.

(2) References: Provide a list of references to all related vendor documents,

data, standards and technical sources used in software development and testing.

(3) Program Analysis: Provide the results of software configuration analysis,

algorithm analysis and selection, timing studies and hardware interface studies reflected

in the final software design and coding.

(4) Security Analysis: Provide a detailed description of the penetration

analysis performed to preclude intrusion by unauthorized persons and fraudulent

manipulation of elections data. Identify security policies and measures and selection

criteria for audit log data categories.

Comment 62--Vendor-directed rather than arms-length process.

(2) Operator Information

This documentation shall include a physical description of the equipment sufficient

to identify all features, control and displays. It shall include a complete procedure for

energizing the equipment, for testing and verifying operational status and for identifying all

abnormal equipment states. It shall include a complete operating procedure for inserting

ballots to be tabulated, for controlling the tabulation process, for monitoring the status of

the equipment, for recovering from error conditions and for preparing output reports.

Comment 63--Instructional information, appears to have been copied from a request for information about a paper-ballot-reading vote-tabulator

(3) Maintenance Information

(a) This documentation shall contain a complete physical and functional

description of the equipment and a theory of operation which fully describes the electrical

and mechanical function of the equipment, how the processes of ballot handling and

reading are performed, how data are handled in the processor and memory sections, how

data output is initiated and controlled, how power is converted or conditioned and how test

and diagnostic information is acquired and used.

Comment 64--Instructional information, appears to have been copied from a request for information about a paper-ballot-reading vote-tabulator

(b) A complete parts and materials list shall be provided which contains

sufficient descriptive information to identify all parts by type, size, value or range and

manufacturer's designation.

Comment 65--Appears to have been copied from a request for information about a different type of device.

(c) Technical illustrations and schematic representations of electronic circuits

shall be provided with indications of all test and adjustment points and the nominal value

and tolerance or waveform to be measured. Fault detection, isolation and correction

procedures or logic diagrams shall be prepared for all operational abnormalities identified

by design analysis and operating experiences.

Comment 66--Appears to have been copied from a request for information about a different type of device.

15

------------------------------------------------------------------------------------------------------------------------------

(4) Logistics, Facilities and Training

The vendor shall identify all operating and support requirements of the system or

component. These requirements include material, facilities and personnel, including

furnishings, fixtures, and utilities which will be required to support system operation,

maintenance and storage.

(5) Maintenance Training and Supply

(a) The vendor shall identify all corrective and preventive maintenance tasks

and the level at which they shall be performed. Levels of maintenance shall include

operator tasks, maintenance personnel tasks and factory repair.

(b) Operator tasks shall be limited to the activation of controls to identify

irrecoverable error conditions and to the replenishment of consumables such as printer

ribbons, paper and the like.

(c) Maintenance personnel tasks shall include all field maintenance actions

which require access to internal portions of the equipment. They shall include the conduct

of tests to localize the source of a malfunction; the adjustment, repair or replacement of

malfunctioning circuits or components and the conduct of tests to verify restoration to

service.

Comment 67--appears to be copied from regulations for other kinds of equipment such as desk top computers. One can hardly imagine maintenance personnel in the middle of an election taking apart the voting equipment and replacing circuits.

(d) Factory repair tasks shall be minimized. They shall only include complex

and infrequent maintenance functions which require access to proprietary or to specialized

facilities and equipment which cannot be obtained by using agency. They shall not

number more than two percent of all maintenance tasks and their frequency shall not

exceed five percent of the total frequency for all corrective maintenance tasks.

Comment 68--appears to be copied from regulations for other kinds of equipment, especially the references to an "agency" and the 2% and 5% limits.

(e) The vendor shall identify by function all personnel required to operate and

support the system. For each functional category, the number of personnel and their skills

and skill levels shall be specified.

(f) The vendor shall specify requirements for the training of each category

of operating and support personnel. The vendor shall prepare all materials required in the

training activity and shall provide or otherwise arrange for the provision of qualified

instructors.

Comment 69--appears to be copied from regulations for other kinds of equipment, since the "operating personnel" will be voters, poll workers, and Elections staff.

(g) The vendor shall recommend a standard complement of supplies, spares

and repair parts which will be required to support system operation. This list shall include

the identification of these materials and their individual quantities and sources from which

they may be obtained. The vendor shall supply, at vendor's expense, any special tools

required to repair or maintain the equipment.

Comment 70--appears to be copied from regulations for other kinds of equipment that require spares, repair parts, and tools.

Section 6209.7 Modifications and Re-examination

Section Comment--Read in isolation, this section seems to require modifications to be submitted, approved, reviewed, etc. However, when read in light of the previous section and the description of the baseline, modifications that do not change the baseline of functionality would not have to be submitted, etc.

ERMA requires re-examination when the "operation or material" of any "feature or component" is changed. If the material of a component is the software or hardware of the component, then any change to a voting system would require re-examination. In these regs, the term "material" is used in several paragraphs to describe documentation materials.

ERMA Page 4 lines 9-14:

9 2. When any change is made in the operation or material of any feature

10 or component of any machine OR SYSTEM which has been approved pursuant

11 to the provisions of this section, such machine OR SYSTEM must be

12 submitted for such re-examination and reapproval pursuant to the

13 provisions of subdivision one of this section as the state board of

14 elections deems necessary.

16

------------------------------------------------------------------------------------------------------------------------------

A. Any prospective modification to a previously certified voting system shall be submitted

to the State Board.

B. No modification of previously certified voting systems equipment shall be used in any

election until such modification has been approved by the State Board.

C. Prospective modification shall be reviewed by the State Board or by an examiner or

laboratory of the Board's choice in accordance with the fee schedule established by section

7-201 of the Election Law.

D. Upon completion of a review of such prospective modification, the State Board may

cause a re-examination of the entire voting system, or within its discretion, grant

continuation of certification pursuant to the provisions of section 7-201 of the Election Law.

Section 6209.8 Rescission of Certification

Section Comment--This section fails to elaborate on ERMA by listing criteria for rescission, specifying procedures for notifying the State Board of problems, who would pay for re-examinations, etc.

ERMA, Page 4, Lines 15-30:

15 3. If at any time after any machine OR SYSTEM has been approved pursu-

16 ant to the provisions of subdivision one or two of this section, the

17 state board of elections has any reason to believe that such machine OR

18 SYSTEM does not meet all the requirements for voting machines OR SYSTEMS

19 set forth in this article, it shall forthwith cause such machine OR

20 SYSTEM to be examined again in the manner prescribed by subdivision one

21 of this section. If the opinions in the report of such examinations do

22 not state that such machine OR SYSTEM can safely and properly be used by

23 voters at elections under the conditions prescribed by this article, the

24 state board of elections shall forthwith rescind its approval of such

25 machine OR SYSTEM. After the date on which the approval of any machine

26 OR SYSTEM is rescinded, no machines OR SYSTEMS of such type may be

27 purchased for use in this state. The state board of elections shall

28 examine all machines OR SYSTEMS of such type which were previously

29 purchased, to determine if they may continue to be used in elections in

30 this state.

ERMA's "any reason to believe" standard (line 17) is subjective and can mean anything. After an election in which some 40% of systems failed, one election official proclaimed that nothing could make him lose faith in the computerized voting systems. A voter or poll worker might have reason to believe upon the first system failure, upon seeing a vote switched by the computer to a different candidate on the screen, upon finding that not all races are displayed, etc. A worker counting the votes on the VVPAT might have reason to believe when the electronic count does not match the VVPAT count.

ERMA says that after approval is rescinded, no further purchase is allowed and the State BOE must examine all such machines that were previously purchased (lines 25-30). It is unlikely that the State Board will rescind any machine approval if they then have to examine, for example, a thousand machines. Please see also Comment 71.

A. If at any time subsequent to the State Board's approval of a voting system, the State

Board determines that the voting system fails to fulfill the criteria prescribed by statute and

these rules, the Board shall notify any users and vendors of that particular voting system

that the State Board's approval or certification of that system for future sale of that system

in New York State is to be withdrawn.

Comment 71--It does not seem appropriate that systems can continue to be used by voters after their approval has been rescinded. In requiring notification to "users," the regs should explicitly mention the notification of all voters who have used such system in the previous two elections, and all poll workers who have worked at elections using them. These users would have the most direct experience and can report first-hand details of problems if any occurred. In addition, all candidates and parties who were on the ballots that were voted using the equipment are important stakeholders and users, and the regs should explicitly require them to be notified.

B. Such notice shall be in writing and shall specify the reasons why the approval or

certification of the system is being rescinded. Such notice shall also specify the date on

which the rescission is to become effective.

C. Any vendor or user of such voting system may request in writing that the State Board

reconsider its decision to rescind approval or certification of the voting system.

D. Upon receipt of such request to reconsider, the State Board shall hold a hearing for the

purpose of reconsidering the decision to rescind the approval or certification. Any

interested party shall be given the opportunity to submit testimony or documentation in

support of or in opposition to the Board's decision to rescind approval or certification.

Comment 72--The regs should require publication of notice of such rescission, the reasons for it, publication of the date, time and place of any hearings a minimum of two weeks in advance, and notification by mail to all parties who have requested notification of such State Board activities.

E. The State Board may affirm or reverse its decision.

Section 6209.9 Contracts

Section Comment--This section deals with training, maintenance, evaluation of poll sites, requirements for delivery time, and acceptance testing by counties. Acceptance testing is elaborated in the next section. Delivery deadlines are short, reflecting the short deadlines for compliance with HAVA and the desire to keep HAVA money, but such short deadlines also force the use of equipment that counties may not be prepared for.

A. In addition to complying with all statutory requirements, all contracts for the purchase

of voting systems shall include the following requirements:

(1) Training

Vendors of voting systems shall provide for training of boards of elections personnel

in the following:

17

------------------------------------------------------------------------------------------------------------------------------

(a) training prior to delivery of voting systems equipment on procedures for

unpacking, assembling and acceptance testing of such equipment;

(b) training for proper use of such equipment including maintenance, storage

and transportation procedures;

(c) the vendor shall provide complete operations manuals (including

operations manuals for any auxiliary features, programming, hardware,

telecommunications systems and central vote tabulating systems) upon delivery of voting

systems equipment to a jurisdiction. Such manuals shall include one copy of procedures

to be followed by inspectors at polling places. The vendor shall permit this copy to be

reproduced and distributed by the county board of elections at its training school for

election inspectors or the vendor shall supply enough copies of the procedures for such

distribution;

Comment 73--Although ERMA allows telecommunications in voting systems, all current and future communications capability should be banned, since it opens the election to tampering by individuals in remote locations, and such tampering cannot be detected by election staff or observers.

(d) the vendor shall assist in the training of all elections personnel (including

election inspectors) during the first two elections, to include a general election, in which

the equipment is used. Such assistance relating to the number of people and the hours

of assistance shall be identified in the executed contract.

(e) training county boards of elections personnel in the procedures to be used

to accomplish ballot face layout and ballot programming.

(2) Service provisions

(a) The contract shall identify the obligations of the vendor to rectify any

problems identified through testing any or all of the voting systems equipment delivered to

the purchaser.

Comment 74--Given the superficiality of state certification testing and the experience of other states with failures of equipment during elections, it is likely that problems with equipment will first be detected during elections. For this reason, the regs should explicitly create a formal mechanism and procedures for voters, poll workers, candidates, and parties to report operational failures of equipment during elections. The regs should require such reports to be dealt with in a timely manner before candidates' rights to request counts of the VVPAT expire, and prior to certification of the election results.

Jurisdictions must be required to acknowledge receipt of such reports, post them in public if the reporting person or party so requests, and investigate and resolve issues related to the reported failures prior to certification of the election. If operational failure of equipment is verified, proper remedies must be listed in the regs -- including the conduct of a new election paid for by the vendor if lesser actions cannot remedy the failures that occurred.

(b) The vendor shall, without additional cost, provide to the purchaser a five-

year guarantee of parts and service, that such voting systems equipment shall be kept in

good working order and that other statutory requirements are met.

(c) The vendor shall provide to the purchaser of said equipment a detailed

listing of proper maintenance, storage and transportation procedures to be carried out by

each purchaser.

(d) The vendor and the purchaser shall agree in writing as to the proper

maintenance procedures to be implemented on each piece of equipment and shall further

agree in writing as to the obligations of each party for servicing and maintenance

procedures.

(e) An agreement as to the time period in which the vendor must correct any

problems or defect in the voting equipment or voting systems.

18

------------------------------------------------------------------------------------------------------------------------------

(f) The vendor shall provide the purchaser with the criteria necessary for the

proper operation of the voting equipment at a polling place.

(3) Polling site survey

(a) The vendor, together with the purchaser, shall survey the present polling

places in a jurisdiction to which its voting equipment has been sold, to determine whether

or not such polling places meet environmental conditions for the proper operation of the

voting equipment. This provision shall apply to those polling places which are in use at the

time of the proposed sale.

(b) If any polling places are not compatible, the vendor shall advise the

jurisdiction purchasing the voting equipment on the methods or procedures that the said

jurisdiction may use to remedy any such problem.

(4) Additional Requirements

(a) delivery deadline shall be not less than three months prior to the first

election in which said units shall be used or, if the contract is for ten or less units, not less

than one month prior to such election;

Comment 75--Three months or one month doesn't leave enough time for training elections staff, ballot programming, Logic and Accuracy testing performed by humans who enter votes and test all parts of the system as described in Comment 26, voter and pollworker training, etc.

(b) acceptance testing requirements;

(c) storage and maintenance responsibilities; and

(d) shipping delivery guidelines and requirements.

Comment 76--Draft standards are not filled in above, but are in the next section.

B. For purposes of the initial purchases of voting machines and systems, pursuant to the

federal Help America Vote Act of 2002, and the state Election Reform and Modernization

Act of 2005, all contracts entered by the State Board of Elections, or local boards of

elections, with vendors, must comply with Office of General Services (OGS) regulations on

Purchasing Procedures and Purchases from Preferred Sources, found in NYCRR Title 9,

Subtitle G, Subchapter A, Part 250, section 250.0 through and including section 250.11.

Section 6209.10 Acceptance Testing

Section Comment--Acceptance testing will be hasty and superficial.

A. County boards of elections, under the supervision of the State Board, shall conduct an

acceptance test on each unit of any voting system purchased by such county. Such

acceptance testing shall begin within seventy-two hours of delivery of the equipment from

the vendor to the purchaser.

Comment 77--With 62 counties, what supervision is possible by the State Board?

B. Such testing shall be conducted under the supervision of the State Board in accordance

with the testing requirements and formats provided by the State Board. This test may

19

------------------------------------------------------------------------------------------------------------------------------

consist in part, of the original certification test deck as utilized by the State Board in the

certification of the system.

Comment 78--The original certification test deck ought to work, since the vendor will have had plenty of time to make sure of that. If no other tests are performed this is inadequate. We have yet to see a requirement for a test election with maximum numbers of voters and votes, and a complete audit including inspection of the audit logs and other printouts from the system, as described in Comment 26.

C. The results of acceptance testing shall be certified to the State Board and entered into

the maintenance log for each piece of equipment.

D. If the acceptance test reveals any improper or faulty absentee ballot counting systems

equipment, the vendor must make corrections to such improper or faulty equipment within

30 days from the date of such acceptance testing.

Comment 79--Here is another cut-and-paste error from central-count optical scanner document.

Comment 80--Timing problems are foreseeable in small counties -- the equipment need not be delivered until one month prior to the election, but if the equipment does not work, the vendor has 30 days to make corrections. Assuming that a percentage of the corrections lead to other problems which also require correction, some counties may have to resort to the use of emergency paper ballots for all voters.

E. The State Board, upon its review of the acceptance testing of such equipment may, at

its discretion, suspend certification of said equipment for future sales in the State of New

York in accordance with the provisions of these regulations.

Comment 81--It would be more efficient to have more rigorous up-front testing than to discover that testing was inadequate after the equipment is purchased and delivered. If acceptance testing shows many problems, such as in Georgia 2002 where approximately one third of 30,000 units did not boot-up upon delivery, then the units should not be used in an election.

Section 6209.11 Routine Maintenance Test of DRE Voting Equipment

Section Comment--Periodic testing is good, but the number of ballots (minimum 200) is not enough to "stress test" the equipment. Many computer errors do not show up until many items of data (ballots) are entered, and malicious code can be programmed to kick in after a large number of ballots have been entered.

It is not clear from these regs whether the entering of the ballots must be done using the same hardware and software that will be used during an election. "Automated testing" which consists of running a program to "test" the machine leaves many parts of the election system untested, with the result that voters discover errors on election day. In contrast, the regs below for Paper-based systems specify that "complete testing" shall be conducted.

A. In addition to vendor-prescribed maintenance tasks and diagnostic tests, a test of DRE

voting equipment shall be conducted on each piece of equipment owned by a county board

of elections.

B. Such testing shall be administered periodically and be completed during the following

periods:

(1) January 15-April 15

(2) April 16-July 15

(3) July 16-September 15

(4) September 16-November 15

C. Such testing shall consist of the casting of a minimum of 200 ballots on each piece of

equipment during each of the prescribed periods outlined.

D. Such tests shall be developed by the State Board, utilizing a ballot format prepared and

programmed by each county board. Each such test shall be approved by the State Board

prior to the first periodic test. The State Board shall reserve the right to revise said testing

format, based upon its audit and review.

E. The test ballot format during the period including July 16 - September 15 shall consist

of the primary ballot as it has been certified by the board of elections, if said equipment is

to be utilized in a primary election.

F. The test ballot format for the period between ballot certification and seven days before

election shall consist of the general election ballot as it has been certified by the board of

elections.

G. The result of each periodic test shall be entered upon the maintenance log for each

20

------------------------------------------------------------------------------------------------------------------------------

such piece of equipment, together with any other information prescribed in said log by the

State Board.

H. The county board of elections shall certify to the State Board, the completion of each

periodic maintenance test. Such certification shall be on a form prescribed by and

furnished by the State Board, and shall be accompanied by copies of each maintenance

log.

I. The State Board may, upon review of the maintenance logs, require further testing of

any such piece of equipment or may, for sufficient cause, remove a piece of equipment

from use in an election until further examination and testing has been completed.

Comment 82--The regs should give examples of what "sufficient cause" might consist of.

J. County boards shall make the equipment available to the State Board for any such

additional testing and shall provide such assistance as may be deemed necessary.

Comment 83--This periodic testing is good, but the number of ballots (minimum 200) is not enough to "stress test" the equipment. Many computer errors do not show up until many items of data (ballots) are entered.

Comment 84--It is not clear from these regs whether the entering of the ballots must be done using the same hardware and software that will be used during an election. "Automated testing" in which only a program is run to "test" the machine leaves many parts of the election system untested, with the result that voters discover errors on election day. In contrast, the regs below for Paper-based systems specify that "complete testing" shall be conducted.

Section 6209.12 Operational and Testing Procedures for Paper-based Voting

Systems

Section Comment--"Complete testing" is not defined, and it is unclear why are DREs do not have to be "completely" tested. This section appears to contain procedures for central-count optical scanners that are used after election day to count absentee ballots, not precinct-based optical scanners that are used on election day.

A. Complete testing of the paper-based voting system shall be conducted before the use

of the system in any election.

Comment 85--"Complete" is not defined, and it is not specified why DREs are not to be "completely" tested.

B. Pre-election Test Deck

Not more than 20 days before the day designated by the county board for the

counting of paper ballots, the board shall test the system to ascertain that it will properly

count the votes cast for all offices and all questions. The test shall be conducted by

processing a test deck for each ballot style. If the system does not accurately count the

test deck, the cause for the error or errors shall be ascertained and corrected and an

errorless count shall be made before the system is approved for use in the count of actual

ballots. The commissioners of the county board shall certify that they have reviewed and

verified the results of said testing.

Comment 86--Paragraph seems to be cut-and-pasted from material for central-count optical scanners.

C. Public Demonstration

In addition to the pre-election test, the county board shall conduct a public

demonstration of the test utilizing all or a portion of the test deck. Appropriate written

notice of the public demonstration shall be sent to the chair of the county committee of

each political party and to each candidate whose name appears on the ballot. One

representative of each political party and one representative of each candidate whose

name appears on the ballot shall be entitled to be present at the test.

Comment 87--Consistently these regs omit voters, pollworkers, and good government groups as stakeholders in the conduct of elections. There should be provision for their inclusion.

Comment 88--Comparable public testing should be required for DREs.

The commissioners of the county board shall certify that they have reviewed and

verified the results of the public demonstration testing.

D. Storage of Test Deck

21

------------------------------------------------------------------------------------------------------------------------------

Following the pre-election testing and public demonstration testing, the test deck

shall be locked in secure storage until immediately preceding the official tabulation of paper

ballots. All copies of test data, including copies of ballot programming, shall be stored with

the test deck, in locked secured storage.

E. Testing Immediately Preceding Official Tabulation of Paper Ballots

Immediately preceding the official tabulation of paper ballots, the following testing

shall be completed:

(1) The paper ballot counting system shall be cleared of all votes and a printed

report shall be produced by the system to confirm that all voting positions are at zero.

(2) The test deck shall be run through the system to demonstrate that the system

can accurately count votes and the results shall be compared to the pre-election test data.

The commissioners of the county board shall certify that they have reviewed and verified

the comparison of the test data before the official tabulation of ballots is conducted.

(3) The system shall again be cleared of all votes and a printed report shall be

produced by the system to confirm that all voting positions are at zero.

F. Testing During Ballot Tabulation

The system shall be so designed and constructed that, at the discretion of the

county board, it shall be possible to halt the ballot tabulation at a point when a portion of

the election districts have been counted, and run the test deck to demonstrate, as in the

pre-count tests listed in section (E) above, the accuracy and dependability of the count

without jeopardizing any official tabulation of results that may be on the equipment at that

time.

G. Testing Following the Machine Tabulation of Ballots

Immediately following the machine tabulation of the ballots from all the election

districts and the production of the county-wide totals of votes, the pre-count tests listed in

section (E) above, shall be run so as to demonstrate the accuracy and dependability of the

count.

H. System Management

(1) The county board of elections shall have management control over all resources

employed during the tabulation process, including the processing of ballots and the testing

of equipment.

(2) If it becomes necessary to transfer control of any equipment back to the vendor

for repairs, operational tabulation activities may not be carried out on the equipment while

22

------------------------------------------------------------------------------------------------------------------------------

it is solely under the vendor's control.

Comment 89--These procedures appear to be cut-and-pasted from a document for a central-count optical scanner.

Comparable requirements should apply to DRE equipment. Moreover, the testing of DRE equipment must consist of votes entered in the same manner as votes are to be entered during an election, including use of all accessible devices and minority language interfaces, inspection of the VVPAT as it is generated, second-chance voting, attempted entry of overvotes and undervotes, extraction of vote tallies after votes have been entered, and inspection of all audit logs produced by the DRE equipment.

I. State Board Support During First Year of Operation

(1) During the first two elections in which such equipment is used, including a

general election, the State Board shall assist and supervise the operation of the paper-

based voting system. Such supervision shall include but not be limited to:

(a) preparation of test deck

(b) supervision of pre-election, public demonstration and pre-tabulation tests

(c) supervision of official tabulation of ballots on the day to be designated by

the county board of elections

(2) During successive years, the State Board, whenever it deems necessary, or at

the request of a county board of elections, shall assist in the operation of the system.

Section 6209.13 Submission of Procedures for Unofficial Tally of Results of Election

County boards of elections which adopt procedures pursuant to section 9-126(3) of

the Election Law shall submit such procedures to the State Board of Elections.

Section 6209.14 Routine Maintenance for Paper-based Voting Equipment

Section Comment--It is unclear why these regs are applied only to paper-based systems and not to DREs.

A. Each county which purchases a paper-based voting system shall keep a detailed log

of maintenance performance and testing procedures.

B. Such logs shall be in a format provided by the State Board and same shall have been

reviewed by the vendor.

C. Such logs shall be provided regularly to the State Board, for their review and inspection.

D. The State Board, upon written request of a vendor or any other interested or

aggrieved party, may, after a hearing, suspend the use of any paper-based voting system

in any county in which proper maintenance procedures or proper servicing by the

manufacturer have not been fully implemented resulting in malfunction of such equipment.

Comment 90--Why is this limited to Paper-based voting equipment, and not applicable to DREs also?

Comment 91--The county boards of elections are supposed to be trained to maintain their own systems, so why does this reg cover only servicing by the manufacturer? (And does “manufacturer” here also mean the “vendor?”)

E. The State Board may reinstate the certification based upon review of these procedures

and a review of the maintenance logs.

Comment 92-- It is unreasonable that these provisions are applied only to paper-based systems and not to DREs also.

Section 6209.15 Demonstration Models

Section Comment--ERMA, Page 2, Lines 46-54

46 {3.} 2. For five years after any voting machine OR SYSTEM of a type

47 approved by the state board of elections {after September first, nine-

48 teen hundred eighty-six} PURSUANT TO THE ELECTION REFORM AND MODERNIZA-

49 TION ACT OF 2005 is first used in any election district, the {city or

50 town which purchased such machine} LOCAL BOARD OF ELECTIONS WHICH OWNS

51 SUCH MACHINE OR SYSTEM shall provide a model OR DIAGRAM of such voting

52 machine OR SYSTEM for each polling place in which any such election

53 district is located. Such models OR DIAGRAMS shall meet the standards

54 set forth in regulations promulgated by the state board of elections.

A. During the first five (5) years after purchase, any county which purchases voting

23

------------------------------------------------------------------------------------------------------------------------------

equipment systems shall provide a model or diagram of such voting system's equipment

for each polling place in its jurisdiction.

B. If a model or diagram is used, such model or diagram must meet the following

specifications:

(1) be approved by the State Board

(2) may not contain the name of any party or independent body which has been

continuously used in New York State.

(3) display a ballot layout which shall consist of at least two party rows and eight

voting positions including at least one multiple-candidate office (vote for two).

C. If a model is used, each model must

(1) be no less than 11 inches by 14 inches

(2) be operated by electricity and/or a battery power source

(3) enable the voter to vote for a candidate

(4) enable the voter to negate or change a vote

(5) enable the voter to cast the ballot.

(6) specify how and where to cast a write-in ballot.

D. If a diagram is used,

(1) shall specify how to mark or cast a ballot

(2) shall specify how and where to mark or cast a write-in ballot

(3) shall be no smaller than 11 inches by 17 inches