This document is a hand-made copy of the NYC BOE Rewrite of Draft Voting Systems Standards which was distributed as part of the NYC BOE testimony before the State Board of Elections on 12/20/05 in New York City.
Draft Voting Systems Standards
Draft Voting Systems Standards with comments by Teresa Hommel
Election Reform and Modernization Act (ERMA) with comments by Teresa Hommel
Table of Contents
Section 6209.1 Definitions . . 3
Section 6209.2 Poll Site Voting System Requirements .8
Section 6209.3 Paper-based Voting Systems .11
Section 6209.4 Application Process . . ...14
Section 6209.5 Submission of Voting Systems Equipment .. 15
Section 6209.6 Examination Criteria . . . .16
Section 6209.7 Modifications and Re-examination . . 30
Section 6209.8 Rescission of Certification . . ..31
Section 6209.9 Contracts .. ...32
Section 6209.10 Acceptance Testing . . .. .37
Section 6209.11 Routine Maintenance Test of Voting Systems 38
Section 6209.12 Testing and Operational Procedures for Voting Systems .40
Section 6209.13 Submission of Procedures for Unofficial Tally of Results of Election 43
Section 6209.14 Demonstration Models .45
Section 6209.15 Voting System Required Functions & Features .. 46
Section 6209.16 Security ..48
Section 6209.17 New York City Volumes ..49
Page 2 of 50
Section 6209.1 Definitions.
The terms used in this part shall have the significance herein defined unless another meaning is clearly apparent in language or content.
1. Acceptance test means a test conducted by the county board of elections and the State Board of Elections, to demonstrate that the voting system software and hardware as delivered and installed in the user's environment, meets all of its functional requirements.
2 Auxiliary components means any device, materials or equipment which is used to give
assistance or aid to the actual voting device but is not a permanent or enclosed part of the
3. Ballot layout means the positioning of all political party names and emblems, and names
and emblems of all independent bodies, office titles, ballot proposals, and candidate names, in accordance with the requirements of the Election Law as to order and rotation.
4. DRE means a Direct Recording Electronic voting system which records votes by means
of ballot display provided with mechanical or electro-optical components which are
operated by the voter. Styles of DRE include ballot overlay style and electronic display style.
5. Optical scan means a voting system which uses optical-scan technology and enables voters to cast paper ballots. Styles of optical scan include precinct-based and central count.
6. Election Management Software (EMS) means all information systems used to administer and conduct county Board of Elections operations.
Page 3 of 50
24. Voting System means any voting equipment and any ancillary equipment and all associated software and firmware (if any) supporting such system supplied by the vendor.
25. Voting System Supporting Software is the vendor-supplied software to configure and control Election Day Tabulation Equipment and accumulate election results.
7. Encrypted copy means a scrambling of the programming code which renders it undecipherable such that only the manufacturer of the program (possessor of the encryption key) may unscramble the code.
8. Escrow account means an account and/or a secure facility held by a third party (who shall be approved by the State Board) for the purpose of taking custody of all materials required to be put in escrow by statute.
9. Activity Log written and/or electronic record of any and all activities (including deployment designation) conducted on a voting system both before and after an election.
9a. Maintenance Log. A written and/or electronic record which contains all information relating to performance of scheduled and non-scheduled maintenance on a voting system (as recommended by the vendor or manufacturer of such equipment) and all service visits performed by vendor or manufacturer.
10. Modification means any changes, substitutions, patches, or updates, to either software, firmware or hardware to the voting systems after certification of the voting system has been granted by the NYS Board of Elections. Certain modifications may require re-certification of the voting system. This will be determined by the NYS Board of Election.
11. Operational manual means (1) a manual of all procedures used to prepare the voting system and provide proper maintenance procedures including the unpacking and storage procedures to be utilized by county boards of elections
Page 4 of 50
personnel and (2) a manual of election day setup and election day operating procedures to be utilized by the inspectors.
12. Pre-qualification test means a predetermined set of tests of the total voting system throughout the election process including votes and vote totals prepared by the State Board. Such votes shall be entered upon the voting system and the results of the casting of said votes shall be compared to the predetermined results of the test.
13. Zero Total Report Printout means the printed copy of zero totals, candidate names and offices and other information produced by the voting system prior to the official opening of the polls.
13a. Closing Total Report Printout means printed copy of the votes cast for each candidate and question, the names of candidates and the offices for each candidate and other information provided after the official closing of the polls.
14. Hardware (for the purposes of voting system certification) means any equipment used in the voting device.
15. Software (for the purposes of voting system certification) means any and all programmed logic for the operation of the voting system.
16. Firmware (for the purposes of voting system certification) means software stored in read-only memory devices embedded in the system and not capable of being altered during system operation.
17. Resident Vote Tabulation means the manufacturer's internal firmware program which shall permanently reside on the voting system's central processing unit, registering, accumulating, and storing votes and ballot images.
 Appears that this definition is not used in the document at all and it is not clear what is attempting to define.
Page 5 of 50
17a. Resident memory means the internal memory of the voting system that stores election results and ballot images.
18. Source code means the human-readable language statements used to program the voting system.
19. Specific Environmental Conditions means the effect of (a) natural environmental conditions such as: temperature, humidity, dust and (b) induced environmental conditions such as handling, storage or transportation or radio frequency interference which may affect the operation of the equipment.
20. State Board means the New York State Board of Elections.
20a. County Board means a countys Board of Elections including the Board of Elections in the City of New York.
21. Testing Laboratory means an NIST (National Institute of Standards & Technology)-certified private or public laboratory used to perform tests on the voting systems and related equipment.
22. Vendor means any manufacturer, company or individual who seeks to supply voting systems (and/or services for such systems) in New York State.
23. Voting Position means the specific area on the face of the displayed ballot where a selection is made for a candidate or proposal.
25. VVPAT means a voter verifiable paper audit trail.
26. Tactile Discernible Controls means a voting feature which allows persons with limited
reach and/or hand dexterity, the ability to cast their vote.
Page 6 of 50
27. Audio Voting feature means a device that allows blind or visually-impaired persons the ability to cast their vote.
28. Sip and Puff Voting Attachment means a device operated by pneumatic switch which
allows persons with certain disabilities the ability to cast their vote.
29. Election Assistance Commission (EAC) is the commission established by Help America Vote
Act of 2002, which serves as a national clearinghouse of information and reviews of procedures with respect to the administration of federal elections, or its successor body.
30. Paper-based Ballot Counting Equipment means any electronic ballot counting system or equipment which tabulates and reports votes cast on paper ballots.
31. Certification Test Data means a pre-audited group of ballots, marked with a
predetermined number of votes cast for each candidate, write-in position and each voting
option which appears on the ballot for the purposes of certifying the voting system.
32. Local Area Network means a group of locally connected computers that can share information.
33. Election Day Tabulation Equipment means the components of The Voting System whose primary purpose is to accumulate vote totals and store cast ballot records during an election.
34. Re-certification means a certification by the State Board of Elections of a modified and previously certified voting system.
35. Re-examination means a review of a certified voting system by the State Board of Elections to determine if a modification requires re-certification.
36. Election Time means the periods beginning ten (10) weeks prior to an Election Day and ending with the election certification.
Page 7 of 50
Section 6209.2 Poll Site Voting System Requirements
A. In order for a poll site voting system to be considered by the State Board for certification, it must comply with the mandates of New York State Election Law, and the Election Assistance Commission's Voting System Guidelines, and meet the following requirements:
(1) Provide a full ballot display on a single surface (full-face).
(2) Provide a mechanism that retains a voter-verifiable permanent paper record, pursuant to statute, which the voter can review and if necessary change his or her ballot prior to the casting
of the ballot.
(3) Provide a device or means by which the votes cast and the total cumulative values on the Election Day Tabulation Equipment can be printed, recorded, visually reviewed and reported after the polls are closed.
(4) Provide a battery powered source in the event that the electric supply used to make the voting system equipment function if disrupted. Such batteries must be rechargeable and have minimum five-year life. They must run the voting system and display for 17 hours and store cast ballots for up to 7 days without loss or corruption.
(5) The ballot must be easily visible in typical indoor light. If illumination is required to achieve visibility of the ballot, illumination source must be provided by the vendor.
(5) The system shall contain software and hardware required to perform (1) a diagnostic
test of system status, and (2) a means of simulating the random selection of votes in quantities sufficient to demonstrate that the system is fully operational and that all voting positions are operable.
(6) The system shall be designed to protect against Specific Environmental Condidtions.
Page 8 of 50
(7) The system must provide for tamper-evident sealing of any port, door and any component of the Election Day Tabulation Equipment. If the system requires an external storage device to transport Election Day supplies then this external storage device must provide tamper-evident sealing.
(8) If the systems within a poll site are linked to a local area network, the network must be secured from non-authorized attachments. The network must not be continuously connected to a wide area network (i.e. public Internet), but may connect over a secure connection temporarily for reporting or diagnostic purposes. Any such connection must be recorded in the activity log. If diagnostics are performed or updates run, that information must be entered into the maintenance log.
(9) The Election Day Tabulation Equipment must be so constructed as to permit a poll worker to activate the correct ballot.
B. In addition to the requirements of subdivision (A) of this section, fully-accessible components of the Election Day Tabulating Equipment certified by the State Board shall meet the following requirements for usability by voters who are disabled:
(1) The fully-accessible components of the Election Day Tabulating Equipment shall be equipped with a voting device with tactile discernable controls, pursuant to statute.
(2) The fully-accessible components of the Election Day Tabulating Equipment shall be equipped with an audio voting feature, pursuant to statute.
(3) The fully-accessible components of the Election Day Tabulating Equipment must be capable of being equipped with voting device of a sip and puff technology, pursuant to statute.
(4) The equipment shall be so constructed as to allow an individual poll worker to place
Page 9 of 50
the Election Day Tabulating Equipment in a wheelchair accessible position with a minimum of effort and provide proper safeguards for the safety of the voter during operation in this position.
C. Standards for noise level
(1) Voting system to be certified by the State Board shall be constructed in a
manner so that noise levels of the equipment during operation will not interfere with the
duties of the election inspectors or the voting public.
 This needs a specific decibel level designation.
(2) The noise level of write-in components of the equipment shall be so minimal that
it will be virtually impossible under normal conditions for someone at the table used by the
inspectors of elections to determine that a write-in vote is being cast or has been cast.
D. Standards for Voter Privacy
(1) Election Day Tabulation Equipment curtains shall be constructed so that no one else within the poll
site will be able to see how a voter is casting a vote.
(2) Privacy barriers shall be so designed as to allow any voter to access the Election Day Tabulation Equipment without obstruction.
Page 10 of 50
Section 6209.3 Paper-based Voting Systems
A. In addition to voting system requirements provided for elsewhere in these rules and regulations, paper-based systems must:
(1) mechanically or electronically prevent a voter from voting for candidates or ballot
proposals for whom or which he or she is not entitled to vote.
 This cannot be accomplished in any automated way.
(2) be able to alert a voter that he or she has:
(b) Has attempted to cast a ballot from which no votes were recognized by the Vote Tabulation Equipment
(c) Voted for candidates of another party in a primary election
B. The system may not count any votes for an office or ballot proposal which has been
over-voted or otherwise improperly voted.
C. An over-vote in one or more office or ballot proposals shall not prevent the counting of
all other offices or ballot proposals contained on the ballot.
D. In the case of candidates who appear on one or more party lines, the system must be
capable of correctly counting the vote according to provisions of Election Law S 9-112.
E. In vote counting, the system shall ignore any mark on a ballot unless that mark is in a:
 New York Election Law currently stipulates that extraneous marks invalidate a ballot. This provision is in conflict with current law.
(1) voting position for a candidate whose name is on the ballot;
(2) voting position designated for write-in voting for a write-in candidate; or
(3) voting position for a ballot proposal.
Page 11 of 50
F. The system shall provide a method for write-in voting and shall report the number of votes cast in each contest in write-in voting positions.
G. The system shall provide a means by which the software may be positively verified to insure that it corresponds to the format of the ballot face.
G1. Each Election Day Tabulation Equipment shall have a protective counter that will increment for each and every ballot cast, including test decks and other diagnostic or training ballots, for the life of the machine. The number shall not be able to be adjusted through any environmental change. This number shall be readily available for inspection throughout the day without compromising the voting process.
H. The Election Day Tabulating Equipment shall have a public counter that shall be capable of accumulating and reporting a count of the number of ballots for each ballot type, tallied for each election district. It shall be capable of separating and tabulating those election district totals to produce a report of the total of ballots tallied by groups of election districts such as legislative districts, wards, etc.
I. The voting system shall be capable of accumulating and reporting by election district the total
votes cast for each candidate (including write in candidates) and the total vote for or against each ballot proposal. The system shall also be capable of tabulating and reporting the vote cast for each candidate and the vote cast for or against each ballot question by groups of election districts such as
legislative districts, wards, etc.
J. Any change in hardware, including standard off the shelf components, shall be required to be submitted for reexamination pursuant to section 6209.7 of these rules.
Page 12 of 50
K. Ballot specifications:
(1) All ballots shall meet the specifications as to form and content required under
section 7-122 of the Election Law.
(2) Ballots shall be printed in black ink on white paper or on paper stock of different
colors to identify different types of ballots (i.e., emergency, affidavit, etc) or in the case of
a primary, to identify ballots for each political party according to the color assigned to such
party pursuant to law.
(3) Coding which is both machine-readable and visually readable shall be used to securely
identify different ballot styles.
(4) Ballots used in the system shall be able to be counted by hand as well as be
counted by machine. The system shall provide an audit trail of all ballots cast, making
possible the reconstruction of the election, starting with the individual votes of all eligible
voters, in the case of a recount.
(5) The types of ballots used and their form, type size and arrangement must be
approved by the State Board of Elections.
L. Where a paper-based system is used for the central counting of absentee, affidavit, military,
emergency and special ballots, the requirements of 6209.2 do not apply.
Page 13 of 50
Section 6209.4 Application Process
A. The Election Operations Unit shall forward an application form, upon request, to any
vendor, together with a copy of applicable rules and regulations and a pre-qualification test
format for both a general and primary election ballot program.
A1. If a vendor supplies several systems that potentially meet state specifications, the vendor must submit all such systems for certification.
B. Said vendor shall return completed ballot layouts based upon the pre-qualification test
format to the Election Operations Unit. Upon approval of the layouts, the vendor shall
program such equipment and complete the pre-qualification tests for both ballot programs
provided, and enter the simulated votes upon said equipment for each election program.
C. The completed application shall be returned, with a printout of tabulated votes from the
primary and general election pre-qualification tests as cast on the voting system equipment
which the applicant requests to have certified. The pre-qualification test programs shall be
retained by the applicant for use in the certification process.
D. The application and printouts shall be reviewed to determine if the voting system shall
be considered for certification and the applicant shall be notified of such determination.
E. No application shall be deemed to be filed until all documentation required by these
rules has been submitted to the State Board or its designee.
F. A certified or bank check in the amount of $5,000 shall accompany such application,
and be applied towards the actual cost of examination.
G. Fees for the examination of a voting system shall be assessed against the vendor by
the State Board based upon the cost to the State Board for examination of such voting
system by an outside contractor, laboratory or other authorized examiner, but the fees
assessed shall not exceed the amount permitted by statute.
Page 14 of 50
Section 6209.5 Submission of Voting Systems Equipment.
A. Voting systems considered for certification by the State Board shall be delivered to the
State Board or its designee. Such equipment shall include auxiliary components and
equipment used to program ballot layout, and any other additional equipment and supplies used in the
operation of said voting system.
B. If the voting systems equipment is certified by the State Board, the specific equipment
and components examined by the State Board shall become the property of the State
Board for as long as the equipment is in use in the State or for such shorter period as the
State Board shall so determine. Voting systems not certified shall be disposed of pursuant
to the vendor's direction.
C. The applicant shall provide service and normal maintenance of said equipment after
certification and shall supply to the State Board, at no cost, any modification to the
equipment for upgrading of any feature during the period that said equipment is offered for
sale and use in the State.
Page 15 of 50
Section 6209.6 Examination Criteria
A. The State Board or its designee shall certify that every voting system for use in the State in additional to federal requirements, shall meet the unique functional and security requirements of the State of New York and all local election jurisdictions. The State Board or its designee shall, prior to certification, test all functional, security, and election management system integration not tested under Federal certification.
C. All laboratory testing shall be conducted or verified by independent testing authorities
accredited by the EAC and in accordance with current federal standards. Testing shall be performed in conformity with written procedures adopted by the State Board and such procedures shall be available for public inspection.
1. Software and Hardware Qualification Tests
Qualification of voting system software and hardware shall consist of a series of tests,
code analyses, and inspection tests performed at the federal level, to verify that the
software and hardware meet design requirements and that characteristics are correctly
described in the documentation items. In order to qualify for C. Functional Tests, the voting system application shall first pass A. Functional Configuration Audit and B. Physical Configuration Audit qualification requirements.
A. Functional Configuration Audit
Page 16 of 50
A functional configuration audit shall be performed to verify that the software complies
with the Software Specification. Vendor test data may be used in partial fulfillment of this
requirement; however, the State Board or its designee may perform or supervise the
performance of additional tests, or order additional laboratory testing, to verify nominal
system performance in all operating modes and to validate, on a sampling basis, the
vendor's test data reports. The Functional Configuration Audit shall be performed in a
facility selected by the State Board.
(1) Vendor Support
The vendor shall provide a list of all documentation and data to be audited and vendor
technical personnel shall be available to assist in the performance of the Functional
(2) Technical Data
The vendor shall provide the following technical data:
(a) copies of all procedures used for module or unit testing, integration testing
and system testing;
(b) copies of all test cases generated for each module and integration test
and sample ballot formats or other test cases used for system;
(c) records of all tests performed by the procedures listed above, including
error correction and retest.
(3) Audit Procedure
The State Board or its designee shall review the vendor's test procedures and test
This review shall include an assessment of the adequacy of test cases and input data
to exercise all system functions and to detect program logic and data processing errors if
such be present.
The review shall also include an examination of all test data which is to be used as
a basis for certification. All changes to the system software shall also be subject to re-examination.
B. Physical Configuration Audit
Page 17 of 50
(1) The Physical Configuration Audit is an examination of the software configuration
against its technical documentation to establish a configuration baseline for approval. The
Physical Configuration Audit shall include an audit of all drawings, specifications, technical
data and test data associated with the system hardware and this audit shall establish the
system hardware baseline associated with the software baseline. All subsequent changes
to the software baseline configuration shall be subject to re-examination. All changes to
the system hardware which may result in a change in the operation of the software shall
also be subject to re-examination.
(2) Vendor Support
The vendor shall provide a list of all documentation and data to be audited and vendor
technical personnel shall be available to assist in the performance of the Physical
(3) Technical Data
The vendor shall provide the following technical data:
(a) identification of all items which are to be a part of the software release;
(b) identification of all hardware which interfaces with the software;
(c) configuration baseline data for all hardware;
(d) copies of all software documentation which is intended for distribution to
users, including program listings, specifications, operator manual, user manual and
software maintenance manual;
(e) user acceptance baseline test procedure and baseline acceptance criteria;
(f) an identification of any changes between the Physical Configuration Audit
and the configuration submitted for the Functional Configuration Audit and a
declaration that these differences do not degrade the functional characteristics.
(4) Audit Procedure
Page 18 of 50
Required data items include draft and formal documentation of the vendor's software
development program which are relevant to the design and conduct of Qualification Tests.
The vendor shall identify all documents, or portions of documents, which contain
proprietary information not approved for public release. The State Board or its designee
shall agree to use the information contained therein solely for the purpose of analyzing and
testing the software and shall refrain from disclosing proprietary information to any other
person or agency without the prior written consent of the vendor. At the conclusion of the
examination, the State Board or its designee shall return to the vendor all such
documentation and shall not retain any copies thereof. The State Board or its designee
shall review the vendor's source code and documentation to verify that the software
conforms to the documentation, and that the documentation is sufficient to enable the user
to install, validate, operate and maintain the voting system. The review shall also include
an inspection of all records of the baseline version against the vendor's release control
system to establish that the configuration, being qualified, conforms to the engineering and
C. Functional Tests
(1) For all equipment, functional tests should consist of validation of equipment
functional performance by means of procedures under the current "Laboratory Environmental Test
Procedures for Hardware and Software".
(2) Functional tests of voting system software which runs on general purpose data
processing equipment shall include all tests similar to those in procedures which are
necessary to validate the proper functioning of the software and its ability to control the
hardware environment. The tests shall also validate the ability of the software to detect
and act correctly upon any error conditions which may result from hardware malfunctions.
Page 19 of 50
Detection capability may be contained in the software, the hardware or the operating
system. It shall be validated by any convenient means up to and including the introduction
of a simulated failure (power off, disconnect a cable, etc.) in any equipment associated with
2. Software, Hardware, Operating and Support Documentation
(A) Software Qualification
All system software and firmware vendor data items shall be submitted as
a precondition of certification of acceptability for elections use.
(B) Vendor Documentation
Complete product documentation shall be provided to the State Board for voting
systems, their components and all auxiliary devices. This documentation shall be sufficient
to serve the needs of the voter, the operator, systems administrator, and the maintenance technician. It shall be prepared and published in accordance with standard industrial practice for electronic and
mechanical equipment such documentation shall include:
(1) Software Specification
The Software Specification shall contain and describe the vendor's architecture, design standards
and conventions, environment and interface specifications, functional specifications,
programming architecture specifications, and test and verification specifications. Pre-
factory material should include document identification, an abstract of the specification,
configuration control status and a table of contents. The body of the specification shall
contain the following material:
(a) System Overview
The vendor shall identify the system hardware and the environment in which the
software will operate and the general design and operational considerations and
constraints which have influenced the design of the software.
Page 20 of 50
(b) Program Description
The vendor shall provide descriptions of the software system concept, the array of
hardware in which it operates, the intended operating environment, the specific software
design objectives and development methodology, the logical structure and algorithms
used to accomplish the objectives, and data structures.
(c) Standards and Conventions
The vendor shall provide information which can be used as a partial basis for code
analysis and test design. It should include a description and discussion of the standards
and conventions used in the preparation of this specification and in the development of the
The vendor shall identify all published and private standards and conventions used
to document software development and testing. Vendor internal procedures shall be
provided as attachments to this Software Specification.
(e) Test and Verification Standards
The vendor shall identify any standards or other documents which are applicable
to determination of program correctness and acceptance criteria.
(f) Quality Assurance Standards
The vendor shall describe all standards or other documents which are applicable
to the examination and testing of the software, including standards for flowcharts, program
documentation, test planning and test data acquisition and reporting.
(g) Operating Environment
The vendor shall provide a description of the system and subsystem interfaces at
which inputs, outputs and data transformations occur. It shall contain or make reference
to all operating environment factors which influence the software design.
(h) Hardware Constraints
Page 21 of 50
The vendor shall identify and describe the hardware characteristics which influence
the design of the software, such as:
(1) the logic and arithmetic capability of the processor,
(2) memory read/write characteristics,
(3) external memory device characteristics
(4) peripheral device interface hardware data I/O device protocols, and
(5) operator controls, indicators and displays.
(i) Software environment
The vendor shall identify the compiler or assembler to be used for the generation
of executable code and a description of the operating system or system monitor. This
section shall also contain an overview of the compile-time interaction of the voting system
software with library calls and linking.
(j) Interface Characteristics
The vendor shall describe the interfaces between executable code and system
input-output and control hardware.
(k) Software Functional Specification
The vendor shall provide a description of the overall functions which the software
performs in the context of its mode or modes of operation. The vendor shall also describe
the capabilities and methods for detecting and handling exceptional conditions, system
failure, data input/output errors, error logging and audit record generation and security
monitoring and control.
(l) Configurations and Operating Modes
The vendor shall describe the various software configurations and operating modes
of the system; such as preparation for opening of the poll site, vote recording and/or
vote processing, closing of the poll site and report generation. For each software
function or operating mode, a definition of the inputs (characteristics, tolerances or
Page 22 of 50
to the function or mode, how the inputs are processed and what
outputs are produced (characteristics, tolerances or acceptable ranges) shall be provided.
(m) External files
In the event that external files are used for data input or output, the definition of
information context and record formats shall be provided. The vendor shall also describe
the procedures for file maintenance, access privileges and security.
Security requirements provisions, and methodology of the software and hardware shall be identified for each system function and operating mode.
(o) Programming Specifications
The vendor shall provide an overview of the software design, structure and
implementation algorithms. Whereas the Functional Specification of the preceding section
provides a description of what functions the software performs and the various modes in
which it operates, this section should be prepared so as to facilitate understanding of the
internal functioning of the individual software modules. Implementation of functions shall
be described in terms of software architecture, algorithms and data structures and all
procedures or procedure interfaces which are vulnerable to degradation in data quality or
security penetration shall be identified.
(p) Test and Verification Specifications
The vendor shall describe the procedures used during software development to
verify logical correctness, data quality and security of application. This description shall include existing standard test procedures, special purpose test procedures, test criteria and experimental
design and validation criteria. In the event that this documentation is not available, the
Qualification Test agency shall design test cases and procedures equivalent to those
ordinarily used as a basis for in-house verification (see below).
(q) Qualification Test Specification
Page 23 of 50
The vendor shall provide a specification for verification and validation of overall
software performance, including baseline acceptance criteria for control and data input/output,
processing accuracy, data quality assessment and maintenance, exceptional handling and
security. The specification shall identify specific procedures by means of which the general
suitability of the software for elections use can be assessed and demonstrated. The
vendor's specification and procedure shall be used to establish the detailed requirements
of the tests described in the current "Laboratory Environmental Test Procedures for Hardware and Software" of this Standard.
(r) Acceptance Test Specification
The vendor shall provide a baseline specification for installations, acceptance and readiness verification. This specification shall identify specific procedures by means of which the
capability of the software to accommodate actual ballot formats and format logic, and pre-
election logic, accuracy and security test requirements of using jurisdictions may be
assessed and demonstrated, and post-election processing. The vendor's specification shall be used to establish the
detailed requirements of the tests described in the current "Laboratory Environmental Test Procedures
for Hardware and Software" of this standard performed to evaluate the adequacy of the
vendor's procedures and it shall be suitable for inclusion in the regulations and procedures
of user counties when preparing for the conduct of actual elections. The acceptance test must demonstrate all State of New York-specific functionality (See Section 6209.15 Voting System Required Functions & Features).
The vendor shall provide descriptive material and data supplementing the various
sections of the body of the Software Specification. The content and arrangement of
appendices shall be at the discretion of the vendor.
Page 24 of 50
Topics recommended for amplification
and treatment in appendix form include:
(1) Glossary: Provide a listing and brief definition of all software module
names and variable names with reference to their locations in the software structure.
Include abbreviations, acronyms and terms which are either not commonly used in data
processing and software development or which are used in an uncommon semantic
(2) References: Provide a list of references to all related vendor documents,
data, standards and technical sources used in software development and testing.
(3) Program Analysis: Provide the results of software configuration analysis,
algorithm analysis and selection, timing studies and hardware interface studies reflected
in the final software design and coding.
(4) Security Analysis: Provide a detailed description of the penetration
analysis performed to preclude intrusion by unauthorized persons and fraudulent
manipulation of elections data. Identify security policies and measures and selection
criteria for audit log data categories.
(2) Operator Information
This documentation shall include a physical description of the equipment sufficient
to identify all features, control and displays. It shall include a complete procedure for
energizing the equipment, for testing and verifying operational status and for identifying all
abnormal equipment states. It shall include a complete operating procedure for casting
ballots to be tabulated, for controlling the tabulation process, for monitoring the status of
the equipment, for recovering from error conditions and for preparing output reports.
(3) Maintenance Information
(a) This documentation shall contain a complete physical and functional
description of the equipment and a theory of operation which fully describes the electrical
and mechanical function of the equipment,
Page 25 of 50
how ballots are processed, how data are handled in the processor and memory sections, how
data output is initiated and [[[[[controlled, how power is converted or conditioned and how test
and diagnostic information is acquired and used.
(b) A complete parts and materials list shall be provided which contains
sufficient descriptive information to identify all parts by type, size, value or range and
(c) Technical illustrations and schematic representations of electronic circuits
shall be provided with indications of all test and adjustment points and the nominal value
and tolerance or waveform to be measured. Fault detection, isolation and correction
procedures or logic diagrams shall be prepared for all operational abnormalities identified
by design analysis and operating experiences.
(4) Logistics, Facilities and Training
The vendor shall identify all operating and support requirements of the system or
component. These requirements include material, facilities and personnel, including
furnishings, fixtures, and utilities which will be required to support system operation,
maintenance and storage.
(5) Maintenance Training and Supply
(a) The vendor shall identify all corrective and preventive maintenance tasks
and the level at which they shall be performed. Levels of maintenance shall include
operator tasks, maintenance personnel tasks and factory repair.
(b) Operator tasks shall be limited to the activation of controls to identify
irrecoverable error conditions and to the replenishment of consumables such as printer
ribbons, paper and the like.
(c) Maintenance personnel tasks shall include all field maintenance actions
which require access to internal portions of the equipment. They shall include the conduct
of tests to localize the source of a malfunction; the adjustment, repair or replacement of
Page 26 of 50
malfunctioning circuits or components and the conduct of tests to verify restoration to
(d) Factory repair tasks shall be minimized. They shall only include complex
and infrequent maintenance functions which require access to proprietary or to specialized
facilities and equipment which cannot be obtained by the using agency. They shall not
number more than two percent of all maintenance tasks and their frequency shall not
exceed five percent of the total frequency for all corrective maintenance tasks.
(e) The vendor shall identify by function all personnel required to operate and
support the system. For each functional category, the number of personnel and their skills
and skill levels shall be specified.
(f) The vendor shall specify requirements for the training of each category
of operating and support personnel. The vendor shall prepare all materials required in the
training activity and shall provide or otherwise arrange for the provision of qualified
(g) The vendor shall recommend a standard complement of supplies, spares
and repair parts which will be required to support system operation. This list shall include
the identification of these materials and their individual quantities and sources from which
they may be obtained.
Page 28 of 50 note that page 27 of the NYCBOE re-write was a cut-and-paste error
The vendor shall supply, at vendor's expense, any special tools required to repair or maintain the equipment.
[[[[[[ Audit procedure here from p.18-19 ]]]]]]
(6) Test Procedure 
 Both A. Functional Configuration Audit and in B. Physical Configuration Audit sections are organized by Vendor Submissions and State Procedure sub-sections. In C. Functional Tests contains vendor submission subsection but no State Procedure sub-section.
Page 29 of 50 =========================================================================
Section 6209.7 Modifications and Re-examination
A. Any prospective modification to a previously certified voting system shall be submitted
to the State Board.
B. No modification of previously certified voting systems shall be used in any
election until such modification has been approved by the State Board.
C. Prospective modification shall be reviewed by the State Board or by an examiner or
laboratory of the Board's choice in accordance with the fee schedule established by Section
7-201 of the Election Law.
D. Upon completion of a review of such prospective modification, the State Board may
cause a re-examination of the entire voting system, or within its discretion, grant
continuation of certification pursuant to the provisions of Section 7-201 of the Election Law.
Page 30 of 50
Section 6209.8 Rescission of Certification
A. If at any time subsequent to the State Board's certification of a voting system, the State
Board determines that the voting system fails to fulfill the criteria prescribed by statute and
these rules, the Board shall notify any users and vendors of that particular voting system
that the State Board's approval or certification of that system for use or future sale of that system
in New York State is withdrawn.
B. Such notice shall be in writing and shall specify the reasons why the approval or
certification of the voting system is being rescinded. Such notice shall also specify the date on
which the rescission is to become effective.
C. Any vendor or user of such voting system may request in writing that the State Board
reconsider its decision to rescind approval or certification of the voting system.
D. Upon receipt of such request to reconsider, the State Board shall hold a hearing for the
purpose of reconsidering the decision to rescind the approval or certification. Any
interested party shall be given the opportunity to submit testimony or documentation in
support of or in opposition to the Board's decision to rescind approval or certification.
E. The State Board may affirm or reverse its decision.
Page 31 of 50
Section 6209.9 Contracts
A. In addition to complying with all statutory requirements, all contracts for the purchase
of voting systems shall include, but not be limited to, the following requirements:
(a) Training Curriculum Areas
Vendors of voting systems shall provide training services and materials for each curriculum area for each component of the voting system. The scope of the training shall include the processes of the entire Election Life-cycle as delineated below:
θ Voting System Delivery
· Acceptance Testing
· Routine Maintenance
· Non-Routine Maintenance
· Setup for Election Cycle
· Pre-Election Configuration
· Election Day Tabulation Equipment
· Supporting Hardware
· Supporting Software
· Post-Election Processing
Page 32 of 50
(b) Target Training Groups
Vendor must provide the following types of instructor-lead training for all phases of the Election Life-cycle for the following categories of personnel:
θ Board Staff
θ Adjunct Trainers
(c) Training Materials
The vendor must provide the following training materials:
The vendor shall permit these materials to be reproduced and distributed by the county board of elections at its training school for election inspectors or the vendor shall supply enough copies of the procedures for such distribution.
(d) Training Personnel
Vendors shall provide sufficient number of qualified trainers to provide sufficient coverage for the county boards training programs.
The detail for requirements for training, including the number of people and the hours of training shall be identified in the executed contract.
Page 33 of 50
(2) Service and Support Provisions
(a) Onsite Vendor Assistance at Election Time The vendor shall assist all elections personnel as necessary for both the entire voting system and its interoperation with other county Boards of Elections systems during the first four years after each piece of equipment is first place in service. The detail of requirements of such assistance including the number of people and the hours of assistance shall be identified in the executed contract. Election Time is the period of ten (10) weeks prior to an Election Day until the election is certified.
(b) Obligations of Vendor to Fix Test Problems The contract shall identify the obligations of the vendor to rectify any problems identified through testing any or all of the voting systems equipment delivered to the County Board of Elections within a reasonable timeframe.
Page 34 of 50
(c) Five Year Parts & Service Guarantee - The vendor shall, without additional cost, provide to the County Board of Elections a five-year guarantee of parts and service, that such voting systems equipment shall be kept in good working order and that other statutory requirements are met.
(d) Maintenance, Storage & Transportation Documentation - The vendor shall provide to the County Board of Elections of said voting systems equipment a detailed listing of proper maintenance, storage and transportation procedures to be carried out by each County Board of Elections. The vendor and the County Board of Elections shall agree in writing as to the proper maintenance procedures to be implemented on each piece of equipment and shall further agree in writing as to the obligations of each party for servicing and maintenance procedures.
(e) Time Limit of Defect Remedy An agreement as to the time period in which the vendor must correct any problems or defect in the voting systems.
(3) Poll site survey
(a) Conduct Poll Site Survey - The vendor, together with the County Board, shall survey every present poll site in every jurisdiction to which its voting system has been sold, to determine whether
or not such poll sites meet environmental conditions for the proper operation of the
voting system. This provision shall apply to those poll sites which are in use at the
time that the order is placed.
(b) Recommend Remediation of Poll Sites - If any poll sites are not compatible with the voting system, the vendor shall advise the jurisdiction purchasing the voting system on the
Page 35 of 50
(4) Delivery Timeframe
(a) Delivery deadline shall be not less than six (6) months prior to the first
election in which said units shall be used or, if the contract is for ten (10) or less units, not less
than one (1) month prior to such election;
(b) acceptance testing requirements;
(c) storage and maintenance responsibilities; and
(d) shipping delivery guidelines and requirements.
Comment 76--Draft standards are not filled in above, but are in the next section.
B. For purposes of the initial purchases of voting machines and systems, pursuant to the
federal Help America Vote Act of 2002, and the state Election Reform and Modernization
Act of 2005, all contracts entered by the State Board of Elections,
with vendors, must comply with Office of General Services (OGS) regulations on
Purchasing Procedures and Purchases from Preferred Sources, found in NYCRR Title 9,
Subtitle G, Subchapter A, Part 250, section 250.0 through and including section 250.11.
Page 36 of 50
Section 6209.10 Acceptance Testing
A. County boards of elections, under the supervision of the State Board, shall conduct an
acceptance test on each unit of any voting system purchased by such county. Such
acceptance testing shall begin within five business days of delivery of the voting system from
the vendor to the County Board of Elections.
B. Such testing shall be conducted under the supervision of the State Board in accordance
with the testing requirements and formats provided by the State Board. This test may
consist in part, of the original certification test data as utilized by the State Board in the
certification of the voting system.
C. The results of acceptance testing shall be certified by the county board to the State Board and entered into the maintenance log for each component of the voting system.
D. If the acceptance test reveals any impropriety or fault in any component of the voting systems, the vendor must make corrections to such improper or faulty equipment within 30 days from the date of notification.
E. The State Board, upon its review of the acceptance testing of any component of the voting system may, at its discretion, suspend certification of said voting system for use in or future sales to any county in the State of New York in accordance with the provisions of these regulations.
Page 37 of 50
Section 6209.11 Routine Maintenance Test of Voting Systems
A. In addition to vendor-prescribed maintenance tasks and diagnostic tests, a test of voting systems shall be conducted on each piece of equipment owned by a county board of elections.
B. Such testing shall be administered periodically and be completed during the following
(1) January 15-April 15
(2) April 16-July 15
(3) July 16-September 15
(4) September 16-November 15
C. Such testing shall consist of the casting of a minimum of 200 ballots on each piece of
equipment during each of the prescribed periods outlined.
D. Such tests shall be developed by the State Board, utilizing a ballot format prepared and
programmed by each county board. Each such test shall be approved by the State Board
prior to the first periodic test. The State Board shall reserve the right to revise said testing
format, based upon its audit and review.
E. The test ballot format during the third period (July 16 - September 15) shall consist
of the official primary ballot and alternates at the time of testing.
F. The test ballot format during the fourth period (September 16 November 15) shall consist of the official general ballot and alternatives at the time of testing.
Page 38 of 50
G. The result of each periodic test shall be entered upon the maintenance and activity logs for each
such piece of equipment, together with any other information prescribed in said logs by the
H. The county board of elections shall certify to the State Board, the completion of each
periodic maintenance test. Such certification shall be on a form prescribed by and
furnished by the State Board, and shall be accompanied by copies of each maintenance and activity
I. The State Board may, upon review of the maintenance and activity logs, require further testing of
any such piece of equipment or may, for sufficient cause, remove a piece of equipment
from use in an election until further examination and testing has been completed.
J. County boards shall make the equipment available to the State Board for any such
additional testing and shall provide such assistance as may be deemed necessary.
K. The State Board, upon the written request of a vendor or any other interested or aggrieved party, may, after a hearing, suspend the use of any voting system in any county in which proper maintenance procedures or proper servicing by the manufacturer have not been fully implemented resulting in malfunction of such equipment.
L. The State Board may allow a voting system to be returned to service based upon review of these procedures and a review of the maintenance and activity logs.
Page 39 of 50
Section 6209.12 Testing and Operational Procedures for Voting Systems
A. Complete testing of the voting system shall be conducted before the use
of the system in any election.
B. Pre-election Test
Before an Election Day the board shall test the system to ascertain that it will properly count the votes cast for all offices and all questions. The test shall be conducted by
processing predetermined test data that represents all possible voting positions for each ballot style, as stipulated in Section 6209.11
C. Public Demonstration of Pre-election Test
In addition to the pre-election test, the county board shall conduct a public
demonstration of the pre-election test methodology. Appropriate written
notice of the public demonstration shall be sent to the chair of the county committee of
each political party and to each candidate whose name appears on the ballot. One
representative of each political party and one representative of each candidate whose
name appears on the ballot shall be entitled to be present at the demonstration. The commissioners of the county board or their designee(s) shall certify, in a manner to be specified, that they have reviewed and verified the results of the public demonstration.
D. Storage of Test Data
Page 40 of 50
Following the pre-election testing and public demonstration, the test data
shall be locked in secure storage until immediately preceding the official tabulation of
ballots. All copies of test results, and ballot programming, shall be stored with
the test data, in locked secured storage.
E. Testing Immediately Preceding Official Tabulation of Ballots
Immediately preceding the official tabulation of ballots, the following testing
shall be completed on a statistically valid set of election day tabulation equipment to be used in the election.
The test data shall be run through the system to demonstrate that the system
accurately counts votes and that the results can be compared to the pre-election results, and they match. The commissioners of the county board shall certify that they have reviewed and verified
the comparison of the test data before certifying the election results.
F. Testing During Ballot Tabulation
The system shall be so designed and constructed that, at the discretion of the
county board, it shall be possible to halt the ballot tabulation at a point when a portion of
the election districts have been counted, and run the test data to demonstrate, as in the
pre-count tests listed in sub-Section (E) above, the accuracy and dependability of the count
Page 41 of 50
without jeopardizing any official tabulation of results that may be on the equipment at that
 NYC BOE believes there are serious consequences to having this capability. At minimum, it would need to be very severely controlled. It may be inadvisable in general to have such capability.
G. Testing Following the Machine Tabulation of Ballots
Immediately following the machine tabulation of the ballots from all the election
districts and the production of the county-wide totals of votes, the pre-count tests listed in
section (E) above, shall be run so as to demonstrate the accuracy and dependability of the
H. System Management
(1) The county board of elections shall have management control over all resources
employed during the tabulation process, including the processing of ballots and the testing
(2) If it becomes necessary to transfer control of any equipment back to the vendor
for repairs, operational tabulation activities may not be carried out on the equipment while
it is solely under the vendor's control.
I. State Board Support During First Year of Operation
(1) During the first two elections in which such equipment is used, including a
general election, the State Board shall assist and supervise the operation of the voting system. Such supervision shall include but not be limited to:
(a) preparation of test data
(b) supervision of pre-election, public demonstration and pre-tabulation tests
(c) supervision of official tabulation of ballots on the day to be designated by
the county board of elections
(2) During successive years, the State Board, whenever it deems necessary, or at
the request of a county board of elections, shall assist in the operation of the system.
Page 42 of 50
Section 6209.13 Submission of Procedures for Unofficial Tally of Results of Election
County boards of elections which adopt procedures pursuant to Section 9-126(3) of
the Election Law shall submit such procedures to the State Board of Elections.
 For the City of New York in which there are a large number of Police Precincts, there are questions to be resolved regarding computing equipment to enable the Police to securely transmit election results on Election Night. The Vendor needs to demonstrate that it has provided capabilities that will enable this for portable memory devices.
Page 43 of 50
Section 6209.14 Routine Maintenance for Paper-based Voting Equipment
NYC BOE deleted this section.
Page 44 of 50
Section 6209.14 Demonstration Models
A. During the first five (5) years after purchase, any county which purchases voting
systems shall provide a model or diagram of such voting system's equipment
for each poll site in its jurisdiction.
B. Such model or diagram must meet the following specifications:
(1) be approved by the State Board
(2) may not contain the name of any party or independent body which has been
continuously used in New York State.
(3) display a ballot layout which shall consist of at least two party rows and eight
voting positions including at least one multiple-candidate office (vote for two). In multiple languages as per VRA.
C. If a model is used, each model must
(1) be no less than 11 inches by 14 inches
(2) be operated by electricity and/or a battery power source
(3) enable the voter to vote for a candidate
(4) enable the voter to negate or change a vote
(5) enable the voter to cast the ballot.
(6) specify how and where to cast a write-in ballot.
D. If a diagram is used,
(1) shall specify how to mark or cast a ballot
(2) shall specify how and where to mark or cast a write-in ballot
(3) shall be no smaller than 11 inches by 17 inches
Page 45 of 50
Section 6209.15 Voting System Required Functions & Features
A. In addition to complying with all statutory requirements, all contracts for the purchase of voting systems shall include, but not be limited to, the following functions and features requirements:
Paper-based Election Day Tabulation vendors must provide complete specifications for ballot layout so that ballot production may be competitively bid.
VVPAT must be designed for ease of use by the voter and by board of elections staff in conducting either 3% required audit or a manual re-canvass of the results of a contest.
VVPAT must be designed to accommodate the various sizes and lengths of NYC ballots while remaining in a readable font.
Performance Testing Stress test system for NYC volumes
Election Tabulating Equipment
Vote Counters: A Public Counter, a Protected Counter and a machine life-to-date counter.
Storage for Transport of Election Day Supplies
Capability to Operate Tabulating Equipment for at Least 17 hours without external power
Capability to sustain power fluctuations without loss of data or operation
Page 46 of 50
Identity and Access Management
Logon Security for all components
Unique Identifier for External Voting Devices
External Device Validation Authentication
Logging of all activities of all components
Selection of Alternate Ballots must be restricted to authorized personnel
Any Election Day Testing, repairs of any Election Day Tabulating Equipment must be restricted to authorized personnel.
Reset Voting Tabulation Equipment Contents Protection
An electronic version of the audit trail of all votes cast shall be available
Security on paper ballot so that counterfeit ballot cannot be introduced
Protection of Services
Security of transmission devices encryption
Access to machines panels and doors
Access to firmware and software
Device used to transmit election night results must be secure
Election materials must be transported securely
Page 47 of 50
Section 6209.16 Security
A. The State Board or its designee must independently test and certify that the voting system meets the security requirements of the State of New York, as outlined in the Security Framework herein.
B. The vendor shall provide security protections for all categories of the security framework.
C. The vendor shall provide documentation and demonstration of the security features and vulnerabilities assessment inherent in the voting system.
D. The vendor shall present its plan for assessing the security of the county Board of Elections end-to-end operations, to be conducted upon the selection of the vendors voting system by a particular county Board of Elections.
E. The Security Framework is:
Business Continuity & Disaster Recovery those security features and functions that prevent and/or permit recovery from, catastrophic loss of data or damage of equipment resulting from hostile environmental events.
Identity and Access Management those security features and functions that ensure efficient, secure, auditable, roll-based access control. Rolls may include staff access control by Borough as well as jobs-based access groups.
Vulnerability Assessment those security features and functions that detect and protect the system from intrusion or contamination by external sources.
Protection of Services those security features and functions that protect the interfaces and related services from contamination, corruption or disclosure.
Physical Security those security features and functions that protect the system from direct physical contamination, corruption or disclosure. This includes the chain of custody of the voting system (hardware, software, & firmware) during manufacture, transport, delivery and storage.
Page 48 of 50
Section 6209.17 New York City Volumes
The five (5) Boroughs of the City of New York contain approximately 38% of the registered voters in the State. Under the Board of Elections in New York City, there is uniformity of voting systems, procedures, and organization. Relevant volumes for the City of New York are included here to insure that prospective applicants for voting system certification are aware of the performance, quantity and data sizes required to meet the needs of the States largest county Elections Board.
(1) Size of Ballot Contain a sufficient number of rows or columns for parties (including party positions) and independent bodies, and accommodate at least 15 ballot proposals. Both the candidate voting positions and the areas for ballot proposals shall be of sufficient size to contain the information required by statute including multiple languages.
(2) Number of Parties Permit the primaries of as many parties as may be recognized in the State of New York to be held on such machine or system at a single election, and accommodate such number of multiple ballots at a single election as may be required by the state board of elections.
(3) Number of Registered Voters 4.3 million
(4) Number of Board Staff 325
(5) NYC Poll Sites Over 1300 active pollsites
(6) Number of Voting Machine Facilities 5
(7) Number of Election Districts 6300
(8) Number of Pollworkers 35,000
(9) Number of Potential Pollworkers Scheduled for Training 67,000
(10) Percent of Novice Pollworkers Each Election
(11) Number of Training Classes 1,400
(12) Length of Training Season 3-4 weeks
(13) Length of Training Class 3 hours
(14) Number of Training Locations 61
Page 49 of 50
(15) Number of Trainees Per Class 10-150
(16) Number of Candidates Files 13,000
(17) Number of Offices (Citywide) 40
(18) Number of Contests (Citywide) - 600