Citizens' Oversight Projects (COPs)

771 Jamacha Rd #148

El Cajon, CA 92019

CitizensOversight.org

(619) 693-8960

support@citizensoversight.org

Aug 5, 2021

California Secretary of State Shirley Weber

1500 11th Street

Sacramento, CA 95814

c/o tkayatta@sos.ca.gov

Ted.Muhlhauser@sos.ca.gov 

                                                                REF: M1977

Dear Secretary Weber:

We hereby respectfully submit the following comments for improvements to the proposed RLA regulations.

The audit principles that guide all our suggestions are transparency, for the public; efficiency for our election officials, and accountability for all. Effective audits will deter unrest after our elections by producing public evidence that the results are accurate, thereby providing the trust necessary for our democracy to function.

Our organization has been active in providing oversight of audits throughout the state and in many other states, and we have investigated the various options for performing audits, including Risk-Limiting Audits.

It has been clear to us and many other observers that deployment of RLA audits has been slow in California and other places. We think this is for several clear reasons: a) they are actually more difficult than people originally thought, b) the ballot-comparison audit, which requires the fewest samples, is incompatible with polling place scanners that do not keep the ballots in order, and then require that they are rescanned, c) the workload required can be unknown and unpredictable, and d) they tend to cover only one or two contests, leaving all other contests unaudited.

The so-called "gold standard" implementation in Colorado includes only one statewide and one county contest -- two contests total. This would be a huge step backwards if that was the result here in California.

In the current regulations as proposed, counties can optionally perform RLAs on just one or a few contests while continuing to use the 1% Manual Tally (OPMT) as outlined by Cal Elec Code 15360 for all other contests. This is a very minimal improvement from where we are today, even if the RLAs of those few contests are performed perfectly. We are concerned that the bad habits and shortcuts used in the OPMT will be repeated in RLAs, eliminating any real improvement in audits in California. In fact, it seems like a step backward from where we are today if the OPMT is eliminated, as advocated by as many RLA proponents. The OPMT is a fixed-percentage batch comparison audit, which has procedures to include at least one batch of all contests. RLAs do not.

We would like to propose a different phase-in process. Instead of implementing all the whistles and bells of the RLA procedures for only one or a few contests, we suggest implementing a portion of the requirements for all contests.

We note that there are a number of improved procedures that are included in proposed RLA regulations but are not fully embraced by the OPMT. Our proposal is to incrementally improve each one of these aspects and thereby improve the audits of all contests, so that the OPMT approaches the same functionality as an RLA, but for all contests. We believe it is preferable to implement this phase-in process as part of the RLA regulations. In fact, by also implementing these aspects in the OPMT audits, the overall risk is reduced, which is indeed the goal of "risk limiting" audits, even if the escalation functionality is only applied to very few contests.

The following table compares and contrasts the aspects of the OPMT vs. RLA audits, with the Severity estimated based on whether the aspect will cause the audit to be essentially worthless or not.

We note and agree that it is desired for the auditing process to be "Risk Limiting" and increase the number of samples required when the margin of victory is narrow. However the issues that have severity of "Critical" or "High" can be fixed in the OPMT by improving SOS regulations of those procedures. If these aspects are not performed correctly, then the audit is nothing more than theater.

Therefore, we have proposed an additional regulation that requires that election officials improve their procedures regarding these aspects.

When the margin of victory is tight in very consequential contests, we believe the public will not settle for anything less than a Full Hand Count Audit (FHCA), and we advocate in other documents that FHCAs be defined more clearly, particularly to include batch comparisons to limit human error. Any review of paper ballots by humans is limited by the underlying error rate imposed by the fact that humans are involved and mistakes are inevitable without an additional method to hold it in check.

Although our comments below are specifically regarding the RLA regulations under review, our research and case studies have demonstrated that Ballot Image Audits provide coverage of all contests to a higher precision than other audit types. Our Ballot Image Auditing service "AuditEngine," can be used independently or in conjunction with audit types, and are particularly helpful to hold human error in check. To find out more, visit AuditEngine.org.

Below, we provide specific changes recommended to the text of the RLA regulations.

Respectfully submitted,

Raymond Lutz
Executive Director, Citizens Oversight

COSIGNED BY THE FOLLOWING INDIVIDUALS AND ORGANIZATIONS

John Brakey, Director Audit Elections USA

Mimi Kennedy, Election Integrity Activist

Wendi Lederman, Election Integrity Activist

Tim White, Election Transparency Advocate

Susan Shamblin, Election Integrity Activist

Darlene Little, Election Integrity Activist

Susan Pynchon, Election Integrity Activist

Paul Burke, VoteWell.net

SPECIFIC COMMENTS TO RLA REGULATIONS

Summary of the most important issues:

1. Public notice solely by social media or paper signs is not enough. We changed the rule from one method to all of a shorter list of methods, and we recommend that election offices also allow members of the public to subscribe to updates by email.
   
2. Audit Boards can be election office workers but it is certainly preferred that the board is composed of individuals that are not also responsible for the canvass, with diverse party affiliations. We also suggest a Supervisory Audit Board to oversee the Audit Board teams, particularly for the batch comparison RLA being added.
   
3. Support for Batch Comparison Audits. We agree that it is necessary to include batch comparison audits because any county that utilizes polling-place scanners is unable to match CVR entries to physical ballots, as those scanners are designed to randomize entries in the CVR and the ballots also fall into the bin without maintaining order. This results in a number of changes to wording where only "ballot" appeared but now "batch" is required to be included, and enhancement in how ballots are inspected and tallied.
   
   Physical boxes of batches should be accessed and brought to the auditing room immediately (i.e. on the same day) upon completion of the random selection, without breaking the seals of the batches, without any preview or precounting, and while still under public observation.
   
   We recommend that batches be split up into smaller easily tallied groups so the tally teams can't attempt to match the published totals. The results from each team should be entered on hand-marked tally sheets, preferably designed to avoid counting or adding tally marks (the count should be readable from the sheet without conversion) and the total entered. The Supervisory Audit Board should then enter the totals into a spreadsheet so that the totals can be directly compared with the official result for each batch.
   
4. Requirement that Manifest be correlated with the CVR. We have found in some counties that the CVR uses different batch id numbers than is used on the boxes. Therefore, the Manifest can't be completely independent from the voting system, because we need to be able to correlate the ballot ID and batch ID numbers to the descriptors in the Manifest. The Manifest must list the same Batch ID and ballot ID as used in the CVR, as well as the box number.
   
5. Minimize time when sampled ballots can be altered and minimize public burden by doing these steps in this order, at one meeting where the ballots are stored: (a) give CVRs or batch totals to observers, (b) publicly select sample seed and sample records, (c) immediately bring forth sampled batches and ballots and examine seals, (d) interpret sampled ballots onto paper, (e) provide copies to observers, (f) compare audit board interpretations to previously released batch totals or CVRs.
   
6. CVRs, Manifest, and Ballot Images should be posted prior to the random draw. This is very important. The prior rule of not posting the CVR due to privacy concerns was unfounded because the provision cited is regarding registration data, whereas CVRs and ballot images are anonymous.
   
7. The Random Selection Meeting must be open to the public, including video recording and photography. Some counties are still restricting access to these meetings. We believe the definition should also allow counties to choose batch numbers directly (rather than choosing a seed then using a random number generator), particularly if only a few dozen batches are to be selected. This procedure, using 10-sided dice, should be specified by the SOS also for the OPMT. Election Code section 15360, specifies that a regulation would be adopted, but there is no corresponding regulation. As a result, methods vary widely, with some being technically incorrect, and others difficult to verify by the public.

15360 (c) The elections official shall use either a random number generator or other method specified in regulations that shall be adopted by the Secretary of State to randomly choose the initial precincts, batches of vote by mail ballots, or direct recording electronic voting machines subject to the public manual tally.

8. Support for two-phase audit. We believe the text of the regulation can be simplified without so much repetition for each phase. A two-phase audit is required to allow counties to start the first phase as soon as the first tranche of ballots have been processed, scanned, and the report published. The second phase must use a new random sample for the additional rows on the manifest, since continuing the previous sample makes it predictable. However, we did not attempt to resolve the exact wording.
   
9. We object generally to the use of an RLA software tool, which is largely not necessary, and results in a DRE-style data input regimen which does not have an adequate paper trail. Without an adequate paper trail, the RLA tool is vulnerable to hacker manipulation. We propose that instead of a custom tool, hard copy tally sheets be used with a simple spreadsheet. In batch-comparison audits, the use of a software tool for inputting totals of each batch is unnecessary. However, randomly choosing batches is best done using weights proportional to the size of the batch or according to maximum error possible in each batch. This is a standard procedure for batch comparison RLAs. For this, a simple open source program may be advisable.
   
10. A central aspect of RLAs is the comparison of human interpretation to machine interpretation. Human interpretation should be resolved to hand-marked tally sheets which can be scanned and published for anyone to review. RLAs call for determining the number of overstatements in any contest and then comparing the number of overstatements to the threshold. An overstatement is when the machine interpretation has more votes for the winning candidate than the human interpretation. If the threshold is exceeded, then the RLA does not confirm the election results, and would call for an expanded number of samples, or resort to a full hand count. It is our opinion that only one expansion is ever necessary, and then it is best to immediately move to a full hand count. This is explained in our document regarding the BRAWL method, Balanced Risk Audit with Workload Limitation^[1].
    
11. When discrepancies are found, the rule needs to call for investigation into whether the machine interpretation as provided in the CVRs is incorrect, or the paper ballot may have been altered during storage by someone whose entry was not detected by the chain of custody. However, it is extremely critical that the auditors not resolve the issue for that batch or sample by changing the machine interpretation.
    
12. Election officials creating codes of conduct without limitation creates disparities between counties and can limit effective observation, such as by telling observers not to run the RLA tool on their own computers. The counties should create a joint code. The random selection process should be fully open to video recording and photography, as no ballots are under review. Within reason, video cameras from the public should be allowed in the counting area. Workers must be aware that they may be shown in videos doing their work.
    
13. Observers directly questioning audit boards cause delay. The rule must require another official be assigned to answer questions. The official in turn, would obtain answers from audit boards only when not otherwise available.
    
14. Regulations for the 1% Manual Tally (OPMT), used by all counties to protect our elections from errors and attacks, are urgently needed, under ELEC 15360(c). Batch totals must be released before sample selection. Sample needs to be chosen by seed or 10-sided dice (as in 20120 below), publicly at the same time and place the batches are pulled and tallied. Observers need to be able to see voters’ marks (as in 20124) and get copies of tally sheets (as in 20123).
    
15. All ballots must be covered by the OPMT. This goal means amending ELEC 15360(a)(1)(A) to add a second phase, auditing 1% of batches which were not included in the semifinal official canvass. We invite problems when we leave a third to a half of our votes unaudited.
    
16. Posting county audit reports online needs to be resumed, as done by the Secretary of State through 2010.
    
17. Phase-in Strategy improves procedures for use in RLA audits even if the OPMT is still used for those contests not covered by the RLA.

Comments by Citizens' Oversight on RLA regulations, in detail

NOTE: Our suggested changes are shown in the following by striking out some wording and underlining new wording and these are also highlighted to distinguish from the changes already proposed by the SOS, which are also shown using strike out and underlining but are not highlighted. Comments regarding our changes are highlighted in blue. 

==========================================================

California Secretary of State

Proposed Regulatory Action

Emergency Action: Risk Limiting Audits

Proposed Regulation Text

Changes to Permanent Regulations

Title 2. Administration

Division 7. Secretary of State

Chapter 2. Risk Limiting Audits

The California Secretary of State is proposing to amend the following existing regulations: Sections 20110, 20111, 20112, 20114, 20121, 20122, 20124, and 20125. Changes to the permanent regulations (including changes previously made by emergency regulatory action and current in effect) are shown as strikethrough for text removed and underlined for the text added.

20110. General Provisions. 

(a) The purpose of this chapter is to establish guidelines and procedures for an elections official to conduct a risk-limiting audit in accordance with the requirements of Division 15, Chapter 4, Article 5.5 of the Elections Code.

(b) Pursuant to Elections Code section 15367(a)(1), commencing with the statewide primary general election held on March 3 November 3, 2020, the elections official conducting an election may conduct a risk-limiting audit in place of the one percent manual tally required by Elections Code section 15360 during the official canvass of any election.

(c) A participating county may exclude any contest that has been subject to a risk-limiting audit from the one-percent manual tally required by Section 15360.

Note: Authority cited: Section 12172.5, Government Code; Section 15367, Elections Code. Reference: Sections 15150, 15360 and 15367, Elections Code. 

20111. Definitions. 

As used in this Chapter, the following terms have the following meanings:

(a) “Ballot” shall have the meaning set forth in Elections Code section 15366(a). A validly cast ballot is any ballot that is eligible to be counted in the canvass of an election, as specified in Division 15, Chapters 2 through 4 of the Elections Code.

(b) “Ballot card” means a card upon which is printed, or identified by reference to the ballot, the names of candidates for nomination or election to one or more offices or the ballot titles of one or more measures. A ballot may be comprised of multiple ballot cards.

(c) “Ballot-level comparison audit” shall have the meaning set forth in Elections Code section 15366(b).

(d) “Ballot manifest” means a detailed description of how the ballots are stored and organized, listing the unique physical location of each and every ballot card cast in the election in such a way that individual ballot cards or batches of ballot cards can be found, retrieved, and examined manually.

(e) “Ballot-polling audit” shall have the meaning set forth in Elections Code section 15366(c). California Secretary of State – Proposed Regulatory Action Risk Limiting Audits – Proposed Regulation Text (Changes to Permanent Regulations) Page 1 of 6

(f) “Batch comparison audit” means a type of risk-limiting audit in which elections officials examine voter markings on randomly selected batches of ballots seeking strong evidence that the reported tabulation outcome is correct. 

(g) “Cast vote record” shall have the meaning set forth in Elections Code section 15366(d). The cast vote record shall be generated by the voting system.

(g) "Cross jurisdictional contest" shall have the meaning set forth in Elections Code section 15366(e) 

(h) “Elections official” shall have the meaning set forth in Elections Code section 320.

(i) “Electoral outcome” shall have the meaning set forth in Elections Code section 15366(f)(e).

(j) “Partial risk limiting audit” or “partial RLA” shall have the meaning set forth in Elections Code section 15366(g). 

(j) “Hybrid audit” means a combination of multiple risk-limiting audit types. 

(k) “Public notice” means the release of information to the public through all one or more of the following readily available communication channels: a website update, a social media post, an email list mailing including at least those requesting it, a press release, and a notice posted at an election office open to the public.

(l) “Random seed” means a number consisting of at least 20 digits that is used to generate a random number sequence to select ballot cards or batches for audit.

(m) “Risk-limiting audit” or “RLA” shall have the meaning set forth in Elections Code section 15366(h)(f).

(n) “Risk-limiting audit software tool” or “RLA software tool” means software that can perform any of the audit types described in Section 20112 and which the election official can install on a local computer to avoid internet vulnerabilities. Pursuant to Elections Code section 15367(b)(2)(E), the algorithms and source code of the RLA software tool shall be disclosed to the public. The SOS shall aeffect this disclosure by posting the source code, or a link to the source code hosted on another publicly available website, on its website. Software must be posted 2 months before each election, for the public to verify, and not change unless a corrected version is posted no less than a month before the election. One option must be in a common spreadsheet format with the calculations visible, not hidden. 

(o) “Voting system” shall have the meaning set forth in Elections Code section 362.

Rhode Island recently demonstrated excellent procedures for conducting a ballot comparison RLA audit of contests in that state, and they livestreamed the updates to a spreadsheet as they collected the data from tally teams.

Note: Authority cited: Section 12172.5, Government Code; Section 15367, Elections Code. Reference: Sections 320, 362, 15302, and 15366, Elections Code.

20112. Audit Types. 

The Audit Boardelections official shall use an RLA software tool provided by the Secretary of State to perform one of the following:

(a) A ballot-level comparison audit, with a five percent risk limit.

(b) A ballot polling audit, with a five percent risk limit.

(c) A batch comparison audit, with a five percent risk limit. 

(d) A hybrid audit, combining two or more of the audit types in subdivisions (a) through (c), with a five percent risk limit. 

Note: Authority cited: Section 12172.5, Government Code; Section 15367, Elections Code.

20113. Audit Initiation.

1. If the audit commences after all validly cast ballot(s) within the jurisdiction have been tabulated, the audit shall be conducted in a single phase as described in section 20121.
2. If the audit commences before all validly cast ballot(s) within the jurisdiction have been tabulated, it shall be conducted in two phases as described in section 20122.

Note: Authority cited: Section 12172.5, Government Code; Section 15367, Elections Code.

20114. Contests to be Audited. 

The election official who elects to conduct an RLA in accordance with the requirements of Division 15, Chapter 4, Article 5.5 of the Elections Code shall do so on each contest fully contained within its jurisdiction's borders, and a partial RLA on each cross jurisdictional contest partially contained within its jurisdiction's borders.

Participating counties shall conduct a risk-limiting audit on one or more contests fully contained within the county’s borders. 

Note: Authority cited: Section 12172.5, Government Code; Section 15367, Elections Code. Reference: Section 15150, Elections Code. 

20115. Audit Board Selection.

Our group spent a great deal of time discussing this issue. The key goal of having one or more Audit Boards is a good one, because it makes the audit more independent. The concept of the original RLA concept was that the Audit Boards are essentially like the tally teams that tally votes in a 1% Manual Tally, and then they have delegated the decision making to the software tool to make the comparisons of the human interpretation with the CVR records for those appropriate ballots, and still include a large role for the election official.

Thus, throughout the document, the audit is conducted by the audit board rather than the elections official.

For batch-comparison audits, the situation is slightly different because there are likely very many tallying teams that are working in parallel, tallying their sub-batches to hand-written tally sheets. Those tally sheets are returned to a supervisory Audit Board, where they will be each entered into a spreadsheet, totaled to provide the total votes for each contest and option in the batch, and then compared with the same totals from the CVR for that batch. This will generate a number of "overstatements", i.e. votes in that batch where the CVR improperly gave too many votes to the reported winner. Then, at the end of the tallying process, the total number of overstatements can be determined and compared with the threshold value for that margin of victory.

The supervisory audit board can also be involved in those situations such as accessing batches from storage to guarantee that no further processing or precounting is performed, and that the batch boxes remain sealed.

We believe that the Supervisory Audit Board should have at least three members, including one person from the election office who understands the process, and two others, perhaps a judge and a county supervisor who was not up for election in this election, as that is how Florida does it.

1 The elections official shall appoint an audit board(s) to perform the duties specified in section 20123 that meets the following criteria:

1. Audit boards shall consist of no fewer than three members. At all times, at least one member shall serve as an observer of the audit and may not make determinations of voter choices. At least two members, excluding the observer, shall make determinations of voter choices for each ballot card examined by the audit board and enter them on a tally sheet. Audit board members may rotate roles.
2. Audit board members shallmay be comprised of election office full-time or temporary staff, volunteers or a combination of staff and volunteers, persons who are not involved in the canvass,
3. Members of the Audit Board should have diverse party affiliation.
4. The two members examining each ballot card may not have been in the same political party in the past two years.
5. A supervisory Audit Board should also exist to collect tally sheets from the individual audit boards, enter them into the master spreadsheet, create totals of overstatements, and compare with the threshold number for each contest margin. Discrepancies will also be researched to determine what the issue is with each one, but the number of discrepancies cannot be changed unless there was an error in the audit process itself.
6. Prior to conducting the audit, each member of an audit board shall complete and sign a declaration of intent to faithfully discharge audit board duties. Each audit board member shall do so in the presence of the elections official on a form provided by the elections official. The elections official shall countersign the declaration. The declaration shall be in the following form:

5. No fewer than two members of the audit board will be in possession of ballot cards at any time, including during the ballot card retrieval process.

(b) If the elections official appoints multiple audit boards, only one audit board shall evaluate each ballot card.

Note that for batch comparison audits, it may be appropriate for there to be a number of audit board teams that will be responsible for auditing one sub-batch at a time on a hand-marked tally sheet, and then returning this sheet to the supervisory audit board, which takes the hand marked tally sheets and enters them into a spreadsheet, totals up the sub-batches to create the audit interpretation of each batch, and then compares it with the totals for that batch from the voting system, to provide the total number of overstatements, which can then compared with the threshold for the margin of victory for each contest being audited.

__snip__

(form omitted)

__snip__

Note: Authority cited: Section 12172.5, Government Code; Section 15367, Elections Code.

20116. Public Education on Risk-Limiting Audits.

1. Prior to conducting an RLA, the elections official shall provide public notice that includes descriptions of how the process will be conducted, the difference between 1% manual tally and an RLA, and the methods for ensuring ballot security. The elections official shall also provide information regarding the process for selection of their audit boards and procedures used to ascertain voter intent manually.
2. The Secretary of State shall make the same materials described in (a) above available on its website.

Note: Authority cited: Section 12172.5, Government Code; Section 15367, Elections Code.

20117. Ballot or Batch Manifest and Ballot Handling.

1. The elections official shall maintain an accurate ballot or batch manifest (Manifest), created independent of the voting system. The ballot mManifest shall uniquely identify the storage container in which each validly cast ballot card or batch is stored after tabulation. The Manifest shall be additionally linked to records in the Cast Vote Record (CVR), such that if a ballot id or batch number in the CVR exists, it can be used either directly or indirectly, to locate the ballot or batch in the storage containers.
2. For ballot-level comparison RLA audits, the format for the ballot manifest shall be in the format required by the RLA software tool in the California Post-Election Risk-Limiting Audit Ballot Manifest Format document dated October 15, 2019, which is hereby incorporated by reference, which the Secretary of State shall post on its website. 
3. If a batch-comparison audit is used, then the batch manifest may be batch-to-box conversion sheets that should be scanned and posted online publicly.
4. If the audit commences after all validly cast ballot(s) have been tabulated, the elections official shall create only one ballot mManifest.
5. If the audit commences before all validly cast ballot(s) have been tabulated, the elections official shall create two ballot mManifests, one before each phase of the audit: an initial ballot mManifest that includes all ballots that have been tabulated before the first phase of the audit starts, followed by the final ballot mManifest including all tabulated ballots.
6. The content of the initial ballot mManifest shall not be changed in the final mManifest; accordingly, no ballots shall be added to the containers included in the initial mManifest. Instead, the tabulated ballots that are not included in the initial ballot mManifest shall be stored in new containers, and the final ballot mManifest shall include all the rows in the initial ballot mManifest plus a row for each new container. Example: 1,000,000 ballots were cast in the county, of which 900,000 had been tabulated when the elections official decided to start the first phase of a two-phase audit. The initial ballot manifest includes those 900,000 ballots. Ultimately, 97,000 of the remaining 100,000 ballots are determined to be validly cast. These ballots are tabulated and are stored in new containers. The final ballot mManifest consists of the initial ballot mManifest plus additional rows that describe the new containers in which these 97,000 ballots are stored, for a total count of 997,000 ballots in the ballot mManifest for the second audit phase.

Note: Authority cited: Section 12172.5, Government Code; Section 15367, Elections Code.

20118. Chain of Custody.

1. The elections official shall maintain and document an uninterrupted chain of custody for each ballot card, each batch, and each ballot storage container. Chain of custody logs should, at a minimum, include an identifying number of each ballot storage container and the number of a tamper-evident seal affixed to each ballot storage container in such a way that the container cannot be opened without breaking the seal. Chain of custody logs shall be available for public inspection after the canvass of the election pursuant to Division 15, Chapters 2 through 4, of the Elections Code is complete.
2. The chain of custody logs must be kept separate from the containers, such that accessing the containers does not allow a malicious actor to also modify the chain of custody documents.
3. The elections official shall secure and maintain in sealed ballot containers all tabulated ballots.
4. The elections official shall establish written procedures to ensure the security, confidentiality, and integrity of any ballots, cast vote records, or any other data collected, stored, or otherwise used pursuant to this section. These procedures shall be published on its website at least five days in advance of the audit.

Note: Authority cited: Section 12172.5, Government Code; Section 15367, Elections Code.

20119. Data Publication Prior to Audit.

1. When the elections official submits data to the RLA software tool or spreadsheet before an audit phase begins they shall also publish the same data on their website before continuing with the audit. The elections official shall also publish any changes made to such data.
2. Prior to random selection of ballots or batches in each phase, the Audit Board shall ensure that the following is published:

1. The Cast Vote Records (CVRs) of all ballots and batches included in that phase, detailed to the ballot, and where each record includes the "Batch ID" of the batch of that ballot, and the style of the ballot, preferably in the standard format as specified by the National Institute of Standards and Technology (NIST) Cast Vote Records Common Data Format (CVR)^[2]
2. Ballot images of all ballots included in that phase,
3. The Manifest of the batches and ballots included in that phase

3. Subject to Elections Code section 2194(a), cast vote record data shall not be posted to the elections official’s website, but shall be made available to the public at the location where the audit is being conducted. 
   [Elections code section 2194(a) covers voter registration data and is not relevant to CVR data, which is anonymous.]

Note: Authority cited: Section 12172.5, Government Code; Section 15367, Elections Code.  Reference: Section 2194, Elections Code.

20120. Random Seed.

1. The random seed shall be generated in order by sequential rolls of one or more fair 10-sided dice.
2. The random seed shall be generated in a public meeting as follows:

1. The elections official shall give at least five days public notice of this public meeting.
2. The elections official shall randomly select members of the public who attend the meeting to take turns rolling a die, and designate one or more staff members to take turns rolling a die.
3. In the event that no members of the public attend the meeting, the elections official can designate someone or themselves to roll the die.
4. The random selection meeting shall be open to the public and shall allow video recording or photography.

3. After the random seed is generated, the elections official shall provide public notice of the random seed.
4. The random seed shall be entered into the RLA software tool as provided in sections 20121(d) and 20122(e). if the tool is used.
5. 10 sided dice may be used to directly select the random ballots or batches to be audited. Each color of the die shall be designated for a given decimal position, ones, tens, hundreds, thousands. The Manifest shall be sequentially numbered, from 1 to n, where n is the total number of ballots or batches. Separate rolls of the dice shall select each ballot or batch to be audited. This method is also recommended for the 1% manual tally random selection per Elections Code 15360.

Note: Authority cited: Section 12172.5, Government Code; Section 15367, Elections Code.

20121. Audit Procedures for Single-Phase Audit. 

(a) The Audit Boardelections official conducting the auditan RLA after all ballots cast within the jurisdiction have been tabulated and reported shall compare: enter the following information into the RLA software tool: 

(1) The number of ballot cards on the ballot mManifest for all ballot card(s);

(2) The total number of ballot cards represented in the CVRs; 

(3) The results for all ballot card(s) tabulated; and 

(4) If a For ballot-level comparison audits or hybrid audits with a ballot-level comparison audit component is being conducted, the contest totals of cast vote records for all ballot cards tabulated; and

(5) For batch comparison audits or hybrid audits with a batch comparison audit component is being conducted, vote totals from each contest being audited for each batch, or cast vote records for all ballot cards tabulated including a field that specifies the batch of that ballot, such that the CVR records can be subtotaled for each batch. 

(b) If the comparison RLA software tool identifies any inconsistencies in the information entered under subdivision (a), the elections official shall resolve the inconsistencies before the audit proceeds.
(c) After the data have been submitted under subdivisions (a) and (b), At the time and place announced for the public audit, the elections official shall generate a random seed or directly select samples pursuant to section 20120.

(d) The elections official shall enter the random seed into the RLA software tool. The RLA software tool will randomly generate a list of particular ballot card(s) or batches of ballot card(s) from the ballot manifest to examine manually.

(e) The elections official may at any point decide to conduct a full manual tally of any contest(s) whose outcomes have not yet been confirmed by the RLA. In the event that the elections official conducts a full manual tally, the RLA of that contest shall be suspended. Such a manual tally shall follow the procedure specified in Elections Code section 15290, but in addition, shall subtotal by batch and compare batch by batch with the official results to limit human error and locate inconsistencies. Any contest with a margin of victory less than 0.5% should not attempt an RLA but a Full Hand Count audit should be performed instead.

Note: Authority cited: Section 12172.5, Government Code; Section 15367, Elections Code. Reference: Section 15290, Elections Code.

20122. Audit Procedures for Two-Phase Audit. 

(a) An elections official conducting a two phase audit under subdivision (d) of section 20117 shall do so in accordance with this section.

(b) In the first phase of the audit, the elections official shall compare enter the following information into the RLA software tool: 

(1) The initial ballot mManifest for all ballot card(s) tabulated as described in subdivision (d) of section 20117;

(2) The results for all ballot card(s) tabulated that are included in the initial ballot mManifest;

(3) If a ballot-level comparison audit or a hybrid audit with a ballot-level comparison audit component is being conducted, the cast vote records for all ballot card(s) tabulated that are included in the initial ballot manifest; and 

(4) If a batch comparison audit or a hybrid audit with a batch comparison audit component is being conducted, vote totals from each contest being audited for each batch; and

(5) The maximum number of ballots to be tabulated.

(c) If the RLA software tool identifies any inconsistencies in the information entered under subdivision (b), the elections official shall resolve the inconsistencies before the audit proceeds. (d) After the data have been submitted under subdivisions (b) and (c), the elections official shall generate the first random seed pursuant to section 20120.

(e) The elections official shall enter the first random seed into the RLA software tool. The RLA software tool will randomly generate a list of particular ballot card(s) or batches of ballot card(s) from the ballot manifest to examine manually.

(f) In the second phase of the audit, the elections official shall enter the following information into the RLA software tool:

(1) The final ballot manifest for all ballot card(s) tabulated as described in subdivision (d) of section 20117;

(2) The results for all ballot card(s) tabulated that are included in the final ballot manifest; and

(3) If a ballot-level comparison audit or a hybrid audit with a ballot-level comparison audit component is being conducted, the cast vote records for all ballot card(s) tabulated; and

(4) If a batch comparison audit or a hybrid audit with a batch comparison audit component is being conducted, vote totals from each contest being audited for each batch.

(g) If the RLA software tool identifies any inconsistencies in the information entered under subdivision (f), the elections official shall resolve the inconsistencies before the audit proceeds. (h) After the data have been submitted under subdivisions (f) and (g), the elections official shall generate the second random seed pursuant to section 20120. 

(h) The elections official shall enter the second random seed into the RLA software tool. The RLA software tool will identify whether the audit can stop or whether further auditing is required to meet the risk limit. If further auditing is required, the RLA software tool will randomly select additional particular ballot card(s) from the final ballot manifest to be examined manually.

(i) The elections official may at any point decide to conduct a full manual tally of any contest(s) whose outcomes have not yet been confirmed by the RLA. In the event that the elections official conducts a full manual tally, the RLA of that contest shall be suspended. Such a manual tally shall follow the procedure specified in Elections Code section 15290.

Note: Authority cited: Section 12172.5, Government Code; Section 15367, Elections Code. Reference: Section 15290, Elections Code. 

20123. Ballot Retrieval and Manual Examination.

1. For batch-comparison audits, the sealed storage containers containing each batch shall be retrieved and moved to the auditing location without opening the containers or breaking any seals.
2. For ballot-level audits: The audit board shall locate and retrieve, or observe the location of and retrieval by elections official or election staff of, each randomly selected ballot card from the appropriate storage container.
3. The audit board shall verify that the seals on the appropriate storage containers are those recorded on the applicable chain of custody log.
4. For batch-comparison audits: each batch shall be split approximately into separate tally sets of about 100 ballots each. Each tally set shall be tallied by a separate tally team using either the read-and-tally method using a hand-marked tally sheet that will not require counting or addition to total the vote counts.
5. The audit board shall examine each randomly selected ballot card. If the selected ballot card was duplicated prior to tabulation, the audit board shall retrieve the original ballot card and report how the original (rather than the duplicate) was marked.
6. The audit board shall interpret voter markings on ballot cards selected for audit in accordance with Elections Code section 15154. If the audit board members cannot unanimously agree on the voter’s intent, they shall indicate their disagreement in the final report in section 20125. They shall then notify the elections official of the disagreement, who shall arbitrate the issue. The elections official may:

1. Instruct the audit board members to replace the ballot card with another ballot card, selected at random, that contains the same contests as the previously selected ballot card;
2. Instruct the audit board members to submit to the RLA software tool that no consensus can be reached on the voter choices; or
3. Make a final determination of the voter markings and instruct the audit board to submit the voter markings or choices in all audited contests to the RLA software tool.

7. The audit board shall record the voters’ choices in every contest on every ballot card selected for audit. Those choices shall be entered into the RLA tool, if one is used, and in any case, written on a hand-tally sheet. Copies of the hand tally sheets shall be given to observers present, and shall be scanned and posted to a public web site.
8. A printout of batch totals or CVRs from the election, for the sampled ballots, shall be made, for visual comparison to the hand-tally sheets. In a batch comparison audit, tally sheets for parts of the batch will be summed before comparison to the printout.
9. In ballot comparison and ballot polling audits, when human interpretation differs from machine interpretation, an investigation will follow. If the ballot image agrees with the ballot, the CVR is wrong. If the ballot image agrees with the CVR, investigation needs to determine if they were both created wrong, or the paper ballot has been altered during storage, by someone whose entry was not detected by the chain of custody. Digital signatures from when the files were created will indicate if the files were altered.
10. The audit shall continue until the risk limit is met for every contest being audited pursuant to section 20114. If the RLA software tool specifies that further auditing is required to meet the risk limit, it will randomly select additional ballot card(s) from the ballot manifest to be examined manually. The elections official shall retrieve those additional ballot card(s) as described in (a), examine those additional ballot card(s) as described in (b) and (c), and record the voters’ choices as described in (d).
11. If the elections official chooses to perform a full manual tally of one or more contests, the audits of those contests will be deemed to have met the risk limit and the results of the full manual tally shall be the official results for those contest(s).
12. The RLA shall be completed no later than the canvass deadline, as specified by Section 15372 of the Elections Code.

Note: Authority cited: Section 12172.5, Government Code; Section 15367, Elections Code.  Reference: Sections 15154 and 15360, Elections Code.

20124. Public Observation and Verification of Audit. 

(a) The elections official shall ensure that the audit process is observable and verifiable to the public. The elections official shall:

(1) Provide at least five days public notice prior to the first phase of the RLA. This notice shall include the single time and place whenof the random seed will be generatedion described in section 20120 and the date the ballot cards will be retrieved and manually examined.

(2) Describe in writing the process of manually examining ballot cards and the selection of the ballots to be used in the audit.

(3) Provide observers with an oral and/or written explanation of the RLA process, a written code of conduct for observation, and any documentation they will need for informed and effective observation.

(A) The code of conduct for observation shall be determined by the counties jointly, or if they fail to do so, by the Secretary of State elections official. The code of conduct for observation shall explain the rights and responsibilities of observers.

(B) Documentation needed for informed and effective observation shall include but not be limited to any data the audit relies upon, including: the ballot manifest and the cast vote records for ballot-level comparison audits. 

(i) The ballot manifest;

(ii) For ballot-level comparison audits or hybrid audits with a ballot-level comparison audit audit component, the cast vote records;

(iii) For batch comparison audits or hybrid audits with a batch comparison audit component, vote totals from each contest being audited for each batch.

(4) Disclose the methods used to select samples and to calculate the risk.

(5) Provide the public the opportunity to observe ballot cards being retrieved and examined during the audit.

(6) Provide the public the opportunity to observe the voters’ marks on every audited ballot card during the audit.

(b) Observers can ask questions of the audit board or a designated official present, who shall obtain answers from others present if s/he does not know them. the audit board, as long as they do not interfere with the conduct of the audit procedures. 

Note: Authority cited: Section 12172.5, Government Code; Section 15367, Elections Code.

20125. Certification of Contest Results and Reporting of Audit Results.

(a) The elections official conducting an RLA shall report the results of the audit in the certification of the official canvass of the vote specified by Section 15372 of the Elections Code. The report shall contain the following:

(1) The type of RLA conducted (e.g. ballot comparison, ballot polling, batch comparison, or hybrid);

(2) The date and time the RLA commenced and finished, and the number of rounds of sampling;

(3) A list of contests audited, with final reported results for those contests and the final measured risk for each contest;

(4) A link to the website where the ballot manifest can be found;

(5) The random seed(s) the elections official generated for use with the RLA software tool, and when and how the random seed(s) was generated;

(6) The number of ballot card(s) examined for each contest under audit, noting whether a full manual tally was conducted. If the audit was conducted in stages or continued beyond the first sample, a description of how the sample was drawn and the number of ballot card(s) selected at each stage.

(7) Notes regarding any ballot card(s) for which the audit board could not determine the voter’s intent, as described in subdivision (c) of section 20123;

(8) A link to the website where the list of the particular ballot card(s) examined can be found;

(9) For ballot-level comparison audits or hybrid audits with a ballot-level comparison audit component:

(A) The number of discrepancies between ballot card(s) and the cast vote record, and a description of each discrepancy; and

(B) A link to the website where the cast vote record file only for the ballot card(s) examined in the audit can be found;

(10) For batch comparison audits or hybrid audits with a batch comparison audit component, discrepancies between batches and batch totals;

(11) The person-hours required to prepare for the audit, and number of people involved; (12) The person-hours required to conduct the audit, and number of people involved; (13) The number of members of the public who observed the audit;

(14) Notes on anything unusual or problematic, or that would be useful to improve the process, or that might be of value to the Secretary of State or other elections officials; and

(15) An attestation from the elections official that the RLA was conducted in accordance with Title 2, Division 7, Chapter 2 of the California Code of Regulations.

(b) The certification of the official canvass of the vote shall be conducted in accordance with Section 15372 of the Elections Code.

(c) The Secretary of State shall publish any report under this section received from an elections official on its website no later than five days after the date it publishes the certification of the official canvass of the vote in accordance with Section 15372 of the Elections Code. This subdivision shall not prohibit the elections official from publishing the same report on its website.

Note: Authority cited: Section 12172.5, Government Code; Section 15367, Elections Code.

20126. Redaction of Personally Identifiable Voter Choices.

No later than the third business day following the expiration of the deadline to request a recount under Elections Code sections 15620 or 15621, or the completion of any recount, whichever is later, an elections official who conducted an RLA shall review its cast vote records file that was published and redact from the public record, but not its own files, the voter choices corresponding to any ballot susceptible to being personally identified with an individual voter.

It is essential for public oversight of the audit that the CVR files are published prior to the random draw. If the CVR file is not published, then it would be possible for insiders to alter the CVR file to thwart the audit. As a simple example, let's say that the insider knows batches 1-100 are modified out of 1000 batches, and the random draw of batches includes batches 50 and 60. The insider could simply alter the labels on the boxes, swapping 50 and 60 with 150 and 160, so that the hacked batches are no longer in the audited set. Or, if the CVR is available to the hacker, then that person can swap the totals for 50 and 60 with those of 150 and 160, and then the hack would not be detectable.

Note: Authority cited: Section 12172.5 Government Code; Section 15367, Elections Code.  Reference: Sections 15620, 15621 Elections Code.

20127. Phase-in of improved auditing procedures.

Even if no or very few contests are selected for RLA procedures, elections officials should  phase-in improved procedures as defined by RLA regulations, for all contests, as follows:

1. Include all ballots in the scope of the random selection process, using a two-phase process if needed to include the later-processed ballots in the audit.
2. Publish the full CVRs detailed to the auditing unit (ballot or batch) for each phase, prior the random selection meeting.
3. Create an audited unit (ballot or batch) Manifest
4. The Random Selection meeting should occur only after the detailed CVRs are published for that phase.
5. Video recording is allowed in the Random Selection Meeting.
6. Use the Random Selection Process as defined in RLA regulations, including using 10-sided dice and a list. For batch-comparison audits, should be ideally weighted by batch size or maximum error.
7. For batch comparison audits, and 1% manual tally, batches should be pulled and moved to the auditing room without breaking any seals and with minimal time after the random selection is performed and during the same public meeting.
8. Discrepancies in each batch audited must be completely reported and not "solved" by rescanning ballots or deciding it was not an important discrepancy, unless it was truly an error in the audit itself and not the election.
9. Use an independent Audit Board, with a Supervisory Component.
10. Escalation of Samples: Use an escalation procedure that draws more samples when the margin is narrow.
11. Publish scanned tally sheets, including the totals of the counts from each tally team.
12. Include all contests. This is already supported by the 1% Manual Tally without change, However, for those counties that are processing only mixed-precinct batches, the number of ballots included for any given contest should be at least 1% of the total ballots in that contest, or higher if risk limiting procedures are used.

Election officials will self-evaluate and provide a report to the SOS regarding how many of these issues are improved so that the SOS can publish an improved auditing procedure compliance report.