An official website of the United States government
We introduce the UConn Bubbles with Swatches dataset. This dataset contains images of voting bubbles, scanned from Connecticut ballots, either captured as grayscale (8 bpp) or color (RGB, 24 bpp) artifacts, and extracted through segmentation using ballot geometry. These images are organized into 4 groups of datasets. The stored file contains all data together in color and we manually convert to greyscale. Each image of a bubble is 40x50 pixels. The labels are produced from an optical lens scanner. The first dataset, Gray-B (Bubbles), uses 42,679 images (40x50, 8 bpp) with blank (35,429 images) and filled (7,250 images) bubbles filled in by humans, but no marginal marks. There are two classes, mark and nonmark. The second dataset, RGB-B, is a 24 bpp color (RGB) version of Bubbles-Gray. The third dataset, Gray-C (Combined), augments Gray-B with a collection of marginal marks called “swatches”, which are synthetic images that vary the position of signal to create samples close to the boundary of an optical lens scanner. The 423,703 randomly generated swatches place equal amounts of random noise throughout each image such that the amount of light is the same. This yields 466,382 labeled images. The fourth dataset, RGB-C, is a 24bpp color (RGB) version of Gray-C. The empty bubbles are bubbles that were printed by a commercial vendor. They have undergone registration and segmentation using predetermined coordinates. Marks are on paper printed by the same vendor. These datasets can be used for classification training. The .h5 has many levels of datasets as shown below. The main dataset used for training is positional. This is only separated into blank (non-mark) and vote (mark). Whether the example is a bubble or a swatch is indicated by batch number. See https://github.com/VoterCenter/Busting-the-Ballot/blob/main/Utilities/LoadVoterData.py for code that creates torch arrays for RGB-B and RGB-C. See the linked Github repo (https://github.com/VoterCenter/Busting-the-Ballot/blob/main/Utilities/VoterLab_Classifier_Functions.py) for grayscale conversion functions and other utilities. Dataset structure: COLOR - POSITIONAL - INFORMATION / / / B/V/Q B/V/Q COLOR/POSITIONAL / / / IMAGE IMAGE B/V/Q / BACKGROUND RGB VALUES Images divided into 'batches' not all of which have dataInformation contains labels for all images. Q is the swatch data, while B and V are non-mark and mark respectively.
more »
« less
Ballot Tabulation Using Deep Learning
Currently deployed election systems that scan and process hand-marked ballots are not sophisticated enough to handle marks insufficiently filled in (e.g., partially filled-in), improper marks (e.g., using check marks or crosses instead of filling in bubbles), or marks outside of bubbles, other than setting a threshold to detect whether the pixels inside bubbles are dark and dense enough to be counted as a vote. The current works along this line are still largely limited by their degree of automation and require substantial manpower for annotation and adjudication. In this study, we propose a highly automated deep learning (DL) mark segmentation model-based ballot tabulation assistant able to accurately identify legitimate ballot marks. For comparison purposes, a highly customized traditional computer vision (T-CV) mark segmentation-based method has also been developed to compare with the DL-based tabulator, with a detailed discussion included. Our experiments conducted on two real election datasets achieved the highest accuracy of 99.984% on ballot tabulation. In order to further enhance our DL model’s capability of detecting the marks that are underrepresented in training datasets, e.g., insufficiently or improperly filled marks, we propose a Siamese network architecture that enables our DL model to exploit the contrasting features between a handmarked ballot image and its corresponding blank template image to detect marks. Without the need for extra data collection, by incorporating this novel network architecture, our DL modelbased tabulation method not only achieved a higher accuracy score but also substantially reduced the overall false negative rate.
more »
« less
- Award ID(s):
- 2154589
- PAR ID:
- 10439532
- Date Published:
- Journal Name:
- 2023 IEEE International Conference on Information Reuse
- Format(s):
- Medium: X
- Sponsoring Org:
- National Science Foundation
More Like this
-
-
null (Ed.)As paper ballots and post-election audits gain increased adoption in the United States, election technology vendors are offering products that allow jurisdictions to review ballot images—digital scans produced by optical-scan voting machines—in their post-election audit procedures. Jurisdictions including the state of Maryland rely on such image audits as an alternative to inspecting the physical paper ballots. We show that image audits can be reliably defeated by an attacker who can run malicious code on the voting machines or election management system. Using computer vision techniques, we develop an algorithm that automatically and seamlessly manipulates ballot images, moving voters’ marks so that they appear to be votes for the attacker’s preferred candidate. Our implementation is compatible with many widely used ballot styles, and we show that it is effective using a large corpus of ballot images from a real election. We also show that the attack can be delivered in the form of a malicious Windows scanner driver, which we test with a scanner that has been certified for use in vote tabulation by the U.S. Election Assistance Commission. These results demonstrate that post-election audits must inspect physical ballots, not merely ballot images, if they are to strongly defend against computer-based attacks on widely used voting systems.more » « less
-
Boardroom Voting: Verifiable Voting with Ballot Privacy Using Low-Tech Cryptography in a Single RoomA boardroom election is an election that takes place in a single room — the boardroom — in which all voters can see and hear each other. We present an initial exploration of boardroom elections with ballot privacy and voter verifiability that use only “low-tech cryptography” without using computers to mark or collect ballots. Specifically, we define the problem, introduce several building blocks, and propose a new protocol that combines these blocks in novel ways. Our new building blocks include “foldable ballots” that can be rotated to hide the alignment of ballot choices with voting marks, and “visual secrets” that are easy to remember and use but hard to describe. Although closely seated participants in a boardroom election have limited privacy, the protocol ensures that no one can determine how others voted. Moreover, each voter can verify that their ballot was correctly cast, collected, and counted, without being able to prove how they voted, providing assurance against undue influence. Low-tech cryptography is useful in situations where constituents do not trust computer technology, and it avoids the complex auditing requirements of end-to-end cryptographic voting systems such as Prêt-à-Voter. This paper’s building blocks and protocol are meant to be a proof of concept that might be tested for usability and improved.more » « less
-
Risk-limiting audits (RLAs) are rigorous statistical procedures meant to detect invalid election results. RLAs examine paper ballots cast during the election to statistically assess the possibility of a disagreement between the winner determined by the ballots and the winner reported by tabulation. The design of an RLA must balance risk against efficiency: "risk" refers to a bound on the chance that the audit fails to detect such a disagreement when one occurs; "efficiency" refers to the total effort to conduct the audit. The most efficient approaches—when measured in terms of the number of ballots that must be inspected—proceed by "ballot comparison." However, ballot comparison requires an (untrusted) declaration of the contents of each cast ballot, rather than a simple tabulation of vote totals. This "cast-vote record table" (CVR) is then spot-checked against ballots for consistency. In many practical settings, the cost of generating a suitable CVR dominates the cost of conducting the audit which has prevented widespread adoption of these sample-efficient techniques. We introduce a new RLA procedure: an "adaptive ballot comparison" audit. In this audit, a global CVR is never produced; instead, a three-stage procedure is iterated: 1) a batch is selected, 2) a CVR is produced for that batch, and 3) a ballot within the batch is sampled, inspected by auditors, and compared with the CVR. We prove that such an audit can achieve risk commensurate with standard comparison audits while generating a fraction of the CVR. We present three main contributions: (1) a formal adversarial model for RLAs; (2) definition and analysis of an adaptive audit procedure with rigorous risk limits and an associated correctness analysis accounting for the incidental errors arising in typical audits; and (3) an analysis of efficiency.more » « less
-
Binary semantic segmentation in computer vision is a fundamental problem. As a model-based segmentation method, the graph-cut approach was one of the most successful binary segmentation methods thanks to its global optimality guarantee of the solutions and its practical polynomial-time complexity. Recently, many deep learning (DL) based methods have been developed for this task and yielded remarkable performance, resulting in a paradigm shift in this field. To combine the strengths of both approaches, we propose in this study to integrate the graph-cut approach into a deep learning network for end-to-end learning. Unfortunately, backward propagation through the graph-cut module in the DL network is challenging due to the combinatorial nature of the graph-cut algorithm. To tackle this challenge, we propose a novel residual graph-cut loss and a quasi-residual connection, enabling the backward propagation of the gradients of the residual graph-cut loss for effective feature learning guided by the graph-cut segmentation model. In the inference phase, globally optimal segmentation is achieved with respect to the graph-cut energy defined on the optimized image features learned from DL networks. Experiments on the public AZH chronic wound data set and the pancreas cancer data set from the medical segmentation decathlon (MSD) demonstrated promising segmentation accuracy and improved robustness against adversarial attacks.more » « less
- Free Publicly Accessible Full Text
-
Accepted Manuscript1.0
- Conference Paper:
- The DOI is not currently available.
