An official website of the United States government
Recently, bug-bounty programs have gained popularity and become a significant part of the security culture of many organizations. Bug-bounty programs enable organizations to enhance their security posture by harnessing the diverse expertise of crowds of external security experts (i.e., bug hunters). Nonetheless, quantifying the benefits of bug-bounty programs remains elusive, which presents a significant challenge for managing them. Previous studies focused on measuring their benefits in terms of the number of vulnerabilities reported or based on the properties of the reported vulnerabilities, such as severity or exploitability. However, beyond these inherent properties, the value of a report also depends on the probability that the vulnerability would be discovered by a threat actor before an internal expert could discover and patch it. In this paper, we present a data-driven study of the Chromium and Firefox vulnerability-reward programs. First, we estimate the difficulty of discovering a vulnerability using the probability of rediscovery as a novel metric. Our findings show that vulnerability discovery and patching provide clear benefits by making it difficult for threat actors to find vulnerabilities; however, we also identify opportunities for improvement, such as incentivizing bug hunters to focus more on development releases. Second, we compare the types of vulnerabilities that are discovered internally vs. externally and those that are exploited by threat actors. We observe significant differences between vulnerabilities found by external bug hunters, internal security teams, and external threat actors, which indicates that bug-bounty programs provide an important benefit by complementing the expertise of internal teams, but also that external hunters should be incentivized more to focus on the types of vulnerabilities that are likely to be exploited by threat actors.
more »
« less
Bounty Everything: Hackers and the Making of the Global Bug Marketplace
In Bounty Everything: Hackers and the Making of the Global Bug Marketplace, researchers Ryan Ellis and Yuan Stevens provide a window into the working lives of hackers who participate in “bug bounty” programs—programs that hire hackers to discover and report bugs or other vulnerabilities in their systems. This report illuminates the risks and insecurities for hackers as gig workers, and how bounty programs rely on vulnerable workers to fix their vulnerable systems. Ellis and Stevens’s research offers a historical overview of bounty programs and an analysis of contemporary bug bounty platforms—the new intermediaries that now structure the vast majority of bounty work. The report draws directly from interviews with hackers, who recount that bounty programs seem willing to integrate a diverse workforce in their practices, but only on terms that deny them the job security and access enjoyed by core security workforces. These inequities go far beyond the difference experienced by temporary and permanent employees at companies such as Google and Apple, contend the authors. The global bug bounty workforce is doing piecework—they are paid for each bug, and the conditions under which a bug is paid vary greatly from one company to the next. Bounty Everything offers to reimagine how bounty programs can better serve the interests of both computer security and the workers that protect our digital world. Ellis & Stevens argue that if bounty programs are not designed and implemented properly, “this model can ironically perpetuate a world full of bugs that uses a global pool of insecure workers to prop up a business model centered on rapid iteration and perpetual beta.
more »
« less
- Award ID(s):
- 1915815
- PAR ID:
- 10467281
- Publisher / Repository:
- Data and Society
- Date Published:
- Edition / Version:
- 1
- Subject(s) / Keyword(s):
- Bug Bounty Programs Bugs Vulnerability Disclosure Cybersecurity Hackers Labor Precarity Gig Work
- Format(s):
- Medium: X Size: 1 MB Other: PDF
- Size(s):
- 1 MB
- Sponsoring Org:
- National Science Foundation
More Like this
-
-
Although researchers have characterized the bug-bounty ecosystem from the point of view of platforms and programs, minimal effort has been made to understand the perspectives of the main workers: bug hunters. To improve bug bounties, it is important to understand hunters’ motivating factors, challenges, and overall benefits. We address this research gap with three studies: identifying key factors through a free listing survey (n=56), rating each factor’s importance with a larger-scale factor-rating survey (n=159), and conducting semi-structured interviews to uncover details (n=24). Of 54 factors that bug hunters listed, we find that rewards and learning opportunities are the most important benefits. Further, we find scope to be the top differentiator between programs. Surprisingly, we find earning reputation to be one of the least important motivators for hunters. Of the challenges we identify, communication problems, such as unresponsiveness and disputes, are the most substantial. We present recommendations to make the bug-bounty ecosystem accommodating to more bug hunters and ultimately increase participation in an underutilized market.more » « less
-
Variability in C software is a useful tool, but critical bugs that only exist in certain configurations are easily missed by conventional debugging techniques. Even with a small number of features, the configuration space of configurable software is too large to analyze exhaustively. Variability-aware static analysis for bug detection is being developed, but remains at too early a stage to be fully usable in real-world C programs. In this work, we present a methodology of finding variability bugs by combining variability-oblivious bug detectors, static analysis of build processes, and dynamic feature interaction inference. We further present an empirical study in which we test our methodology on two highly configurable C programs. We found our methodology to be effective, finding 88 true bugs between the two programs, of which 64 were variability bugs.more » « less
-
Deep Learning (DL) is a class of machine learning algorithms that are used in a wide variety of applications. Like any software system, DL programs can have bugs. To support bug localization in DL programs, several tools have been proposed in the past. As most of the bugs that occur due to improper model structure known as structural bugs lead to inadequate performance during training, it is challenging for developers to identify the root cause and address these bugs. To support bug detection and localization in DL programs, in this article, we propose Theia, which detects and localizes structural bugs in DL programs. Unlike the previous works, Theia considers the training dataset characteristics to automatically detect bugs in DL programs developed using two DL libraries,KerasandPyTorch. Since training the DL models is a time-consuming process, Theia detects these bugs at the beginning of the training process and alerts the developer with informative messages containing the bug’s location and actionable fixes which will help them to improve the structure of the model. We evaluated Theia on a benchmark of 40 real-world buggy DL programs obtained fromStack Overflow. Our results show that Theia successfully localizes 57/75 structural bugs in 40 buggy programs, whereas NeuraLint, a state-of-the-art approach capable of localizing structural bugs before training localizes 17/75 bugs.more » « less
-
There is an increasing need for knowledgeable K-12 computer science (CS) teachers. It is necessary to inform teachers how to debug and help their students debug programs. Research has shown that debugging is difficult for novices because the process requires different skills from creating programs and instructing students how to debug can help them acquire these skills. To this end, we developed a CS professional development for middle grade teachers (grades 5th-8th/ages 10-13) that includes lessons on debugging. The teachers completed debugging activities that involved finding bugs in Scratch programs and explaining how they would help their students in debugging. We qualitatively analyzed their responses and found that teachers successfully identified the problem but they struggled to locate it in the code. In considering how they would help students who had such a bug, the teachers often focused on helping the student find a solution for the bug rather than on identifying the problem or its source. Finally, teachers’ ability to identify bugs and the pedagogical strategies to engage students in this process differed based on CS teaching experience and prior CS knowledge. This work contributes to our understanding of teachers’ debugging abilities and advances our knowledge on how to support teachers in teaching their students how to debug their programs.more » « less
