More Like this

1. BigFuzz: efficient fuzz testing for data analytics using framework abstraction

   https://doi.org/10.1145/3324884.3416641  

   Zhang, Qian; Wang, Jiyuan; Gulzar, Muhammad Ali; Padhye, Rohan; Kim, Miryung (December 2020, The 35th IEEE/ACM International Conference on Automated Software Engineering)

   null (Ed.)

   As big data analytics become increasingly popular, data-intensive scalable computing (DISC) systems help address the scalability issue of handling large data. However, automated testing for such data-centric applications is challenging, because data is often incomplete, continuously evolving, and hard to know a priori. Fuzz testing has been proven to be highly effective in other domains such as security; however, it is nontrivial to apply such traditional fuzzing to big data analytics directly for three reasons: (1) the long latency of DISC systems prohibits the applicability of fuzzing: naïve fuzzing would spend 98% of the time in setting up a test environment; (2) conventional branch coverage is unlikely to scale to DISC applications because most binary code comes from the framework implementation such as Apache Spark; and (3) random bit or byte level mutations can hardly generate meaningful data, which fails to reveal real-world application bugs. We propose a novel coverage-guided fuzz testing tool for big data analytics, called BigFuzz. The key essence of our approach is that: (a) we focus on exercising application logic as opposed to increasing framework code coverage by abstracting the DISC framework using specifications. BigFuzz performs automated source to source transformations to construct an equivalent DISC application suitable for fast test generation, and (b) we design schema-aware data mutation operators based on our in-depth study of DISC application error types. BigFuzz speeds up the fuzzing time by 78 to 1477X compared to random fuzzing, improves application code coverage by 20% to 271%, and achieves 33% to 157% improvement in detecting application errors. When compared to the state of the art that uses symbolic execution to test big data analytics, BigFuzz is applicable to twice more programs and can find 81% more bugs. 

   more » « less
2. Multi-level Fuzzing for Document File Formats with Intermediate Representations

   https://doi.org/10.1109/ISSRE62328.2024.00037  

   Wang, Yifan; Xu, Jun (October 2024, IEEE)

   This paper focuses on fuzzing document software or precisely, software that processes document files (e.g., HTML, PDF, and DOCX). Document software typically requires highly-structured inputs, which general-purpose fuzzing cannot handle well. We propose two techniques to facilitate fuzzing on document software. First, we design an intermediate document representation (DIR) for document files. DIR describes a document file in an abstract way that is independent of the underlying format. Reusing common SDKs, a DIR document can be lowered into a desired format without a deep understanding of the format. Second, we propose multi-level mutations to operate directly on a DIR document, which can more thoroughly explore the searching space than existing single-level mutations. Combining these two techniques, we can reuse the same DIR-based generations and mutations to fuzz any document format, without separately handling the target format and re-engineering the generation/mutation components.To assess utility of our DIR-based fuzzing, we applied it to 6 PDF and 6 HTML applications (48-hour) demonstrated superior performance, outpacing general mutation-based fuzzing (AFL++), ML-based PDF fuzzing (Learn&Fuzz), and structure-aware mutation-based fuzzing ((NAUTILUS) by 33.87%, 127.74%, and 25.17% in code coverage, respectively. For HTML, it exceeded AFL++ and generation-based methods (FreeDom and Domato) by 28.8% and 14.02%. 

   more » « less
3. FUME: Fuzzing Message Queuing Telemetry Transport Brokers

   https://doi.org/10.1109/INFOCOM48880.2022.9796755  

   Pearson, Bryan; Zhang, Yue; Zou, Cliff; Fu, Xinwen (May 2022, IEEE INFOCOM 2022 - IEEE Conference on Computer Communications)

   Message Queuing Telemetry Transport (MQTT) is a popular communication protocol used to interconnect devices with considerable network restraints, such as those found in Internet of Things (IoT). MQTT directly impacts a large number of devices, but the software security of its server ("broker") implementations is not well studied. In this paper, we design, implement, and evaluate a novel fuzz testing model for MQTT. The fuzzer combines aspects of mutation guided fuzzing and generation guided fuzzing to rigorously exhaust the MQTT protocol and identify vulnerabilities in servers. We introduce Markov chains for mutation guided fuzzing and generation guided fuzzing that model the fuzzing engine according to a finite Bernoulli process. We implement "response feedback", a novel technique which monitors network and console activity to learn which inputs trigger new responses from the broker. In total, we found 7 major vulnerabilities across 9 different MQTT implementations, including 6 zero-day vulnerabilities and 2 CVEs. We show that when fuzzing these popular MQTT targets, our fuzzer compares favorably with other state-of-the-art fuzzing frameworks, such as BooFuzz and AFLNet. 

   more » « less
4. From Quantum Fuzzing to the Multiverse: Possible Effective Uses of Quantum Noise

   https://doi.org/10.1007/978-3-030-98012-2_30  

   Rosch-Grace, Dominic; Straub, Jeremy (March 2022, Advances in Information and Communication. FICC 2022)

   Arai, Kohei (Ed.)

   Quantum noise is seen by many researchers as a problem to be resolved. Current solutions increase quantum computing system costs significantly by requiring numerous hardware qubits to represent a logical qubit to average the noise away. However, despite its deleterious effects on system performance and the increased costs it creates, it may have some potential uses. This paper evaluates those. Specifically, it considers how quantum noise could be used to support the fuzzing cybersecurity and testing technique and AI techniques such as certain swarm artificial intelligence algorithms. Fuzzing is used to identify vulnerabilities in software by generating massive amounts of input cases for a program. Quantum noise provides an effective built-in fuzzing capability that is centered around the actual answer to a computation. These same phenomena, of clustered and centered fuzz-noise around the answer of an operation, could be similarly useful to AI techniques that can make effective use of lots of point values for optimization. Effectively, by concurrently considering the ‘multiverse’ of possible results to an operation, created by compounding noise, more beneficial solutions that are proximal to the actual result of an operation can be identified via testing quantum noise points with an effectiveness algorithm. Both of these potential uses for quantum noise are considered herein. 

   more » « less
5. Guiding Greybox Fuzzing with Mutation Testing

   https://doi.org/10.1145/3597926.3598107  

   Vikram, Vasudev; Laybourn, Isabella; Li, Ao; Nair, Nicole; OBrien, Kelton; Sanna, Rafaello; Padhye, Rohan (July 2023, ISSTA 2023: Proceedings of the 32nd ACM SIGSOFT International Symposium on Software Testing and Analysis)

   Greybox fuzzing and mutation testing are two popular but mostly independent fields of software testing research that have so far had limited overlap. Greybox fuzzing, generally geared towards searching for new bugs, predominantly uses code coverage for selecting inputs to save. Mutation testing is primarily used as a stronger alternative to code coverage in assessing the quality of regression tests; the idea is to evaluate tests for their ability to identify artificially injected faults in the target program. But what if we wanted to use greybox fuzzing to synthesize high-quality regression tests? In this paper, we develop and evaluate Mu2, a Java-based framework for incorporating mutation analysis in the greybox fuzzing loop, with the goal of producing a test-input corpus with a high mutation score. Mu2 makes use of a differential oracle for identifying inputs that exercise interesting program behavior without causing crashes. This paper describes several dynamic optimizations implemented in Mu2 to overcome the high cost of performing mutation analysis with every fuzzer-generated input. These optimizations introduce trade-offs in fuzzing throughput and mutation killing ability, which we evaluate empirically on five real-world Java benchmarks. Overall, variants of Mu2 are able to synthesize test-input corpora with a higher mutation score than state-of-the-art Java fuzzer Zest. 

   more » « less