More Like this

1. Unsupervised Replay Strategies for Continual Learning with Limited Data

   https://doi.org/10.1109/IJCNN60899.2024.10650116  

   Bazhenov, Anthony; Dewasurendra, Pahan; Krishnan, Giri P; Delanois, Jean Erik (June 2024, IEEE)

   Artificial neural networks (ANNs) show limited performance with scarce or imbalanced training data and face challenges with continuous learning, such as forgetting previously learned data after new tasks training. In contrast, the human brain can learn continuously and from just a few examples. This research explores the impact of ’sleep’ an unsupervised phase incorporating stochastic network activation with local Hebbian learning rules on ANNs trained incrementally with limited and imbalanced datasets, specifically MNIST and Fashion MNIST. We discovered that introducing a sleep phase significantly enhanced accuracy in models trained with limited data. When a few tasks were trained sequentially, sleep replay not only rescued previously learned information that had been forgotten following new task training but also often enhanced performance in prior tasks, especially those trained with limited data. This study highlights the multifaceted role of sleep replay in augmenting learning efficiency and facilitating continual learning in ANNs. 

   more » « less
2. Towards Replay-resilient RFID Authentication

   https://doi.org/10.1145/3241539.3241541  

   Wang, Ge; Cai, Haofan; Qian, Chen; Han, Jinsong; Li, Xin; Ding, Han; Zhao, Jizhong (January 2018, Proceedings of the 24th Annual International Conference on Mobile Computing and Networking)
3. MicroScope: enabling microarchitectural replay attacks

   https://doi.org/10.1145/3307650.3322228  

   Skarlatos, Dimitrios; Yan, Mengjia; Gopireddy, Bhargava; Sprabery, Read; Torrellas, Josep; Fletcher, Christopher W. (June 2019, International Symposium on Computer Architecture)
4. Experience replay for continual learning

   Rolnick, D.; Ahuja, A.; Schwarz, J.; Lillicrap, T.; Wayne, G. (December 2019, NeurIPS)
5. Processor-Oblivious Record and Replay

   https://doi.org/10.1145/3018743.3018764  

   Utterback, Robert; Agrawal, Kunal; Lee, I-Ting Angelina; Kulkarni, Milind (February 2017, Proceedings of the 22Nd ACM SIGPLAN Symposium on Principles and Practice of Parallel Programming)

   Record-and-replay systems are useful tools for debugging non-deterministic parallel programs by first recording an execution and then replaying that execution to produce the same access pattern. Existing record-and-replay systems generally target thread-based execution models, and record the behaviors and interleavings of individual threads. Dynamic multithreaded languages and libraries, such as the Cilk family, OpenMP, TBB, etc., do not have a notion of threads. Instead, these languages provide a processor-oblivious model of programming, where programs expose task-parallelism using high-level constructs such as spawn/sync without regard to the number of threads/cores available to run the program. Thread-based record-and-replay would violate the processor-oblivious nature of these programs, as they incorporate the number of threads into the recorded information, constraining the replayed execution to the same number of threads. In this paper, we present a processor-oblivious record-and-replay scheme for such languages where record and replay can use different number of processors and both are scheduled using work stealing. We provide theoretical guarantees for our record and replay scheme --- namely that record is optimal for programs with one lock and replay is near-optimal for all cases. In addition, we implemented this scheme in the Cilk Plus runtime system and our evaluation indicates that processor-obliviousness does not cause substantial overheads. 

   more » « less