More Like this

1. Micron-BERT: BERT-based Facial Micro-Expression Recognition

   Han-Seok Seo, Khoa Luu (January 2023, 2023 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR))
2. BERT-ER: Query-specific BERT Entity Representations for Entity Ranking

   https://doi.org/10.1145/3477495.3531944  

   Chatterjee, Shubham; Dietz, Laura (July 2022, Proceedings of the 45th International ACM SIGIR Conference on Research and Development in Information Retrieval)
3. Understanding BERT rankers under distillation

   https://doi.org/10.1145/3409256.3409838  

   Gao, Luyu; Dai, Zhuyun; Callan, Jamie (September 2020, ACM SIGIR International Conference on the Theory of Information Retrieval)

   null (Ed.)

   Deep language models, such as BERT pre-trained on large corpora,have given a huge performance boost to state-of-the-art information retrieval ranking systems. Knowledge embedded in such models allows them to pick up complex matching signals between passages and queries. However, the high computation cost during inference limits their deployment in real-world search scenarios. In this paper, we study if and how the knowledge for search within BERT can be transferred to a smaller ranker through distillation.Our experiments demonstrate that it is crucial to use a proper distillation procedure, which produces up to nine times speed upwhile preserving the state-of-the-art performance. 

   more » « less
4. Revealing the Dark Secrets of BERT

   https://doi.org/10.18653/v1/D19-1445  

   Kovaleva, Olga; Romanov, Alexey; Rogers, Anna; Rumshisky, Anna (January 2019, Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP))
5. LogBERT: Log Anomaly Detection via BERT

   https://doi.org/10.1109/IJCNN52387.2021.9534113  

   Guo, Haixuan; Yuan, Shuhan; Wu, Xintao (July 2021, 2021 International Joint Conference on Neural Networks (IJCNN))