More Like this

1. Fine-Tuning BERT Model for Materials Named Entity Recognition

   https://doi.org/10.1109/BigData52589.2021.9671697  

   Zhao, Xintong; Greenberg, Jane; An, Yuan; Hu, Xiaohua Tony (December 2021, 2021 IEEE International Conference on Big Data (Big Data))

   Scientific literature presents a wellspring of cutting-edge knowledge for materials science, including valuable data (e.g., numerical data from experiment results, material properties and structure). These data are critical for accelerating materials discovery by data-driven machine learning (ML) methods. The challenge is, it is impossible for humans to manually extract and retain this knowledge due to the extensive and growing volume of publications.To this end, we explore a fine-tuned BERT model for extracting knowledge. Our preliminary results show that our fine-tuned Bert model reaches an f-score of 85% for the materials named entity recognition task. The paper covers background, related work, methodology including tuning parameters, and our overall performance evaluation. Our discussion offers insights into our results, and points to directions for next steps. 

   more » « less
2. BOND: Bert-Assisted Open-Domain Named Entity Recognition with Distant Supervision

   https://doi.org/10.1145/3394486.3403149  

   Liang, Chen; Yu, Yue; Jiang, Haoming; Er, Siawpeng; Wang, Ruijia; Zhao, Tuo; Zhang, Chao. (August 2020, ACM SIGKDD International Conference on Knowledge Discovery and Data Mining)

   We study the open-domain named entity recognition (NER) prob- lem under distant supervision. The distant supervision, though does not require large amounts of manual annotations, yields highly in- complete and noisy distant labels via external knowledge bases. To address this challenge, we propose a new computational framework – BOND, which leverages the power of pre-trained language models (e.g., BERT and RoBERTa) to improve the prediction performance of NER models. Specifically, we propose a two-stage training algo- rithm: In the first stage, we adapt the pre-trained language model to the NER tasks using the distant labels, which can significantly improve the recall and precision; In the second stage, we drop the distant labels, and propose a self-training approach to further improve the model performance. Thorough experiments on 5 bench- mark datasets demonstrate the superiority of BOND over existing distantly supervised NER methods. The code and distantly labeled data have been released in https://github.com/cliang1453/BOND. 

   more » « less
3. EMBA: Entity Matching using Multi-Task Learning of BERT with Attention-over-Attention

   Zhang, Jing; Sun, Huan; Ho, Joyce C (March 2024, Proceedings of the 27th International Conference on Extending Database Technology)
4. BERT might be Overkill: A Tiny but Effective Biomedical Entity Linker based on Residual Convolutional Neural Networks

   https://doi.org/10.18653/v1/2021.findings-emnlp.140  

   Lai, Tuan; Ji, Heng; Zhai, ChengXiang (November 2021, Proc. The 2021 Conference on Empirical Methods in Natural Language Processing (EMNLP2021) Findings)
5. Micron-BERT: BERT-based Facial Micro-Expression Recognition

   Han-Seok Seo, Khoa Luu (January 2023, 2023 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR))