QDEE: Question Difficulty and Expertise Estimation in Community Question Answering Sites
- Award ID(s):
- 1645599
- NSF-PAR ID:
- 10064837
- Date Published:
- Journal Name:
- Twelfth International AAAI Conference on Web and Social Media (ICWSM)
- Format(s):
- Medium: X
- Sponsoring Org:
- National Science Foundation
More Like this
-
Synthesizing QA pairs with a question generator (QG) on the target domain has become a popular approach for domain adaptation of question answering (QA) models. Since synthetic questions are often noisy in practice, existing work adapts scores from a pretrained QA (or QG) model as criteria to select high-quality questions. However, these scores do not directly serve the ultimate goal of improving QA performance on the target domain. In this paper, we introduce a novel idea of training a question value estimator (QVE) that directly estimates the usefulness of synthetic questions for improving the target-domain QA performance. By conducting comprehensive experiments, we show that the synthetic questions selected by QVE can help achieve better target-domain QA performance, in comparison with existing techniques. We additionally show that by using such questions and only around 15% of the human annotations on the target domain, we can achieve comparable performance to the fully-supervised baselines.more » « less