skip to main content


Title: How Widely Can Prediction Models Be Generalized? Performance Prediction in Blended Courses
Award ID(s):
1821475
NSF-PAR ID:
10119194
Author(s) / Creator(s):
; ; ; ;
Date Published:
Journal Name:
IEEE Transactions on Learning Technologies
Volume:
12
Issue:
2
ISSN:
2372-0050
Page Range / eLocation ID:
184 to 197
Format(s):
Medium: X
Sponsoring Org:
National Science Foundation
More Like this
  1. This paper describes our pathloss prediction system submitted to the ICASSP 2023 First Pathloss Radio Map Prediction Challenge. We describe the architecture of PMNet, a neural network we specifically designed for pathloss prediction. Moreover, to enhance the prediction performance, we apply several machine learning techniques, including data augmentation, fine-tuning, and optimization of the network architecture. Our system achieves an RMSE of 0.02569 on the provided RadioMap3Dseer dataset, and 0.0383 on the challenge test set, placing it in the 1st rank of the challenge. 
    more » « less
  2. Remaining Useful Life (RUL) estimation is critical in many engineering systems where proper predictive maintenance is needed to increase a unit's effectiveness and reduce time and cost of repairing. Typically for such systems, multiple sensors are normally used to monitor performance, which create difficulties for system state identification. In this paper, we develop a semi-supervised left-to-right constrained Hidden Markov Model (HMM) model, which is effective in estimating the RUL, while capturing the jumps among states in condition dynamics. In addition, based on the HMM model learned from multiple sensors, we build a Partial Observable Markov Decision Process (POMDP) to demonstrate how such RUL estimation can be effectively used for optimal preventative maintenance decision making. We apply this technique to the NASA Engine degradation data and demonstrate the effectiveness of the proposed method. 
    more » « less
  3. Modern machine learning algorithms are capable of providing remarkably accurate point-predictions; however, questions remain about their statistical reliability. Unlike conventional machine learning methods, conformal prediction algorithms return confidence sets (i.e., set-valued predictions) that correspond to a given significance level. Moreover, these confidence sets are valid in the sense that they guarantee finite sample control over type 1 error probabilities, allowing the practitioner to choose an acceptable error rate. In our paper, we propose inductive conformal prediction (ICP) algorithms for the tasks of text infilling and part-of-speech (POS) prediction for natural language data. We construct new ICP-enhanced algorithms for POS tagging based on BERT (bidirectional encoder representations from transformers) and BiLSTM (bidirectional long short-term memory) models. For text infilling, we design a new ICP-enhanced BERT algorithm. We analyze the performance of the algorithms in simulations using the Brown Corpus, which contains over 57,000 sentences. Our results demonstrate that the ICP algorithms are able to produce valid set-valued predictions that are small enough to be applicable in real-world applications. We also provide a real data example for how our proposed set-valued predictions can improve machine generated audio transcriptions. 
    more » « less