Search for: All records

Since the introduction of the original BERT (i.e., BASE BERT), researchers have developed various customized BERT models with improved performance for specific domains and tasks by exploiting the benefits of transfer learning. Due to the nature of mathematical texts, which often use domain specific vocabulary along with equations and math symbols, we posit that the development of a new BERT model for mathematics would be useful for many mathematical downstream tasks. In this resource paper, we introduce our multi-institutional effort (i.e., two learning platforms and three academic institutions in the US) toward this need: MathBERT, a model created by pre-training the BASE BERT model on a large mathematical corpus ranging from pre-kindergarten (pre-k), to high-school, to college graduate level mathematical content. In addition, we select three general NLP tasks that are often used in mathematics education: prediction of knowledge component, auto-grading open-ended Q&A, and knowledge tracing, to demonstrate the superiority of MathBERT over BASE BERT. Our experiments show that MathBERT outperforms prior best methods by 1.2-22% and BASE BERT by 2-8% on these tasks. In addition, we build a mathematics specific vocabulary ‘mathVocab’ to train with MathBERT. We discover that MathBERT pre-trained with ‘mathVocab’ outperforms MathBERT trained with the BASE BERT vocabulary (i.e., ‘origVocab’). MathBERT is currently being adopted at the participated leaning platforms: Stride, Inc, a commercial educational resource provider, and ASSISTments.org, a free online educational platform. We release MathBERT for public usage at: https://github.com/tbs17/MathBERT. 

Glioblastoma ranks among the most lethal of primary brain malignancies, with glioblastoma stem cells (GSCs) at the apex of tumor cellular hierarchies. Here, to discover novel therapeutic GSC targets, we interrogated gene expression profiles from GSCs, differentiated glioblastoma cells (DGCs), and neural stem cells (NSCs), revealing EYA2 as preferentially expressed by GSCs. Targeting EYA2 impaired GSC maintenance and induced cell cycle arrest, apoptosis, and loss of self-renewal. EYA2 displayed novel localization to centrosomes in GSCs, and EYA2 tyrosine (Tyr) phosphatase activity was essential for proper mitotic spindle assembly and survival of GSCs. Inhibition of the EYA2 Tyr phosphatase activity, via genetic or pharmacological means, mimicked EYA2 loss in GSCs in vitro and extended the survival of tumor-bearing mice. Supporting the clinical relevance of these findings, EYA2 portends poor patient prognosis in glioblastoma. Collectively, our data indicate that EYA2 phosphatase function plays selective critical roles in the growth and survival of GSCs, potentially offering a high therapeutic index for EYA2 inhibitors.