Search for: All records

During disaster events, emergency response teams need to draw up the response plan at the earliest possible stage. Social media platforms contain rich information which could help to assess the current situation. In this paper, a novel multi-task multimodal deep learning framework with automatic loss weighting is proposed. Our framework is able to capture the correlation among different concepts and data modalities. The proposed automatic loss weighting method can prevent the tedious manual weight tuning process and improve the model performance. Extensive experiments on a large-scale multimodal disaster dataset from Twitter are conducted to identify post-disaster humanitarian category and infrastructure damage level. The results show that by learning the shared latent space of multiple tasks with loss weighting, our model can outperform all single tasks. 

Abstract: Deep Learning (DL) has made significant changes to a large number of research areas in recent decades. For example, several astonishing Convolutional Neural Network (CNN) models have been built by researchers to fulfill image classification needs using large-scale visual datasets successfully. Transfer Learning (TL) makes use of those pre-trained models to ease the feature learning process for other target domains that contain a smaller amount of training data. Currently, there are numerous ways to utilize features generated by transfer learning. Pre-trained CNN models prepare mid-/high-level features to work for different targeting problem domains. In this paper, a DL feature and model selection framework based on evolutionary programming is proposed to solve the challenges in visual data classification. It automates the process of discovering and obtaining the most representative features generated by the pre-trained DL models for different classification tasks. 

Advancements in Artificial Intelligence (AI), Information Technology, Augmented Reality (AR) and Virtual Reality (VR), and Robotic Automation is transforming jobs in the Architecture, Engineering and Construction (AEC) industries. However, it is also expected that these technologies will lead to job displacement, alter skill profiles for existing jobs, and change how people work. Therefore, preparing the workforce for an economy defined by these technologies is imperative. This ongoing research focuses on developing an immersive learning training curriculum to prepare the future workforce of the building industry. In this paper we are demonstrating a prototype of a mobile AR application to deliver lessons for training in robotic automation for construction industry workers. The application allows a user to interact with a virtual robot manipulator to learn its basic operations. The goal is to evaluate the effectiveness of the AR application by gauging participants' performance using pre and post surveys.