More Like this

1. User Response Prediction in Online Advertising

   https://doi.org/10.1145/3446662  

   Gharibshah, Zhabiz ; Zhu, Xingquan ( June 2021 , ACM Computing Surveys)

   null (Ed.)

   Online advertising, as a vast market, has gained significant attention in various platforms ranging from search engines, third-party websites, social media, and mobile apps. The prosperity of online campaigns is a challenge in online marketing and is usually evaluated by user response through different metrics, such as clicks on advertisement (ad) creatives, subscriptions to products, purchases of items, or explicit user feedback through online surveys. Recent years have witnessed a significant increase in the number of studies using computational approaches, including machine learning methods, for user response prediction. However, existing literature mainly focuses on algorithmic-driven designs to solve specific challenges, and no comprehensive review exists to answer many important questions. What are the parties involved in the online digital advertising eco-systems? What type of data are available for user response prediction? How do we predict user response in a reliable and/or transparent way? In this survey, we provide a comprehensive review of user response prediction in online advertising and related recommender applications. Our essential goal is to provide a thorough understanding of online advertising platforms, stakeholders, data availability, and typical ways of user response prediction. We propose a taxonomy to categorize state-of-the-art user response prediction methods, primarily focusing on the current progress of machine learning methods used in different online platforms. In addition, we also review applications of user response prediction, benchmark datasets, and open source codes in the field. 

   more » « less
2. Modeling Aesthetics and Emotions in Visual Content: From Vincent van Gogh to Robotics and Vision

   Wang, James Z. ( October 2020 , Proceedings of the Joint Workshop on Aesthetic and Technical Quality Assessment of Multimedia and Media Analytics for Societal Trends, in conjunction with the ACM International Conference on Multimedia)

   null (Ed.)

   As inborn characteristics, humans possess the ability to judge visual aesthetics, feel the emotions from the environment, and comprehend others’ emotional expressions. Many exciting applications become possible if robots or computers can be empowered with similar capabilities. Modeling aesthetics, evoked emotions, and emotional expressions automatically in unconstrained situations, however, is daunting due to the lack of a full understanding of the relationship between low-level visual content and high-level aesthetics or emotional expressions. With the growing availability of data, it is possible to tackle these problems using machine learning and statistical modeling approaches. In the talk, I provide an overview of our research in the last two decades on data-driven analyses of visual artworks and digital visual content for modeling aesthetics and emotions. First, I discuss our analyses of styles in visual artworks. Art historians have long observed the highly characteristic brushstroke styles of Vincent van Gogh and have relied on discerning these styles for authenticating and dating his works. In our work, we compared van Gogh with his contemporaries by statistically analyzing a massive set of automatically extracted brushstrokes. A novel extraction method is developed by exploiting an integration of edge detection and clustering-based segmentation. Evidence substantiates that van Gogh’s brushstrokes are strongly rhythmic. Next, I describe an effort to model the aesthetic and emotional characteristics in visual contents such as photographs. By taking a data-driven approach, using the Internet as the data source, we show that computers can be trained to recognize various characteristics that are highly relevant to aesthetics and emotions. Future computer systems equipped with such capabilities are expected to help millions of users in unimagined ways. Finally, I highlight our research on automated recognition of bodily expression of emotion. We propose a scalable and reliable crowdsourcing approach for collecting in-the-wild perceived emotion data for computers to learn to recognize the body language of humans. Comprehensive statistical analysis revealed many interesting insights from the dataset. A system to model the emotional expressions based on bodily movements, named ARBEE (Automated Recognition of Bodily Expression of Emotion), has also been developed and evaluated. 

   more » « less
3. Issues in the Reproducibility of Deep Learning Results

   https://doi.org/10.1109/SPMB47826.2019.9037840  

   Jean-Paul, S. ; Elseify, T. ; Obeid, I. ; Picone, J. ( December 2019 , IEEE Signal Processing in Medicine and Biology Symposium (SPMB))

   Obeid, I. ; Selesnik, I. ; Picone, J. (Ed.)

   The Neuronix high-performance computing cluster allows us to conduct extensive machine learning experiments on big data [1]. This heterogeneous cluster uses innovative scheduling technology, Slurm [2], that manages a network of CPUs and graphics processing units (GPUs). The GPU farm consists of a variety of processors ranging from low-end consumer grade devices such as the Nvidia GTX 970 to higher-end devices such as the GeForce RTX 2080. These GPUs are essential to our research since they allow extremely compute-intensive deep learning tasks to be executed on massive data resources such as the TUH EEG Corpus [2]. We use TensorFlow [3] as the core machine learning library for our deep learning systems, and routinely employ multiple GPUs to accelerate the training process. Reproducible results are essential to machine learning research. Reproducibility in this context means the ability to replicate an existing experiment – performance metrics such as error rates should be identical and floating-point calculations should match closely. Three examples of ways we typically expect an experiment to be replicable are: (1) The same job run on the same processor should produce the same results each time it is run. (2) A job run on a CPU and GPU should produce identical results. (3) A job should produce comparable results if the data is presented in a different order. System optimization requires an ability to directly compare error rates for algorithms evaluated under comparable operating conditions. However, it is a difficult task to exactly reproduce the results for large, complex deep learning systems that often require more than a trillion calculations per experiment [5]. This is a fairly well-known issue and one we will explore in this poster. Researchers must be able to replicate results on a specific data set to establish the integrity of an implementation. They can then use that implementation as a baseline for comparison purposes. A lack of reproducibility makes it very difficult to debug algorithms and validate changes to the system. Equally important, since many results in deep learning research are dependent on the order in which the system is exposed to the data, the specific processors used, and even the order in which those processors are accessed, it becomes a challenging problem to compare two algorithms since each system must be individually optimized for a specific data set or processor. This is extremely time-consuming for algorithm research in which a single run often taxes a computing environment to its limits. Well-known techniques such as cross-validation [5,6] can be used to mitigate these effects, but this is also computationally expensive. These issues are further compounded by the fact that most deep learning algorithms are susceptible to the way computational noise propagates through the system. GPUs are particularly notorious for this because, in a clustered environment, it becomes more difficult to control which processors are used at various points in time. Another equally frustrating issue is that upgrades to the deep learning package, such as the transition from TensorFlow v1.9 to v1.13, can also result in large fluctuations in error rates when re-running the same experiment. Since TensorFlow is constantly updating functions to support GPU use, maintaining an historical archive of experimental results that can be used to calibrate algorithm research is quite a challenge. This makes it very difficult to optimize the system or select the best configurations. The overall impact of all of these issues described above is significant as error rates can fluctuate by as much as 25% due to these types of computational issues. Cross-validation is one technique used to mitigate this, but that is expensive since you need to do multiple runs over the data, which further taxes a computing infrastructure already running at max capacity. GPUs are preferred when training a large network since these systems train at least two orders of magnitude faster than CPUs [7]. Large-scale experiments are simply not feasible without using GPUs. However, there is a tradeoff to gain this performance. Since all our GPUs use the NVIDIA CUDA® Deep Neural Network library (cuDNN) [8], a GPU-accelerated library of primitives for deep neural networks, it adds an element of randomness into the experiment. When a GPU is used to train a network in TensorFlow, it automatically searches for a cuDNN implementation. NVIDIA’s cuDNN implementation provides algorithms that increase the performance and help the model train quicker, but they are non-deterministic algorithms [9,10]. Since our networks have many complex layers, there is no easy way to avoid this randomness. Instead of comparing each epoch, we compare the average performance of the experiment because it gives us a hint of how our model is performing per experiment, and if the changes we make are efficient. In this poster, we will discuss a variety of issues related to reproducibility and introduce ways we mitigate these effects. For example, TensorFlow uses a random number generator (RNG) which is not seeded by default. TensorFlow determines the initialization point and how certain functions execute using the RNG. The solution for this is seeding all the necessary components before training the model. This forces TensorFlow to use the same initialization point and sets how certain layers work (e.g., dropout layers). However, seeding all the RNGs will not guarantee a controlled experiment. Other variables can affect the outcome of the experiment such as training using GPUs, allowing multi-threading on CPUs, using certain layers, etc. To mitigate our problems with reproducibility, we first make sure that the data is processed in the same order during training. Therefore, we save the data from the last experiment and to make sure the newer experiment follows the same order. If we allow the data to be shuffled, it can affect the performance due to how the model was exposed to the data. We also specify the float data type to be 32-bit since Python defaults to 64-bit. We try to avoid using 64-bit precision because the numbers produced by a GPU can vary significantly depending on the GPU architecture [11-13]. Controlling precision somewhat reduces differences due to computational noise even though technically it increases the amount of computational noise. We are currently developing more advanced techniques for preserving the efficiency of our training process while also maintaining the ability to reproduce models. In our poster presentation we will demonstrate these issues using some novel visualization tools, present several examples of the extent to which these issues influence research results on electroencephalography (EEG) and digital pathology experiments and introduce new ways to manage such computational issues. 

   more » « less
4. Artificial Intelligence for Smart Transportation

   Michael, Wilbur ; Sivagnanam, Amutheezan ; Ayman, Afiya ; Samitha Samaranayeke ; Abhishek Dubey ; Aron Laszka ( January 2023 , Vorobeychik, Yevgeniy., and Mukhopadhyay, Ayan., (Eds.). (2023). Artificial Intelligence and Society. ACM Press Subjects: Artificial Intelligence (cs.AI) Cite as: arXiv:2308.07457 [cs.AI] (or arXiv:2308.07457v1 [cs.AI] for this version) https://doi.org/10.48550/arXiv.2308.07457 Focus to learn more)

   There are more than 7,000 public transit agencies in the U.S. (and many more private agencies), and together, they are responsible for serving 60 billion passenger miles each year. A well-functioning transit system fosters the growth and expansion of businesses, distributes social and economic benefits, and links the capabilities of community members, thereby enhancing what they can accomplish as a society. Since affordable public transit services are the backbones of many communities, this work investigates ways in which Artificial Intelligence (AI) can improve efficiency and increase utilization from the perspective of transit agencies. This book chapter discusses the primary requirements, objectives, and challenges related to the design of AI-driven smart transportation systems. We focus on three major topics. First, we discuss data sources and data. Second, we provide an overview of how AI can aid decision-making with a focus on transportation. Lastly, we discuss computational problems in the transportation domain and AI approaches to these problems. 

   more » « less
5. Self-Assembling Polypeptides in Complex Coacervation

   https://doi.org/10.1021/acs.accounts.3c00689  

   Sathyavageeswaran, Arvind ; Bonesso Sabadini, Júlia ; Perry, Sarah L. ( February 2024 , Accounts of Chemical Research)

   Intracellular compartmentalization plays a pivotal role in cellular function, with membrane-bound organelles and membrane-less biomolecular 'condensates' playing key roles. These condensates, formed through liquid-liquid phase separation (LLPS), enable selective compartmentalization without the barrier of a lipid bilayer, thereby facilitating rapid formation/dissolution in response to stimuli. Intrinsically disordered proteins (IDPs) and/or proteins with intrinsically disordered regions (IDRs), which are often rich in charged and polar amino acid sequences, scaffold many condensates, often in conjunction with RNA. Comprehending the impact of IDP/IDR sequences on phase separation poses a challenge due to the extensive chemical diversity resulting from the myriad amino acids and post-translational modifications. To tackle this hurdle, one approach has been to investigate LLPS in simplified polypeptide systems, which offer a narrower scope within the chemical space for exploration. This strategy is supported by studies that have demonstrated how IDP function can largely be understood based on general chemical features, such as clusters or patterns of charged amino acids, rather than residue-level effects, and the ways in which these kinds of motifs give rise to an ensemble of conformations. Our lab has utilized complex coacervates assembled from oppositely-charged polypeptides as a simplified material analogue to the complexity of liquid-liquid phase separated biological condensates. Complex coacervation is an associative LLPS that occurs due to the electrostatic complexation of oppositely-charged macro-ions. This process is believed to be driven by the entropic gains resulting from the release of bound counterions and the reorganization of water upon complex formation. Apart from their direct applicability to IDPs, polypeptides also serve as excellent model polymers for investigating molecular interactions due to the wide range of available side-chain functionalities and the capacity to finely regulate their sequence, thus enabling precise control over interactions with guest molecules. Here, we discuss fundamental studies examining how charge patterning, hydrophobicity, chirality, and architecture affect the phase separation of polypeptide-based complex coacervates. These efforts have leveraged a combination of experimental and computational approaches that provide insight into the molecular level interactions. We also examine how these parameters affect the ability of complex coacervates to incorporate globular proteins and viruses. These efforts couple directly with our fundamental studies into coacervate formation, as such ‘guest’ molecules should not be considered as experiencing simple encapsulation and are instead active participants in the electrostatic assembly of coacervate materials. Interestingly, we observed trends in the incorporation of proteins and viruses into coacervates formed using different chain length polypeptides that are not well explained by simple electrostatic arguments and may be the result of more complex interactions between globular and polymeric species. Additionally, we describe experimental evidence supporting the potential for complex coacervates to improve the thermal stability of embedded biomolecules such as viral vaccines. Ultimately, peptide-based coacervates have the potential to help unravel the physics behind biological condensates while paving the way for innovative methods in compartmentalization, purification, and biomolecule stabilization. These advancements could have implications spanning from medicine to biocatalysis. 

   more » « less