An official website of the United States government
This paper presents Tofu, a system that partitions very large DNN models across multiple GPU devices to reduce per-GPU memory footprint. Tofu is designed to partition a dataflow graph of fine-grained tensor operators used by platforms like MXNet and TensorFlow. In order to automatically partition each operator, we propose to describe the semantics of an operator in a simple language inspired by Halide. To optimally partition different operators in a dataflow graph, Tofu uses a recursive search algorithm that minimizes the total communication cost. Our experiments on an 8-GPU machine show that Tofu enables the training of very large CNN and RNN models. It also achieves 25% - 400% speedup over alternative approaches to train very large models.
more »
« less
ToFU: Topology functional units for deep learning
We propose ToFU, a new trainable neural network unit with a persistence diagram dissimilarity function as its activation. Since persistence diagrams are topological summaries of structures, this new activation measures and learns the topology of data to leverage it in machine learning tasks. We showcase the utility of ToFU in two experiments: one involving the classification of discrete-time autoregressive signals, and another involving a variational autoencoder. In the former, ToFU yields competitive results with networks that use spectral features while outperforming CNN architectures. In the latter, ToFU produces topologically-interpretable latent space representations of inputs without sacrificing reconstruction fidelity.
more »
« less
- Award ID(s):
- 1821241
- PAR ID:
- 10303167
- Date Published:
- Journal Name:
- Foundations of Data Science
- Volume:
- 0
- Issue:
- 0
- ISSN:
- 2639-8001
- Format(s):
- Medium: X
- Sponsoring Org:
- National Science Foundation
More Like this
-
-
The process of fermenting tofu extends back thousands of years and is an indispensable part of Chinese culture. Despite a cultural resurgence in fermented foods and interest in microbiomes, there is little knowledge on the microbial diversity represented in fermented ‘hairy’ tofu, known locally in China as Mao tofu. High-throughput metagenomic sequencing of the ITS, LSU and 16S rDNA was used to determine Mao tofu’s fungal and bacterial community diversity across four wet markets in Yunnan, China. The results show that hairy tofu in this region consists of around 170 fungal and 365 bacterial taxa, and that microbial taxa differ between markets. Diversity also differed based on the specific niche of the tofu block, comparing the outside rind-like niche to that of the inside of the tofu block. Machine learning random forest models were able to accurately classify both the market and niche of sample origin. An over-abundance of yeast and Geotrichum was found, and Mucor (Mucoromycota) was abundant in the outside rind-like niche, which consists of the visible ‘hairy’ mycelium. The majority of the bacterial OTUs belonged to Proteobacteria, Firmicutes, and Bacteroidetes, with Acinetobacter, Lactobacillus, Sphingobacterium and Flavobacterium the most abundant genera. Putative fungal pathogens of plants (Cercospora, Diaporthe, Fusarium) and animals (Metarhizium, Entomomortierella, Pyxidiophora, Candida, Clavispora) were also detected, as were putative bacterial pathogens identified as Legionella. Non-fungal eukaryotic taxa detected by LSU amplicon sequencing included soybean (Glycine max), Protozoa, Metazoa (e.g., Nematoda and Platyhelminthes), Rhizaria and Chromista, indicating that additional biodiversity exists in the hairy tofu microbiome.more » « less
-
Sleep is a fundamental feature of life for virtually all multicellular animals, but many questions remain about how sleep is regulated by circadian rhythms, homeostatic sleep drive that builds up with wakefulness, and modifying factors such as hunger or social interactions, as well as about the biological functions of sleep. Substantial headway has been made in the study of both circadian rhythms and sleep in the fruit flyDrosophila melanogaster, much of it through studies of individual fly activity usingDrosophilaactivity monitors (DAMs). Here, we describe approaches for the activation of specific neurons of interest using optogenetics (involving genetic modifications that allow for light-based neuronal activation) and thermogenetics (involving genetic modifications that allow for temperature-based neuronal activation) so that researchers can evaluate the roles of those neurons in controlling rest and activity behavior. In this protocol, we describe how to set up a rig for simultaneous optogenetic or thermogenetic stimulation and activity monitoring for analysis of sleep and circadian rhythms inDrosophila, how to raise appropriate flies, and how to perform the experiment. This protocol will allow researchers to assess the causative role in the regulation of sleep and activity rhythms of any genetically tractable subset of cells.more » « less
-
Abstract Chimeric antigen receptor (CAR) T cell therapy has achieved remarkable clinical success in the treatment of hematological malignancies. However, producing these bespoke cancer‐killing cells is a complicated ex vivo process involving leukapheresis, artificial T cell activation, and CAR construct introduction. The activation step requires the engagement of CD3/TCR and CD28 and is vital for T cell transfection and differentiation. Though antigen‐presenting cells (APCs) facilitate activation in vivo, ex vivo activation relies on antibodies against CD3 and CD28 conjugated to magnetic beads. While effective, this artificial activation adds to the complexity of CAR T cell production as the beads must be removed prior to clinical implementation. To overcome this challenge, this work develops activating lipid nanoparticles (aLNPs) that mimic APCs to combine the activation of magnetic beads and the transfection capabilities of LNPs. It is shown that aLNPs enable one‐step activation and transfection of primary human T cells with the resulting mRNA CAR T cells reducing tumor burden in a murine xenograft model, validating aLNPs as a promising platform for the rapid production of mRNA CAR T cells.more » « less
-
Abstract Microtubules regulate cell polarity and migration via local activation of focal adhesion turnover, but the mechanism of this process is insufficiently understood. Molecular complexes containing KANK family proteins connect microtubules with talin, the major component of focal adhesions. Here, local optogenetic activation of KANK1-mediated microtubule/talin linkage promoted microtubule targeting to an individual focal adhesion and subsequent withdrawal, resulting in focal adhesion centripetal sliding and rapid disassembly. This sliding is preceded by a local increase of traction force due to accumulation of myosin-II and actin in the proximity of the focal adhesion. Knockdown of the Rho activator GEF-H1 prevented development of traction force and abolished sliding and disassembly of focal adhesions upon KANK1 activation. Other players participating in microtubule-driven, KANK-dependent focal adhesion disassembly include kinases ROCK, PAK, and FAK, as well as microtubules/focal adhesion-associated proteins kinesin-1, APC, and αTAT. Based on these data, we develop a mathematical model for a microtubule-driven focal adhesion disruption involving local GEF-H1/RhoA/ROCK-dependent activation of contractility, which is consistent with experimental data.more » « less
- Free Publicly Accessible Full Text
-
Accepted Manuscript1.0
- Journal Article:
- https://doi.org/10.3934/fods.2021021
