An official website of the United States government
Abstract Deep learning requires solving a nonconvex optimization problem of a large size to learn a deep neural network (DNN). The current deep learning model is of asingle-grade, that is, it trains a DNN end-to-end, by solving a single nonconvex optimization problem. When the layer number of the neural network is large, it is computationally challenging to carry out such a task efficiently. The complexity of the task comes from learning all weight matrices and bias vectors from one single nonconvex optimization problem of a large size. Inspired by the human education process which arranges learning in grades, we propose a multi-grade learning model: instead of solving one single optimization problem of a large size, we successively solve a number of optimization problems of small sizes, which are organized in grades, to learn a shallow neural network (a network having a few hidden layers) for each grade. Specifically, the current grade is to learn the leftover from the previous grade. In each of the grades, we learn a shallow neural network stacked on the top of the neural network, learned in the previous grades, whose parameters remain unchanged in training of the current and future grades. By dividing the task of learning a DDN into learning several shallow neural networks, one can alleviate the severity of the nonconvexity of the original optimization problem of a large size. When all grades of the learning are completed, the final neural network learned is astair-shapeneural network, which is thesuperpositionof networks learned from all grades. Such a model enables us to learn a DDN much more effectively and efficiently. Moreover, multi-grade learning naturally leads to adaptive learning. We prove that in the context of function approximation if the neural network generated by a new grade is nontrivial, the optimal error of a new grade is strictly reduced from the optimal error of the previous grade. Furthermore, we provide numerical examples which confirm that the proposed multi-grade model outperforms significantly the standard single-grade model and is much more robust to noise than the single-grade model. They include three proof-of-concept examples, classification on two benchmark data sets MNIST and Fashion MNIST with two noise rates, which is to find classifiers, functions of 784 dimensions, and as well as numerical solutions of the one-dimensional Helmholtz equation.
more »
« less
A deep learning solution for crystallographic structure determination
The generalde novosolution of the crystallographic phase problem is difficult and only possible under certain conditions. This paper develops an initial pathway to a deep learning neural network approach for the phase problem in protein crystallography, based on a synthetic dataset of small fragments derived from a large well curated subset of solved structures in the Protein Data Bank (PDB). In particular, electron-density estimates of simple artificial systems are produced directly from corresponding Patterson maps using a convolutional neural network architecture as a proof of concept.
more »
« less
- PAR ID:
- 10512516
- Editor(s):
- Moffat, K
- Publisher / Repository:
- CC
- Date Published:
- Journal Name:
- IUCrJ
- Volume:
- 10
- Issue:
- 4
- ISSN:
- 2052-2525
- Page Range / eLocation ID:
- 487 to 496
- Format(s):
- Medium: X
- Sponsoring Org:
- National Science Foundation
More Like this
-
-
In this paper, we propose a new deep learning method for the nonlinear Poisson-Boltzmann problems with applications in computational biology. To tackle the discontinuity of the solution, e.g., across protein surfaces, we approximate the solution by a piecewise mesh-free neural network that can capture the dramatic change in the solution across the interface. The partial differential equation problem is first reformulated as a least-squares physics-informed neural network (PINN)-type problem and then discretized to an objective function using mean squared error via sampling. The solution is obtained by minimizing the designed objective function via standard training algorithms such as the stochastic gradient descent method. Finally, the effectiveness and efficiency of the neural network are validated using complex protein interfaces on various manufactured functions with different frequencies.more » « less
-
ABSTRACT Komagataella phaffii, also known asPichia pastoris, is a powerful host for recombinant protein production, in part due to its exceptionally strong and tightly controlled PAOX1promoter. MostK. phaffiibioprocesses for recombinant protein production rely on PAOX1to achieve dynamic control in two‐phase processes. Cells are first grown under conditions that repress PAOX1(growth phase), followed by methanol‐induced recombinant protein expression (production phase). In this study, we propose a methanol‐free approach for dynamic metabolic control inK. phaffiiusing optogenetics, which can help enhance input tunability and flexibility in process optimization and control. The light‐responsive transcription factor EL222 fromErythrobacter litoralisis used to regulate protein production from the PC120promoter inK. phaffiiwith blue light. We used two system designs to explore the advantages and disadvantages of coupling or decoupling EL222 integration with that of the gene of interest. We investigate the relationship between EL222 gene copy number and light dosage to improve production efficiency for intracellular and secreted proteins. Experiments in lab‐scale bioreactors demonstrate the feasibility of the outlined optogenetic systems as potential alternatives to conventional methanol‐inducible bioprocesses usingK. phaffii.more » « less
-
Abstract High‐resolution seismic models of the Earth's lithosphere are critical for understanding its structure and evolution, yet current global models lack the details that can be provided by ambient noise data. A primary bottleneck is reliably extracting phase velocities from the vast, often noisy data sets produced by ambient noise cross‐correlations (noise correlation functions). To address this, we introduceAkiNet, a Physics‐Informed Neural Network designed as a “zero‐shot” solver for this difficult inverse problem. Unlike supervised learning approaches, AkiNet operates without pre‐training or labeled data by directly embedding the governing physics of wave propagation (Aki's theoretical framework) into its loss function. When compared against a modern waveform‐fitting algorithm, AkiNet yields more reliable dispersion estimates, particularly for Love wave data with low signal‐to‐noise ratios. AkiNet provides a robust and scalable tool that presents a feasible pathway toward the construction of a global, high‐resolution, and noise‐derived dispersion model.more » « less
-
Asymmetric cell division (ACD) allows stem cells to generate differentiating progeny while simultaneously maintaining their own pluripotent state. ACD involves coupling mitotic spindle orientation with cortical polarity cues to direct unequal segregation of cell fate determinants. InDrosophilaneural stem cells (neuroblasts; NBs), spindles orient along an apical-basal polarity axis through a conserved complex of Partner of Inscuteable (Pins; human LGN) and Mushroom body defect (Mud; human NuMA). While many details of its function are well known, the molecular mechanics that drive assembly of the cortical Pins/Mud complex remain unclear, particularly with respect to the mutually exclusive Pins complex formed with the apical scaffold protein Inscuteable (Insc). Here we identify Hu li tai shao (Hts; human Adducin) as a direct Mud-binding protein, using an aldolase fold within its head domain (HtsHEAD) to bind a short Mud coiled-coil domain (MudCC) that is adjacent to the Pins-binding domain (MudPBD). Hts is expressed throughout the larval central brain and apically polarizes in mitotic NBs where it is required for Mud-dependent spindle orientation.In vitroanalyses reveal that Pins undergoes liquid-liquid phase separation with Mud, but not with Insc, suggesting a potential molecular basis for differential assembly mechanics between these two competing apical protein complexes. Furthermore, we find that Hts binds an intact Pins/Mud complex, reduces the concentration threshold for its phase separation, and alters the liquid-like property of the resulting phase separated droplets. Domain mapping and mutational analyses implicate critical roles for both multivalent interactions (via MudCColigomerization) and protein disorder (via an intrinsically disordered region in Hts; HtsIDR) in phase separation of the Hts/Mud/Pins complex. Our study identifies a new component of the spindle positioning machinery in NBs and suggests that phase separation of specific protein complexes might regulate ordered assembly within the apical domain to ensure proper signaling output.more » « less
- Free Publicly Accessible Full Text
-
Accepted Manuscript
- Journal Article:
- https://doi.org/10.1107/S2052252523004293
