Search for: All records

Physical computing toolkits for children expose young minds to the concepts of computing and electronics within a target activity. To this end, these kits usually make use of a custom Visual Programming Language (or VPL) environment that extends past the functionality of simply programming, often also incorporating representations of electronics aspects in the interface. These representations of the electronics function as a scaffold to help the child focus on programming, instead of having to handle both the programming and details of the electronics at the same time. This paper presents a review of existing physical computing toolkits, looking at the What, How, and Where of electronics representations in their VPL interfaces. We then discuss potential research directions for the design of VPL interfaces for physical computing toolkits for children. 

We present LBW-Net, an efficient optimization based method for quantization and training of the low bit-width convolutional neural networks (CNNs). Specifically, we quantize the weights to zero or powers of 2 by minimizing the Euclidean distance between full-precision weights and quantized weights during back-propagation (weight learning). We characterize the combinatorial nature of the low bit-width quantization problem. For 2-bit (ternary) CNNs, the quantization of N weights can be done by an exact formula in O(N log N) complexity. When the bit-width is 3 and above, we further propose a semi-analytical thresholding scheme with a single free parameter for quantization that is computationally inexpensive. The free parameter is further determined by network retraining and object detection tests. The LBW-Net has several desirable advantages over full-precision CNNs, including considerable memory savings, energy efficiency, and faster deployment. Our experiments on PASCAL VOC dataset show that compared with its 32-bit floating-point counterpart, the performance of the 6-bit LBW-Net is nearly lossless in the object detection tasks, and can even do better in real world visual scenes, while empirically enjoying more than 4× faster deployment. 

Training activation quantized neural networks involves minimizing a piecewise constant function whose gradient vanishes almost everywhere, which is undesirable for the standard back-propagation or chain rule. An empirical way around this issue is to use a straight-through estimator (STE) (Bengio et al., 2013) in the backward pass only, so that the “gradient” through the modified chain rule becomes non-trivial. Since this unusual “gradient” is certainly not the gradient of loss function, the following question arises: why searching in its negative direction minimizes the training loss? In this paper, we provide the theoretical justification of the concept of STE by answering this question. We consider the problem of learning a two-linear-layer network with binarized ReLU activation and Gaussian input data. We shall refer to the unusual “gradient” given by the STE-modifed chain rule as coarse gradient. The choice of STE is not unique. We prove that if the STE is properly chosen, the expected coarse gradient correlates positively with the population gradient (not available for the training), and its negation is a descent direction for minimizing the population loss. We further show the associated coarse gradient descent algorithm converges to a critical point of the population loss minimization problem. Moreover, we show that a poor choice of STE leads to instability of the training algorithm near certain local minima, which is verified with CIFAR-10 experiments. 

Quantized deep neural networks (QDNNs) are attractive due to their much lower memory storage and faster inference speed than their regular full-precision counterparts. To maintain the same performance level especially at low bit-widths, QDNNs must be retrained. Their training involves piece-wise constant activation functions and discrete weights; hence, mathematical challenges arise. We introduce the notion of coarse gradient and propose the blended coarse gradient descent (BCGD) algorithm, for training fully quantized neural networks. Coarse gradient is generally not a gradient of any function but an artificial ascent direction. The weight update of BCGD goes by coarse gradient correction of a weighted average of the full-precision weights and their quantization (the so-called blending), which yields sufficient descent in the objective value and thus accelerates the training. Our experiments demonstrate that this simple blending technique is very effective for quantization at extremely low bit-width such as binarization. In full quantization of ResNet-18 for ImageNet classification task, BCGD gives 64.36% top-1 accuracy with binary weights across all layers and 4-bit adaptive activation. If the weights in the first and last layers are kept in full precision, this number increases to 65.46%. As theoretical justification, we show convergence analysis of coarse gradient descent for a two-linear-layer neural network model with Gaussian input data and prove that the expected coarse gradient correlates positively with the underlying true gradient. 

We propose BinaryRelax, a simple two-phase algorithm, for training deep neural networks with quantized weights. The set constraint that characterizes the quantization of weights is not imposed until the late stage of training, and a sequence of pseudo quantized weights is maintained. Specifically, we relax the hard constraint into a continuous regularizer via Moreau envelope, which turns out to be the squared Euclidean distance to the set of quantized weights. The pseudo quantized weights are obtained by linearly interpolating between the float weights and their quantizations. A continuation strategy is adopted to push the weights towards the quantized state by gradually increasing the regularization parameter. In the second phase, exact quantization scheme with a small learning rate is invoked to guarantee fully quantized weights. We test BinaryRelax on the benchmark CIFAR and ImageNet color image datasets to demonstrate the superiority of the relaxed quantization approach and the improved accuracy over the state-of-the-art training methods. Finally, we prove the convergence of BinaryRelax under an approximate orthogonality condition. 

A search for pair production of squarks or gluinos decaying via sleptons or weak bosons is reported. The search targets a final state with exactly two leptons with same-sign electric charge or at least three leptons without any charge requirement. The analysed data set corresponds to an integrated luminosity of 139 fb⁻¹of proton-proton collisions collected at a centre-of-mass energy of 13 TeV with the ATLAS detector at the LHC. Multiple signal regions are defined, targeting several SUSY simplified models yielding the desired final states. A single control region is used to constrain the normalisation of theWZ+ jets background. No significant excess of events over the Standard Model expectation is observed. The results are interpreted in the context of several supersymmetric models featuring R-parity conservation or R-parity violation, yielding exclusion limits surpassing those from previous searches. In models considering gluino (squark) pair production, gluino (squark) masses up to 2.2 (1.7) TeV are excluded at 95% confidence level.

 

A search for supersymmetry targeting the direct production of winos and higgsinos is conducted in final states with either two leptons (eorμ) with the same electric charge, or three leptons. The analysis uses 139 fb⁻¹ofppcollision data at$$ \sqrt{s} $$$\sqrt{s}$= 13 TeV collected with the ATLAS detector during Run 2 of the Large Hadron Collider. No significant excess over the Standard Model expectation is observed. Simplified and complete models with and withoutR-parity conservation are considered. In topologies with intermediate states including eitherWhorWZpairs, wino masses up to 525 GeV and 250 GeV are excluded, respectively, for a bino of vanishing mass. Higgsino masses smaller than 440 GeV are excluded in a naturalR-parity-violating model with bilinear terms. Upper limits on the production cross section of generic events beyond the Standard Model as low as 40 ab are obtained in signal regions optimised for these models and also for anR-parity-violating scenario with baryon-number-violating higgsino decays into top quarks and jets. The analysis significantly improves sensitivity to supersymmetric models and other processes beyond the Standard Model that may contribute to the considered final states.

 

Search for a new pseudoscalar a-boson decaying to muons in events with additional top quark pairs.