Spiking neural networks(SNNs) on neuromorphic hardware has become a promising machine learning technique as it mimics the behavior of the human's biological neural systems [1] in both spatial and temporal dynamics. Due to the sparse activities and event driven operation, SNNs is highly energy efficient when implemented in hardware. In recent studies, SNNs on neuromorphic hardware have made considerable progress in general machine learning tasks such as computer vision [2], object detection [3] and speech recognition [1].
In dynamically changing and non-stationary environments, distribution shift is an inevitable problem [4]. Such shift can be found in the operations of many autonomous edge devices that interact with physical world through sensors, where the input data distribution may alter slowly due to changes in the physical environment and sensor conditions, such as the sensor angle change or pixel damage due to aging. The shift of input distribution violates the condition of proper function of neural network models, which requires the testing and training data drawn from the same distribution, and will seriously degrade the model performance. Some existing works mitigate the performance degradation using supervised training [5] [6] [7], by augmenting the training set with sufficient amount of data that cover all possible variations of testing domains. This usually associates with a high cost and unpredictable results as the actual testing environment is hard to anticipate, and annotated data in realworld settings is hard to obtain. Gradual domain adaptation, on the other hand, aims at unsupervised/semi-supervised model adaptation and self-calibration. It takes the advantage of the fact that the domain shift is a slow process. By leveraging unlabeled intermediate domains where subtle changes take place, it incrementally adapts the target model. In particular, given a model trained on limited data from the source domain D 0 , and unlabeled data drawn from the intermediate domain's D i , (1 ≤ i ≤ N -1), where i indicates their sequential order of occurrence. Gradual domain adaptation aims to assist the generalization of the pre-trained model when tested on target domain D N . Although the target domain is significantly different from the source domain, the shift takes place gradually, i.e., ), measures the distance between two distributions and ϵ is a very small number.
While Domain shift and adaptation is a concern for both ANN and SNN, almost all of the existing works in this area targets at ANN [6] [8] [9]. Adaptation of SNNs has its own challenges and considerations. This work discusses unsupervised gradual domain adaption of SNN models. To make sure that the proposed framework supports truly online in-situ adaptation, we build it on top of Error-Mouldated Spike Timing Dependent plasticity [10](EMSTDP). EMSTDP is an SNN learning rule that incorporates the idea of difference target propagation [11]. It updates the local weight with the difference between "target spikes" and the "propagated spikes". The simple weight update rule enables EMSTDP to be implemented on low power neuromorphic hardware such as Loihi [12].
We present a self-supervised framework. Our framework consists of a feature extractor that is trained on the source domain D 0 , and two "swapping" classifiers with different architecture, one of them will serve as the "teacher" model to generate pseudo labels, and the other serves as the "student" model trained with unlabeled input data and their corresponding pseudo labels in every self-training episode. The classifiers will exchange their roles once finishing a selftraining episode. Although domain adaptation using teacherstudent models has been studied for ANNs in [13], the spiking domain inference and online learning impose extra challenges. This paper discusses techniques that are necessary to mitigate those challenges, such as "swapping" teacher and student models, new label calibration algorithm and novel neuron circuits for online normalization, etc. Our contributions are summarized as follows:
• We present a self-training framework based on EMSTDP learning for Spiking Neural Networks to realize gradual domain generalization.
• We have demonstrated that techniques such as label calibration, online neuron normalization are necessary to regularize the firing activities of neurons for better adaptation.
• We test our approach on 4 different datasets. The results show that the gradual domain adaptation can enhance the robustness of the SNN model and improve the classification accuracy by over 10% on average.
Different learning algorithms of SNNs have been explored. The majority of them are based on Backpropagation-Through-Time(BPTT) [14] [15], they adopt similar neural network topology as traditional artificial neural networks(ANNs), but use surrogate activation functions to replace the nondifferentiable spike function. Another class of SNN learning algorithms are based on spike-timing-dependent plasticity (STDP) [16] [17], which is a local learning rule. According to the STDP rule, the change of the synaptic weight is determined by the timing of pre-synaptic and post-synaptic spikes, both of which can be obtained in a single neuron locally.
We use Error-modulated STDP (EMSTDP) [10] as our learning rule. The neurons adopt integrate and fire model specified by the following equations:
where u l i (t) denotes the membrane potential of the ith neuron in layer l at time t, s l-1 j (t) is input at time t, w ji is the synaptic weight between the pre-synaptic neuron j in layer l -1, and the post-synaptic neuron i in layer l, b l i here denotes the bias. Equation 2 gives the Heaviside step activation functions. A spike will be generated once the membrane potential u l i (t) exceeds the threshold V th .
An SNN with EMSTDP learning consists of two networks, a feedforward network that performs the inference and learning, and a feedback network that propagate back errors (i.e., gradients). Feedback alignment is adopted, such that the two networks do not need to keep the same copy of weights. A set of random weight, B, is used in the feedback network. The error propagated in the feedback network is calculated as the following: e l i = j e l+1 j B i,j h ′ i (u l i ) where e l+1 j is the error of neuron j in layer l + 1 and h ′ i (u l i ) is the derivative of the output activation function of neuron i in layer l. The Heaviside step activation is not differentiable, gradient surrogation has to be adopted. Given that the relationship between the number of output spikes and neuron's accumulated sub-membrane potential can be approximated as a shifted ReLU function, h(u i ) = max( ui vth -1, 0), EMSTDP replaces h ′ i using the following equation:
The EMSTDP operates in two phases, free running phase and error correction phase. In the free running phase, the feedforward network makes prediction,h l j ,based on the received input. In the error correction phase, the prediction is compared with the target and the feedback network calculates errors for each layer. The error is then sent to the feedforward network to correct the prediction and generate the new prediction ĥl i , where ĥl
The preliminary of gradual domain adaptation is that one has access to the source domain D 0 := {X 0 , Y 0 } with input x 0 and its label y 0 , where x 0 ∈ X 0 , y 0 ∈ Y 0 . The knowledge learned from the source domain must be applied to the target domain D N , which is notably different from the source domain, i.e., D(D N , D 0 ) ≫ ϵ, where ϵ is a very small number. Due to the discrepancy between source and target domain, applying the model trained on source domain to target domain can cause severe accuracy degradation. The transition from the source domain to the target domain is gradual. A sequence of intermediate domains,
Both the intermediate domains and the target domain are unlabeled. Our goal is to adapt the knowledge learned in the source domain and apply it to the target domain during testing time by utilizing the unlabeled intermediate domains from the slow transition process. To address above issues, several ANN works investigate gradual domain adaptation [8] [18] [9]. The most relevant existing work in spiking domain is [7], which deploys transfer learning to SNN. However, [7] requires labeled training data from the target domain, while we consider more realistic case where both target and intermediate domains are unlabeled.
We propose a self-training framework for gradual domain adaptation with a focus on its unique characteristics designed In most ANN self-training frameworks, the same network f (x; θ) is used as both teacher and student models in every self-training iteration, and the student model is adapted directly from the teacher model. However, the same strategy gives poorl results in SNN based implementation. Due to the similarity between the adjacent intermediate domains, each pair of teacher and student models are very similar. Which means the learning process operates on small error signals and low gradients all the time. With spike domain learning, the error signals e and the learning gradients must be represented using spiking activities, hence are highly quantized. When the values of error signal and gradients are too low, the rounding error will be more significant than the signal itself and cause the model to update inaccurately. In contrast to the conventional approach, we utilize networks with slightly different architectures as teacher and student models, and swap them for different intermediate domains, so that we can get sufficiently large error and gradient to keep the learning going on.
Specifically, our self-training framework consists of a convolutional spiking neural network G ( •; Φ), which serves as a feature extractor and two 3-layer EMSTDP classifiers f a (•; θ i )
The motivation behind label filtering is to filter noisy labels based on the confidence score [9,26]. The confidence score is defined as the entropy of the classification output and is specified by (5), where h i denotes the spike count of the output neuron corresponding to output layer, and N is the size of output layer. A high enytropy means high uncertainty and low confidence. The label and its associated input will be discarded if the confidence score exceeds the threshold. We set the entropy threshold to 0.7 in this paper.
Instead of directly using the output of the teacher model as the target, we perform label sharpening to amplify error(gradient) for learning. Given the pseudo label t = arg max i h i , the target sequence of output neuron t is a spiking sequence with rate 1 and the target of all other neurons is expected to have no spikes. We further reduce the spiking threshold of error neurons in the feedback path to make them Algorithm 1 Self-training framework Batch normalization is a go-to tool for training traditional ANNs and it mitigates covariance shifts of the inputs. As stated in [8] the batch normalization also improves the performance of gradual domain adaptation. Several spiking-based batch normalization methods have been proposed recently, for example, [19] Time (BNTT) to every convolutional layer along the time axis with a given batch. More specifically, BNTT directly normalizes j w ij s t j in every time step within a batch. However, batch normalization is hardware expensive as it requires extra memories to store neuron's activities for the entire batch. Furthermore, it is not compatible with online learning , where batch size is set to 1. To solve this problem, we propose an online normalization technique that whitens the firing rates of a given layer for EMTDP.
The proposed online normalization has an auxiliary neuronbased normalization is inspired by NeuNorm network [20]. The dynamics of the auxiliary neuron is stated in Eq.6, where x t+1,n is the membrane potential of the auxiliary neuron x at time step t + 1 for layer n, α is the hyper coefficient of time constant, H is the number of neurons in layer n and s t+1,n j is the output spike at time t + 1 in layer n. In short, the membrane potential x t+1,n is a leaky integration of all the spiking activities of layer n. Eq.7 describes how the auxiliary neuron affects the neurons in the subsequent layer, where U n+1,t i is the membrane potential of the ith neuron in the (n + 1)th layer, g(•) is the Heaviside step function, which transforms auxiliary neuron's membrane potential to discrete spikes. C i is a trainable negative parameter. Intuitively, the auxiliary neuron can be regarded as a dynamic bias that regulates the spiking dynamics from the previous layer to the next layer. If the firing rate of the previous layer is too high, then the auxiliary neuron will assist to decrease the membrane potential and vice versa. Consequently, the auxiliary neuron normalizes the firing activity of the next layer by using the feature-wise statistics.
We tested the proposed framework on both neuromorphic datasets (neuromorphic MNIST) and static datasets (MNIST, Potraits, and Fashion MNIST). For each dataset, parametric non-linear transformation is applied to the data to emulate domain shifting. By controlling the parameters in transformation, we generate different intermediate and target domains.
MNIST [21] is a popular dataset for classification task, it consists of handwritten digits from zero to nine. Two variations of MNIST dataset are generated to represent different domain shifts. 2) Cropped MNIST: In cropped MNIST dataset, we randomly set the pixels of images to 0 to emulate inputs from an aging camera with dead pixels. The partition of the source, intermediate, and target domains is the same as the Rotated MNST. The intermediate domains contain images with the ratio of dead pixels increasing from 0% to 60%. The target domain consists of images that have 60% dead pixels.
Fashion MNIST [22] is a more complex dataset comprising of 28 × 28 grayscale images of 70, 000 fashion products from 10 categories. We further generate rotated version where the setting is similar to Rotated MNIST, 20000 samples are chosen as the source domain, 50000 samples as the intermediate domains, which are also split into 5 runs. The target domain consists of 10000 30-degree rotated samples.
The Neuromorphic-MNIST(N-MNIST) [23] converts static MNIST images to 3 saccades by using the DVS camera. The first saccade starts from the first 105 ms (0-105 ms), the second saccade in the next 105 ms (105-210 ms), and the third saccade in the subsequent 105ms. To process this eventbased data, we rotate the sampled frames up to 30 degrees. The domain settings of Rotated N-MNIST are the same as Rotated MNIST, and the event based data transformation is the same as Rotated MNIST.
The portraits dataset [24] consists of photos of high school students with different haircuts and facial features across decades. The hair cut and facial features of those people varies from time. For this dataset, we adopt settings from [8], where they build a binary classifier to classify genders of given images across ages. The source and target domains consist of 2000 images while the intermediate domains have 14000 images that are split into 5 runs.
The network comprises of 3 spiking convolutional layers [25] for feature extraction and two fully connected layers for
we compare the classifiers with gradual domain adaptation to models without domain adaptation. We also compare gradual domain adaptation with target domain adaptation where model directly retrains on the target domain. Finally, we also list the accuracy of a model that is trained on the target domain with labels which is referred as oracle. Obviously, the oracle sets the upper bound of the accuracy of the domain adaptation. Table II lists the accuracy of the gradual domain adaptation and three baseline models for the 5 datasets. We can see that our self-trained gradual domain adaptation model outperforms both the baseline models on all datasets. For Rotated MNIST dataset, it boosts accuracy for over 10%. Compared to the model without adaptation, the self-training with gradual domain adaptation improves the accuracy by more than 5% on every dataset. The result also indicates that our self-training approach can achieve competitive accuracy as the oracle model, which is trained supervisly on the target domain. The results in Table II demonstrate that the model is able to adapt to the shifted data distribution in the target domain.
We further evaluate the impact of different enhancement techniques on the self-trained gradual domain adaptation.
Table I shows the impact of using different teacher and student models. As we can see, compared to the baseline where the same SNN architecture is used for both teacher and student, our approach improves accuracy by 3.3% to 16.3% Table III compares the accuracy of self-training with and without online normalization. As we can see, using online normalization improves the accuracy by 1.47% on average over 5 datasets. Table IV compares