R ESIDENTIAL buildings consume a significant percentage of total energy consumption worldwide, 22% in the USA [1], 28% in the U.K. [2], 40% in China [3], and 44% in Malaysia [4]. Among those, heating, ventilation, and air conditioning (HVAC) systems consume around 50% of total residential buildings' energy consumption [5], [6]. To reduce such vast HVAC energy consumption, it is essential to have an accurate occupancy sensing system [7]. The occupancy information-based control can save about 11%-34 % of energy and does not sacrifice thermal comfort in rooms [8]. Therefore, a reliable occupancy sensing technique is being high demand. Significant efforts have been made to develop various occupancy sensing solutions to satisfy this requirement.
With the widespread use of machine learning (ML), more and more occupancy sensing platforms are incorporating ML techniques to detect occupancy status [9]. For example, a network of modulated LED light sources could detect occupancy, and estimate the occupant's positions using a convolutional neural network (CNN) to process active scene illumination [10]. CNN also works with radio frequency identification (RFID) tags to extract features. In addition, the transferable training makes RFID tags monitor occupants in different furnishing situations [11]. Other popular ML algorithms like support vector machine (SVM) [12], [13] Gaussian Naive Bayes [14], K -nearest nNeighbors [15], multilayer perception (MLP) [16], random forests [17], and particle filter-based ML [18] also are applied to thermal sensing platforms to predict occupancy and monitor occupants' behaviors. Some other thermal sensing platforms also fuse with ultrasonic sensors and radio frequency to detect the occupancy and occupant's behaviors data through image processing, computer vision methods [19], and SVM-based sensor fusion [20]. However, selecting a learning model for time-series inputs is challenging as it needs to identify informative features and optimal time lags for the time-series model. The long short-term memory (LSTM) model is good at processing time-series inputs because it could store and extract historical data. The LSTM model with various sensing data that could be adapted for occupancy detection and prediction in uncontrolled environments, such as optical messages in the hall [21], room's light, temperature, humidity, and CO 2 concentration [22], or the education building's ventilation rate [23]. The LSTM model significantly reduces the effects of uncertain environments in these cases. Although different sensors' data could make the LSTM model deal with stochastic situations, their importance is different for the LSTM model to predict occupancy. After many experiments, the CO 2 level is verified to be a crucial environmental feature by comparing multiple deep learning model occupancy results, including deep neural networks, LSTM models, and gated recurrent units (GRUs). In experiments, the sensor Wi-Fi connected number is another important feature to predict occupancy state [24]. However, detecting the CO 2 level faces the environmental restriction problem, and monitoring the Wi-Fi connected number may cause privacy invasion and high costs in daily life.
The SLEEPIR sensor compares with traditional occupancy detection sensors, their advantages, and limitations as shown in Table I [25].
Environmental sensors could sense various surrounding information, such as CO 2 density, lighting, and room temperature. Through sensor fusion algorithms, environmental sensors could estimate the room's occupancy state, but their results cannot immediately reflect the current occupancy state change due to the occupants' behaviors change and then slowly showing in environmental factors. The WiFi/Bluetooth monitors and the camera may cause privacy invasion when they monitor occupants' devices and motions.
Passive infrared (PIR) sensors are another popular sensing solution for occupancy detection due to their larger field of view (FOV), low cost, small size, and privacy reservation.
PIR sensors commonly produce binary output to indicate the detection of occupants in motion [26], [27]. PIR sensors with the hidden Markov model (HMM) and the CNN model only could extract occupants' seizures and movement patterns to determine the room's occupancy [28], [29]. Our group recently developed a synchronized low-energy electronically chopped PIR (SLEEPIR) sensor module that can detect stationary and moving occupants. The idea is to use liquid crystal (LC) shutters to periodically chop the PIR signals to enable stationary occupancy detection [30]. Compared to mechanical chopping mechanisms [31], [32], [33], the LC shutter can reduce the power consumption, weight, and signal noise by at least 80% [30]. Integrating with our temperature-sensitive adaptive algorithm, the SLEEPIR sensor node can detect uncontrolled stationary occupancy with an accuracy of over 92% [34]. Though promising, the system-level detection accuracy is restricted by node-level accuracy. Thus, in this article, we aim to further improve the node-level detection accuracy of the SLEEPIR sensor using two different inputs LSTM models. Based on our previous research, we used fixed threshold voltage to compare the stationary occupancy results through ML models [35] and the adaptive algorithm [34]. We also noted that environmental factors should be considered more since environmental disturbances can cause the threshold voltage to fluctuate and reduce the accuracy of the occupancy result. The LSTM model can store the past 50 min of the SLEEPIR sensor output data as memories, so the memory advantage of LSTM models improves detection accuracy according to environmental changes and the SLEEPIR sensor node's various output data. This is useful in filtering the current uncontrolled/controlled environmental disturbance and detecting the occupancy state.
In this article, we developed two LSTM models, the state switch detection (SSD) algorithm and the MLP ML classifier with the statistical feature inputs (Stat.ML) to understand their advantages and disadvantages in uncontrolled/controlled environments. The SSD algorithm only compares the SLEEPIR sensor node's analog voltage output with the defined threshold voltage value to determine the occupancy change state. The MLP is a forward artificial neural network that utilizes supervising learning for training. The two LSTM models, the baseline LSTM (Base.LSTM) and the statistical LSTM (Stat.LSTM), have the same architecture but with different inputs. The SLEEPIR sensor node's raw output and its statistical features are the inputs of the Base.LSTM and Stat.LSTM models, respectively.
To validate their detection accuracy, we consider two environmental disturbance scenarios: daily occupancy status without and with EDGE cases. The first scenario considers both female and male occupants and different skin radiations. The second scenario considers various room temperatures and occupancy postures, including lying down. Both scenarios are carried out in both lab and apartment rooms under uncontrolled and controlled environmental settings, with the purpose to simulate complex infrared environmental disturbances and occupancy patterns that often occur in daily life. We confirmed that both the Base.LSTM and the Stat.LSTM models report an accuracy exceeding 95%, higher than that of the other two models in most scenarios. In summary, this article has three major contributions: 1) we proposed two LSTM models that could use the SLEEPIR sensor node's raw output data to detect both stationary and moving occupancy; 2) we tested our proposed models in different scenarios to reflect real-world environmental settings, including daily occupancy scenarios and two EDGE cases, to test the accuracy of the sensor node with two LSTM models under various environmental disturbances; and 3) we validated that the two LSTM models outperform (>95% accuracy) two other algorithms we developed (SSD and MLP).
The rest of the article is organized as follows. Section II outlines the working principle of a SLEEPIR sensor node. Section III presents the experimental setup and dataset collection. Section IV describes the occupancy detection algorithms and ML models. Section V reports the result analysis, and finally, Section VI concludes the major findings and future works.
As presented in Fig. 1(a) and (b), a SLEEPIR sensor node includes two SLEEPIR sensor modules (module1 and mod-ule2), a digital PIR (D-PIR) sensor (Panasonic AMN34112, digital output), a temperature sensor (TEMP.) si7021, a microcontroller unit (MCU) EFR32BG, with control circuit, and two AA batteries. The SLEEPIR sensor module consists of an LC shutter and an analog PIR sensor equipped with the analog signal amplifier circuit (Panasonic AMN24112, analog output). Here we block two diagonal sensing elements of the analog PIR sensor to avoid canceling polarization from opposite directions. An overview of the SLEEPIR sensor module is shown in Fig. 1(c). The LC shutter is placed above the analog PIR sensor. To modulate the transmission ratio of the LC shutter, the MCU uses a pulsewidth modulation (PWM) signal, which has a periodicity of 8 s consisting of 4 s of the "ON" state and 4 s of the "OFF" state. Once the LC shutter switches on, it takes 4 s for the LC shutter to reach its maximum transmission rate and the analog voltage output of the SLEEPIR sensor node to stabilize [36]. The MCU data sampling frequency is 20 Hz, and it could wirelessly transfer data to the Bluetooth Hub. The hub connects to the Raspberry Pi for storing data and running models. During the operation status, each actuation period is 30 s, including 8 s for ON/OFF states and 22 s for standby for sending data to the Bluetooth hub. The Bluetooth hub updates data from the SLEEPIR sensor nodes every 30 s. The Raspberry Pi also updates the data to run models every 30 s and calculates the occupancy detection result within two seconds in real-time performance.
Each SLEEPIR sensor module receives infrared thermal energy and then converts it to the analog voltage output. A general relationship between the analog voltage (V out ) and the received infrared thermal energy (W ) in the time domain is represented in the following equation [30]:
Here, R f is the feedback resistor; ⌘ is the analog PIR sensing element's emissivity; p is the perpendicular component of the pyroelectric coefficient; A p represents the pyroelectric sensing element's area; ⌧ T and ⌧ E correspond to the thermal constant and the electrical constant separately. Due to manufacturer factors, the energy transmission ratios of the LC shutter are different in the OFF to ON (OFF-ON) state and the ON to OFF (ON-OFF) state. This produces a different output of the analog PIR sensor (W ON ) and (W OFF ) in different states.
Based on (1), detailed mathematical relationships between analog voltage and received infrared thermal energy could be expressed for OFF-ON and ON-OFF states, as shown in the following equations:
(2)
Here, ! r represents the rising frequency for the OFF-ON state; ! d corresponds to the decay frequency for the ON-OFF state. An example output of the SLEEPIR sensor module is shown in Fig. 2, where the PWM signal includes OFF-ON, ON, and ON-OFF states. In an actuation period, V peak,r is the minimum analog voltage of the OFF-ON state; V peak,d represents the maximum analog voltage of the ON-OFF state. The analog voltage variance between V peak,r and V peak,d reflects the analog voltage's peak-to-peak value (V pp ), which is a fundamental feature in algorithms and learning models.
We test our models in two scenarios: daily occupancy and EDGE case, as shown in Table II. Daily occupancy tests LAB1, LAB2, and APT are held under uncontrolled environments. Uncontrolled environments could include various room temperatures, sunlight, and humidity. For EDGE1 and EDGE2, we consider a larger range of room temperatures and different occupancy postures.
The daily occupancy tests are held in a lab room and an apartment room as shown in Fig. 3. The SLEEPIR sensor node is placed in the center of each room's ceiling. A surveillance camera is used to provide the ground truth (GT) in the lab room. The apartment's GT is manually recorded by the occupant. An example of the SLEEPIR sensor node's daily test raw output is shown in Fig. 4(a) and (b) in the lab and apartment room. Here the GT "1" is for the occupied and "0" is for the unoccupied state. The D-PIR sensor digital output provides another reference for GT.
The GT of the occupied states and room temperature of Daily occupancy tests are shown in Fig. 5(a)-(c) for lab and apartment room, respectively. In addition to GT, we use the D-PIR sensor and the SLEEPIR sensor node to further classify the occupied states are "Motion" and "Stationary," respectively. The room's temperatures are recorded to analyze their impact on occupancy status detection. in Fig. 6(a), and then detect a male occupant's occupancy in the lab room. The SLEEPIR sensor node needs to differentiate infrared radiation of human from that of nonhuman warm subjects in the FOV. Around 800-1200 min, the TEMP. sensor's readings indicate that the SLEEPIR sensor node detects the thermal airflow from heat gun as shown in Fig. 7(a).
At the same time, the male occupant entered the lab room, and the sensor node detects him regardless of warm environmental disturbances. The EDGE2 case aims to validate the detection accuracy of both female and male occupants when lying down as shown in Fig. 6(b). We notice that the human body radiation when sitting is different from when lying down. When sitting, only the radiation of the head and legs is strong enough to be detected. When lying down, the radiation of the whole body can be detected. Therefore, different postures may disrupt the SLEEPIR sensor node's occupancy detection.
Advanced ML classifiers can use multiple input features to determine the occupancy state. Table III lists 20 requested features extracted from the raw output of each SLEEPIR sensor module and updates them 8 s data within an actuation period (30 s). All the data are single-dimensional data.
Extraction of time-series features is an essential step for ML models. It frequently consumes a large of time and requires people to consider a wide range of domain knowledge variables and detailed coding. The time-series feature extraction library (TSFEL) computes various features from temporal, statistical, and spectral domains [37], [38]. The TSFEL uses features to facilitate fast data analysis and feature extraction for time-series computational evaluation in different scenarios. We select 20 characteristic features from the TSFEL, with
The recurrent neural network (RNN) model family includes three kinds: the classical RNN model, the GRU model, and the LSTM model [39]. The classical RNN model is a good fit for very short-term input. The GRU model could process the long-term time-series input data but has no output gate and fewer training parameters than the LSTM model. Compared to other models like the HMM's linear dependencies assumption and discrete data, and the CNN model's fixed input and output, the LSTM model could handle long-term dependencies data, non-linear transformation, and variable length sequence data. To better understand the environmental effect and obtain higher detection accuracy, the LSTM model is our first option, and we propose the Base.LSTM and the Stat.LSTM. Fig. 8 shows the processes of the Base.LSTM, Stat.ML and Stat.LSTM separately. The time-series inputs of the Base.LSTM only consists of peak-to-peak voltages, TEMP. sensor readings, and digital outputs from the D-PIR. The Stat.LSTM's time-series inputs need statistical feature sets and TEMP. sensor readings. Finally, LSTM models and Stat.ML will respond occupancy state as 0 or 1. The "Occupied" state is 1 and the "Unoccupied" state is 0.
The unique construction nature of the LSTM model enables feature extraction from any historical time point that is available so that the LSTM model has a long-term data memory. A general LSTM cell is represented in Fig. 9. Each cell includes an input gate i t , a forget gate f t , a cell gate g t , and an output gate o t . These gates' functions are given from the following equations:
A forget gate f t is to filter which previous features should be discarded in a previous cell state c t 1 . If f t is 1, the previous c t 1 could be remained. If f t is 0, the c t 1 will be filtered. The Base.LSTM and the Stat.LSTM includes two cell layers, as shown in Fig. 10. All feature inputs are normalized to the range [0, 1]. Every LSTM cell transfers cell state c i, j , hidden state h i, j , and output(i, j), the layer number is denoted as i and the cell number is represented as j. The initial bias and initial weight values are 0 in every layer's first LSTM cell. Each layer contains 100 cells to store 100 samples in the past 50 min. A fully connected (FC) layer finally decides the current occupancy state. The FC layer contains 128 neurons, and it is helpful to simplify the previous layer's 100 outputs to an occupancy state value, 0 or 1.
The training model includes the BCELoss, a loss function, and the Adama optimizer. The BCELoss essentially is a sigmoid function that limits the output range from 0 to 1. The Adam optimizer is a replacement optimizer for gradient descent, and it combines momentum and root mean square propagation. In the initialization of the Adam optimizer step, the learning rate is given 0.001. The training progress will stop in 50 epochs or when the loss value is less than 0.0001. The training model's output values finalize weight and bias parameters in two LSTM models.
Input A, B matrices, V th Output StateSwitch (True, False)
The SSD algorithm is developed by characterizing the occupancy state change between occupied and unoccupied states following the detection logic shown in Algorithm 1.
. The A and B matrices separately contain 3 OFF-ON switching states. The A matrix contains the last three switching states before the motion period and the B matrix contains the first three switching states after the motion period as shown in Fig. 8. Below is the A or B matrix form
As mentioned in Fig. 2, each OFF-ON switching state is composed of one OFF state's voltage as the minimum voltage as V min and one ON state's voltage as the maximum voltage V max . The voltage difference of the minimum voltage and the maximum voltage is represented as the V pp , so each matrix includes 3 V pp values.
In the initialization of the SSD algorithm, 1V pp,min and 1V pp,max represent average difference values of minimum voltages and maximum voltages from A and B matrices. The SSD algorithm needs the magnitude between the 1V pp1 and the 1V pp2 as a comparative parameter 1V pp . If 1V pp is larger than the fixed V th , the output StateSwitch is "True" and it indicates the room's occupancy state is changed.
Despite the SSD algorithm and the Base.LSTM model requires the raw output data from the SLEEPIR sensor node, while the Stat.ML and Stat.LSTM models need the statistical feature set of the raw output data, these algorithms and models can generate detection results within two seconds and have the same operational efficiency. Based on the presence of the PIR sensor's analog signal amplifying circuit, all models and algorithms are able to acquire a reliable and well-amplified analog voltage. This characteristic proves advantageous for accurately calculating the V pp value, which is a valuable metric for our methods.
The detection accuracy of all models under ten-day daily occupancy testing scenario are represented in Fig. 12. The first five-day data are used for training, and results indicate average detection accuracy of each model in the last five days. The D-PIR sensor only uses the digital output to determine motion periods as occupancy states. The detection accuracy of the SSD algorithm is around 85%-94%. The large fluctuation is due to the fact that the V pp value is easily affected by circuit noise and thus false determination of the occupancy state change can occur. The SSD algorithm only detects the occupancy state switch. If one of the middle occupancy state switches is wrongly determined, the future states will be false. Stat.ML's statistical features could effectively prevent such a false detection accumulation, but its result accuracy is still not high (around 80%-90%). Stat.ML doesn't need the D-PIR sensor's digital output, so it loses the motion information. Also, statistical features are similar between occupancy states without temperature differences and unoccupied states with temperature differences. These reasons lead to lower occupancy detection accuracy than the SSD algorithm. The detection accuracy of the Base.LSTM and Stat.LSTM models is higher (>95%) than that of the other models due to their memory advantage. Different kinds of input features do not cause obvious accuracy differences in the two LSTM models. Another daily occupancy testing result presented in Fig. 13, considers both female and male occupants wearing coats to keep uniform body radiation surface as much as possible. These results agree very well with these from Fig. 12. It demonstrates that the proposed Base.LSTM and Stat.LSTM model's accuracy is not affected by different genders' radiation.
Two EDGE cases test results indicate that both the Base.LSTM and the Stat.LSTM models demonstrate higher accuracy than the other two models, as shown in Fig. 14. In the EDGE1 case, the accuracy of the two LSTM models still are higher than 95%. It means that LSTM models could filter thermal airflow disturbances. The EDGE2 case test results also show that changing sitting posture to lying down does not affect the accuracy of the two LSTM models. LSTM models could recognize the occupant's thermal radiation and keep more than 95% accuracy in the EDGE2 case test.
To show a comprehensive comparison and superiority of LSTM in the RNN family and its two-cell layer model structure that owns higher accuracy, we compare their result's accuracy in overall daily occupancy tests and EDGE1 and EDGE2 case tests. The classical RNN and GRU models use statistical features as inputs and then detect occupancy results. The accuracy of each model is shown in Fig. 15(a), indicating the average accuracy of the Stat.LSTM model reaches 96.6% with a standard deviation of 0.017. The accuracy ranges from 93.6% to 99.7%. The accuracy comparison of among one to three cell layer structures of Stat.LSTM models from various testing scenarios are shown in Fig. 15(b). It clearly shows that the average accuracy of the two-cell layer structure is the highest (96.6%), with the smallest standard deviation (0.017). The two-cell layer structure also shows the highest accuracy, ranging from 93.6% to 99.7%. These statistical data strongly suggest that the Stat.LSTM model performs better in our designed occupancy tests than other RNN models, and the two-cell layer structure has the highest detection accuracy for the Stat.LSTM model.
In this article, the accuracy of the SLEEPIR sensor node with Base.LSTM and Stat.LSTM models are verified in both uncontrolled and controlled environments under two testing scenarios: 1) daily occupancy tests in uncontrolled environments in a lab room and an apartment room and 2) EDGE case tests in controlled environments. Daily occupancy tests intend to reflect the daily life complex environment as much as possible and then verify their detection accuracy under uncontrolled environmental disturbances. EDGE1 case aims to verify detection accuracy in different surrounding temperatures. EDGE2 case is designed to validate their detection accuracy considering different postures including sitting and lying down. The SSD algorithm detects occupancy state and status change by comparing the sensor analog voltage values V pp with the predefined threshold voltage values. However, a single analog voltage value is easily affected by circuit noise and thus causes false detections. We use selected 20 statistical features from each SLEEPIR sensor module's raw outputs to form a statistical feature set. It is used to train the MLP classifier in the Stat.ML. The Stat.ML has various input features and updates inputs every 8 s, so it avoids false detection accumulation. However, the Stat.ML accuracy is still not reaching 90% in all tests due to similar statistical features between occupied states with constant temperatures and unoccupied states with temperature differences. The Base.LSTM model inputs come from the SLEEPIR sensor node's raw outputs that include sensor modules, the D-PIR sensor, and the TEMP. sensor. The Stat.LSTM model gains input features through the statistical feature set. Based on the results of daily occupancy tests and EDGE case tests, the two LSTM models could reach more than 95% accuracy and they keep stable in both controlled and uncontrolled environments.
In the future, the Base.LSTM and the Stat.LSTM could be applied at the system-level sensing network. With the node-level accuracy improvement, the system-level accuracy can be better improved in both uncontrolled and controlled environments. Another aspect is the LC shutter fabrication improvement. If the LC shutter's transmission ratio could be enhanced, the v pp will be increased. It will improve the feature characteristics of the LSTM models. To enable our LSTM models to determine the occupancy state accurately, we require the voltage output from the PIR sensor. This output is obtained through the PIR sensor's analog signal-amplifying circuit. However, we believe that utilizing a more advanced amplifying circuit has the potential to improve the quality of the V pp feature, consequently enhancing the accuracy of our learning model. Therefore, we intend to explore other PIR sensors and the customized amplifying circuit.
Authorized licensed use limited to: Texas A M University. Downloaded on March 05,2024 at 03:13:55 UTC from IEEE Xplore. Restrictions apply.