With the proliferation of smart Internet of Things (IoT) devices and objects, new methods of interacting with the IoT will be pivotal for an intelligent physical world. IoT-enabled devices have found their way into mundane, everyday objects such as light switches, toaster ovens, and even faucets [17]. These traditionally simple objects now require embedded active circuitry and external controllers, such as smartphones, to operate and interact with, which overly complicates the basic functions they were intended for. The overabundance of IoT devices entering users' homes has also been shown to hinder the performance of wireless networks [37], creating a snowballing effect on technology complexity. Given these challenges, a novel, passive method of remote interaction with objects is necessary to restore simplicity and efficiency to the IoT landscape.
A variety of solutions have been proposed recently to interface and interact with IoT devices. Examples include acoustic methods [15,28,34], infrared technologies [6], and even more unconventional ones such as resistive graphs used in Holman's work [16]. These new Human-Computer-Interaction (HCI) modalities aspire to deliver reliable and high-capacity gesture inputs for remotely controlling IoT devices. However, they often require expensive and power-hungry electronics, and do not offer significant advantages over the classical necessitates control circuitry and an external power source, inherently limiting these systems' capability to provide an extensive list of modulation states. Past approaches have attempted to enhance the number of modulation states by incorporating numerous RFID tags, each linked to a specific input gesture. Regrettably, these methods often demand intricate calibration procedures and encounter scalability constraints stemming from the need for a substantial number of tags. In contrast, KeyStub uses a resonant stub model paired with a robust button design. This configuration can alter the back-scattered signal in a deterministic way, allowing for as many modulation states or input gestures as the reader can accurately measure and discriminate between. Another challenge emerges from detection errors prompted by environmental fluctuations, like multi-path interference and ambient movements. Earlier design approaches relied on the absolute amplitude and phase of the back-scattered signal to detect gesture inputs. Unfortunately, this method's sensitivity to environmental changes necessitates tag training or recalibration when used in a new location. To surmount this challenge, our KeyStub design employs dual RFID ICs for collecting relative measurements, effectively eliminating all environmental influences, barring those arising from the resonant stub key presses themselves.
Moreover, the incorporation of multiple antennas can adversely affect gesture detection caused by the distinct antenna patterns generated by each tag. These patterns direct signals in varying directions, causing irregularities in amplitude and phase features, especially in environments rich with multi-path interactions. To mitigate the disparities introduced by antenna pattern variations and the ramifications of multi-path environments, KeyStub combines the signals from both RFID ICs into a single antenna via a Wilkinson power combiner [38]. This technique removes the need for calibration, given that common mode noise, affecting both RFID ICs through the same channel, is efficiently filtered out in the process.
In summary, the main contributions of KeyStub include: (i) A new approach to encoding discrete keystrokes using electromagnetic stubs connected to passive RFID tags. KeyStub presents a novel method for encoding key button press information through passive electromagnetic stubs that act as an extension to RFID antennas. The stubs are built on a circuit model that relates the backscattered signal's desired amplitude and phase shift to the stub geometries. This "antenna as a sensor" design can be straightforwardly extended to other frequencies and attached to other back-scatter radios. It evades the need for complicated, black-box, feature-based gesture definitions.
(ii) A dual-IC design and differential detection scheme that enable multi-path resistance and environmentindependent sensing. The KeyStub keypad design can operate in harsh multi-path environments without needing calibration. Owing to the dual-IC design performing relative phase/amplitude measurements through a single common-mode antenna, KeyStub cancels out multi-path and environmental effects and is reliable in dynamic scenarios.
(iii) Implementation and experimental verification of the KeyStub keypad prototype. KeyStub has been implemented using standard PCB fabrication with low-cost PCB materials, allowing for mass production. Our experiments in homes, offices, and industrial settings attest to the applicability of KeyStub as a new passive, battery-free component in novel human-computer interactive use cases.
Human-Object Interaction and Sensing. Human-Object and Human-Computer interaction research has exploded due to the desire to simplify and improve the way we interact with objects as expressed in IBM's and Khurana's et al. work [19,25]. In [64], Zhang et al. use electric field tomography to detect touch gestures through the use of a conductive material and a voltage-controlled current source with eight electrodes. As a user touches the irregular surface, signals between the eight electrodes can be used to detect the location. The major drawback, however, is that this system is not passive and requires the conductive material to be placed on objects with specific resistivities. Otherwise, detection is too difficult. Furthermore, the detection hardware motherboard and signal processing requires the use of specialized Printed-Circuit-Boards (PCBs). In addition, in Zhang's et al. other work, [63], Electrical Impedance Tomography (EIT) is used to detect hand gestures through the use of a wristband. Electrodes are mounted to the wristband and then sampled by a motherboard for gesture detection. However, it also requires dedicated hardware. Furthermore, Arora et al. in [2] developed a battery-free wireless interface system for a variety of HCI inputs such as touch, swipe and speech using energy harvesters. The prototype system uses a frequency modulation back-scatter method to interact with different objects. Although each HCI input requires different hardware and circuit designs, the signal processing is the same. Compared to KeyStub however, MARS can only encode two touch points corresponding to separate oscillator frequencies. KeyStub is passive and does not require dedicated hardware aside from the RFID reader and the keyboard itself. The concepts presented here can be applied to other areas aside from RFID, such as WiFi or mmWave, where many back-scatter technologies already exist.
RFID Back-scatter Sensing. Due to the availability, versatility, and low cost of RFID ICs and tags, RFID technologies have witnessed explosive growth and are projected to reach almost $45 billion in market size [41]. RFID tags are commonly used for asset tracking, barcoding, and object labeling. They also have found other uses, including wireless sensing [48,58,60]. A large body of research has been dedicated to utilizing RFID's back-scatter method of communication to create a variety of passive, battery-less sensing technologies. More specifically, RFID has been used for gesture recognition and object interaction as discussed in Sun et al. and Yun et al.'s work [45,61]. Examples of gestures include swiping, rotating, and pressing. In Katsuragawa et al.'s work, Tip-Tap [24], use 6 RFID tags to create nine different two-finger gesture inputs. Gestures are created by the connection and disconnection of the tags. Finger touches placed upon different locations create various gestures. Tip-Tap requires a specially designed glove for housing the RFID tags and tag training which reduces the versatility of the device. Similarly, Bainbridge et al. [3] design a glove mounted with a reader and programmable RFIDs to detect input gestures. Due to the need to mount a reader onto the hand, however, the potential for usage in practical applications is limited. In PaperID [29], Li et al. use multiple tags to create binary light switches and interactive object controls. Yet the need for multiple tag arrays for each control setting limits the scalability of the PaperID design. RIO [39] takes advantage of the phase information provided in the physical layer of the RFID protocol to detect gesture inputs on off-the-shelf tags. To create eight control knobs, RIO has to use eight separate RFID tags. In addition, RIO requires training for each tag which must be performed whenever the tags are moved to new locations. Lastly, Wang et al. [51] use two customized RFID tags with dipole antennas separated /2 apart to detect 10 swiping gestures. Each predefined finger-swiping gesture induces an impedance mismatch between the two antennas, which can be detected by measuring the two RFID tags' relative received signal strengths (RSS). Whereas this design is resistant to device location changes once calibrated at that location, it is not resilient against multi-path because the relative RSS values can change significantly in the presence of strong reflectors. In Lin et al.'s work, DropMonitor [31], Lin uses two RFID tags that are attached to a drip chamber, such as the ones found in medical settings, for monitoring the drip rate. Dual tags are used to mitigate multi-path by using a reference tag. However, by using two separate antennas, the multi-path cannot be completely removed due to the undesired effects on the individual antenna patterns in heavy dynamic environments. In MicroFluID by Sun et al. [46], three RFID chips are able to encode six different states, which can be used to detect a variety of different sensing modalities, such as temperature or movement. The gesture encoding rate, however, is low (2 gestures/chip) and requires a conductive fluid to be applied to each chip for encoding the gesture state.
Aside from gesture detection, RFID tags are capable of object tracking, orientation tracking, and localization [49,52,57], allowing for interacting with objects in novel ways. For example, IDSense by Li et al. [30] use physical layer information such as RSS, phase, and read rate to detect certain human interaction events such as translation, rotation, or swiping of objects embedded with tags. The detection and resolution of these events require the calibration of many RFID tags, restricting the number of applications and their scalability. RF-IDraw [53] and WiSh [22] use RFID tags to create virtual touch screens in 3D space or smart surfaces on any object. RF-IDraw uses multiple readers and antennas to increase location accuracy, whereas WiSh uses many RFID tags to generate a map of the surface. In RF-Identity by Feng et al. [11], personnel identification can be achieved without large RFID bandwidth. However, the identification requires a large dataset for training and a significant amount of RFID tags.
Back-scatter Communications. Other technologies, such as WiFi [14], have been used to design back-scatter communication devices that can harvest and modulate the WiFi return signals. For example, in [21], objects are 3D printed and embedded with passive sensors that modulate the antenna connection and disconnection, allowing data to be sent back to the WiFi access point. These devices harness energy from the user or the environment through the use of specially designed 3D-printed objects. Back-scatter technologies usually draw from RF switch technology that can modulate the return signal in small form factor devices. The drawback to RF switches. However, as expressed in Iyer et al. [21] they require external power and a frequency source for each gesture input. This complicates the design and reduces the seamless integration into everyday devices. Passive, battery-less interaction technologies have even seen commercial success. For example, the Phillips Hue Tap [36,42] uses the kinetic energy extracted from the user to power a switch for smart home appliances. Unfortunately, these devices only provide a single gesture input. Combining multiple such devices may increase the number of inputs. However, the hardware complexity, form factor, and cost will likely grow in a linear manner.
KeyStub keypad consists of a single patch antenna, two RFID UHF ICs, surface printed circuitry, and a collection of microwave stubs corresponding to different keys, as in Fig. 1A.
RF Circuitry. KeyStub uses an inset-fed patch antenna which is fed by a Wilkinson power combiner [38]. The KeyStub patch design uses a ground plane. With this measure, the amplitude/phase responses of the stubs will not be disturbed when KeyStub is mounted on external objects, unlike traditional RFID designs. The second and third ports of the power combiner are connected by microstrip transmission lines, connecting the RFID ICs to the antenna. The Wilkinson is used to combine the signals of the two RFID ICs which are separated by 13.865 mm, providing sufficient space to install the Wilkinson power combiner resistor on the keypad and to allow for the sharing of a single antenna (refer to Sec. 5.2 for details). As the interrogating signals from the RFID reader are collected by the antenna, the signal power is split and fed into both ICs. Following the EPC Gen2 medium access protocol, each IC will back-scatter its signal and propagate out of the patch antenna without mutual interference.
Alongside each microstrip transmission line are a set of microwave stubs placed 500 away from the edge of the line. The separation distance was selected to ensure manufacturability by avoiding extremely small geometrical features. Additionally, the separation distance ensures proper stub behavior by not creating large gaps which could interfere with the stub design. The shorting of a stub to the transmission line induces a known predefined amplitude and phase shift, which can be detected and recognized by RFID readers.
Detection Methods. As a stub button is pressed, the variance of the channel will change wildly, which KeyStub uses as a feature to detect the key press event. To discriminate the different keys, KeyStub compares the amplitude and phase samples against an electromagnetic simulation model. Specifically, KeyStub employs a differential detection method, wherein the amplitude/phase of both RFID ICs are used to remove the effects of environmental multi-path and interference. The signal differences between the two ICs will naturally remove environmental noise and leave the effects of the microwave stubs intact. The following sections discuss the design details of each component of KeyStub.
A conventional RFID tag modulates the back-scattered signal by using an RF switch to create an impedance mismatch between the antenna and the IC. Similar to the coding mechanism of the RFID tag, KeyStub exploits the use of stubs to create different impedance mismatches. When a user presses a KeyStub button, the metal below the button connects the stub to the transmission line. This creates an attenuation in amplitude and a phase shift of the back-scattered signal, which can be sensed by the RFID reader and used to recognize the user input. In this section, we elaborate on KeyStub's stub design as a basic gesture input mechanism.
Stubs have traditionally been used to match a load impedance (complex or real) to the characteristic impedance of a transmission line [38]. Instead, KeyStub stubs are repurposed to create mismatches between one of the RFID ICs and the antenna. The impedance discontinuity and its impact on the signal can be mathematically described and are quantified by the reflection and transmission coefficient [38] as follows: Fig. 3. Electric field distribution when a stub is connected. Fig. 4. Electric field distribution when a stub is disconnected.
where is the impedance of the antenna, and is the characteristic impedance of the transmission line. By default, the circuit model of the RFID IC and the antenna match the characteristic impedance of the transmission line, i.e., = = , as depicted in Fig. 2. A discontinuity appears with a resulting reflection coefficient and transmission coefficient as follows:
where "||" means the parallel combination of the impedance:
The input impedances of resonant stubs are purely reactive, meaning they don't have a real component to their impedance. They can also appear capacitive or inductive, depending on their lengths and whether they are short or open-circuited. The input impedance of a short and open-circuited stub follows a classical model:
) Fig. 3 and Fig. 4 show the simulated electric field of the RFID tag with and without the stub connected to the transmission line. As shown in Fig. 3, when the stub is connected, power is robbed from the main transmission line and reflects the wave back due to the discontinuity at the end of the stub. When the stub is not connected, as in Fig. 4, all power is transferred to the antenna directly without being disturbed by the nearby stub.
For a certain key of KeyStub, we need to assign a specific attenuation value to it, thus making it distinguishable from other keys. To find the constraint on the achievable amplitude attenuation and phase shift, we denote the transmission coefficient in Eq. ( 2) as = -. In a lossless circuit, the total energy, including the transmitted 2) and ( 6), we have:
(7) This result means that the amplitude attenuation and the phase shift are dependent. Specifically, given an amplitude attenuation , there are two possible phase shifts and -. Nonetheless, we can still create a set of attenuation values as discriminating features.
We take amplitude attenuation as the independent variable to determine the stub impedance. Given an amplitude attenuation and the two corresponding phase shifts, the transmission coefficients can be determined. According to Eq. ( 2), the stub impedance can be found as:
Finally, following Eq. ( 4) and ( 5), the short or open-circuited stub geometries can be determined. Note that the short and open circuited stubs induce opposite phase shifts with the same amplitude attenuation, creating two additional distinguishable states for a given amplitude.
Microstrip transmission lines connect the RFID ICs to KeyStub. Each transmission line has a width of 2.1mm and a characteristic impedance of 80Ω and were optimized using the Ansys HFSS full-wave electromagnetic simulator. The characteristic input impedance, of a lossless transmission line terminated by a load is given by [9]
where is the wavenumber of the UHF signal and is the distance from the load. A mismatched load will create impedance variations along the length of the transmission line and reduce the delivered power. The load impedance must match the transmission line impedance for optimal power delivery. To mitigate this discontinuity when installing the RFID ICs, which have complex impedances [40], we design a matching network consisting of a short-circuited stub to counteract the reactive component of the RFID IC impedance. The matching stub network has two adjustable parameters, the length and position along the transmission line . Inserting the impedance of the RFID IC into Eq. ( 9) as , the stub parameters can be calculated. The matching stub introduces a reactive component at location on the transmission line, which counteracts the reactive component of the RFID IC. We find that using Eq. ( 9) as a starting point and Ansys HFSS to optimize further, the position of the short-circuited stub is 34.81mm, and the length is 3.4mm as shown in Fig. 5 along with the via pad dimensions. Plated-through-hole vias of 0.863mm were used to short the stubs to the ground plane on the bottom layer. KeyStub creates multiple stubs on the RFID tag as different keys. Our prototype implementation has 6 stubs, three open-circuited and three short-circuited. The stubs are placed near the transmission line of the RFID IC. Following the design method in Sec. 4.2, the final stub design parameters are listed in Tab. 1. The stubs were designed using half the listed amplitude values since RFID signals propagate in a round-trip fashion. All stubs have a width of 2.1 cm. Amplitude attenuation of 2, 5, and 8 dB are selected to provide enough amplitude isolation for accurate detection. The 8 dB stubs were added to verify the potential limit of KeyStub since the attenuation of these keys is high. The various stubs placed along the transmission line are independent of the RFID IC mounted onto the keypad. This holds true as long as the RFID IC is properly matched to the transmission line. Therefore, if an RFID IC with a different chip impedance is used, a different matching circuit design is required. Furthermore, the impedance is independent of the tag location since the stub and IC impedance do not change with location.
The simulated phase shifts may differ from the theory due to the presence of other electromagnetic phenomena that the ideal circuit model does not account for. For example, microstrip spurious radiation can cause unwanted phase shifts in the back-scattered signal. Furthermore, coupling between adjacent stubs will also shift the performance from the expected result. The inter-stub coupling will de-tune the desired impedance of the stub, thus distorting the back-scattered signal. Lastly, the discrepancy between the theoretical stub lengths and the simulated results comes from the contributions of the aforementioned electromagnetic phenomena and RFID IC coupling, which required stubs with smaller impedance than predicted to achieve the same relative amplitude shift. As stubs are selected and pressed, coupling between the two RFID ICs can increase the gain seen from one IC while the second IC gain is decreased, thus requiring an even greater attenuation to achieve the same desired relative amplitude difference.
Due to inter-stub interference, there is a minimum distance limit to which stubs can be placed next to each other. Coupling between stubs increases as the inter-stub distance decreases, potentially causing wildly different stub impedances than expected. Therefore, care must be taken to ensure correct amplitude values are realized through full-wave simulation. In addition, the inter-stub distance limit also sets an overall minimum size for the keypad design. To accommodate more keys, longer transmission lines are necessary in order to maintain the same minimum inter-stub distance. Miniaturization techniques can be used to reduce the sizes of the RF components, such as the antenna, power combiner, stub lengths, widths, and thus the inter-stub distance. Using substrate materials with larger dielectric constants can allow for miniaturization. However, this will require the human-keypad interface to shrink, which may hamper its usability.
Our current KeyStub prototype realizes a simple two-row keypad layout. Organizing the stubs into more complicated geometries can be quite difficult. For example, multiple rows of buttons can be created to mimic a number pad by interlacing multiple transmission lines, each with its own set of microwave stubs. However, routing the stubs and transmission lines complicates the design by necessitating multiple dielectric layers to route stubs around each other. In addition, higher RF losses could be incurred by requiring multiple power combiners to select the various branches of the interlaced stub buttons.
Lastly, as previously discussed, the impedance mismatch caused by the resonant stub causes attenuation of the back-scattered signal of the corresponding RFID IC, reducing its communication distance. Since KeyStub relies on the successful decoding of packets from two RFID tags, there is a trade-off between the communication distance of the keypad and the number of keys (in terms of maximum amplitude attenuation). As shown in Tab. 1, the maximum amplitude attenuation introduced by a resonant stub is 8 dB. At shorter reader-to-KeyStub distances, more keys can be deployed with an amplitude attenuation stronger than 8 dB while still maintaining a large signal-to-noise ratio for button detection and decoding.
KeyStub maps the keys with stubs and detects the user's pressing actions using the mismatch loss of the backscattered signal. While feasible in theory, the stub design can be fragile due to two main practical challenges. (a) Baseline (b) Body Interference (c) Metal Sheet Interference Fig. 7. Comparison of single and multiple antenna designs in different environments.
First, the RFID reader may not read the exact theoretical loss induced by the stubs due to wireless channel dynamics, such as multi-path and temporal variation. Second, due to human factors, including the wetness or thickness of a finger, the key press may dramatically change the impedance of the stub and induce a different mismatch loss from the designed value. The KeyStub design introduces robust key detection mechanisms to combat the above interference. KeyStub performs relative measurements between two RFID ICs to subtract the common-mode noise caused by multi-path and temporal variations within the channel. In addition, it avoids the varying electromagnetic properties of human users by using a mechanical structure that selects keys via metallic discs. This section introduces these mechanisms in detail.
The RFID back-scattered signal is impacted not only by the tag but also by the channel. Due to the multipath effect and temporal variation, the RFID reader cannot recover the theoretical mismatch loss induced by the stubs on the tag. Past works [39,51] use an additional tag as the reference. However, since different RFID tags are spaced apart, the channel still impacts the relative features, such as the amplitude ratio and phase difference.
To demonstrate the impact of the channel dynamics, we fabricate a dual-antenna KeyStub with two identical RFID tags, each with its own antenna, IC, and stubs, as shown in Fig. 6. This use of a dual-antenna design is consistent with the multi-tag design in the current state-of-the-art HCI solutions [2,28,39,51], and serves as a baseline comparison. We measure the relative amplitude and phase of the two ICs under the unpressed state along various reader antenna viewing angles from -45 to 45 degrees. Ideally, the amplitude and phase would return 0 dB and 0 deg, respectively, since the transmission lines are unaffected by disconnected stubs. However, as shown in Fig. 7a, the dual-antenna design can have relative amplitude and phase shifts of up to 3.7 dB and 145 deg, respectively!
In real-world scenarios, environmental objects can create multi-path and further distort the detection of input keystrokes. We verify this effect by asking a human participant to stand close to the keypad under test without pressing any keys. The same baseline relative amplitude and phase are measured along the reader's viewing angles. Fig. 7b shows that the dual-antenna approach suffers from heavy amplitude and phase distortion. The average amplitude and phase for the dual antenna designs are 1.2 dB and 43 deg, respectively, with maximum values of 4.8 dB and 145 deg. Lastly, we place a metal sheet 0.5 m from the keypad to create a severe multi-path environment. Fig. 7c shows a prominent amplitude skew versus viewing angle and a large overall phase shift. The average shift remains at a high level of 2.4dB and 60deg, respectively.
To overcome the unpredictable channel effects, we propose a single-antenna design, i.e., the two ICs share the same patch antenna, as shown in Fig. 1. Unlike the dual-antenna baseline, our design uses a power combiner to connect both ICs to a single antenna. By using a single antenna, both ICs experience the same channel. Thus, the relative measurements between the two ICs cancel out the channel diversities and still retain the internal variations caused by the key stubs, which can be used as consistent keystroke indicators.
Common-Mode Antenna Design. RFID tags generally come with a balanced antenna, such as a dipole, and an integrated RFID IC at the center. The dipole antenna dimensions and geometry are adjusted to match the RFID IC by tuning out the reactive component of the impedance [40]. KeyStub adopts an unbalanced antenna system, specifically a microstrip patch antenna. An unbalanced antenna topology requires a ground reference, such as the ground plane of the patch antenna, consisting of a large metal rectangle. Balanced antennas, in contrast, do not have a ground reference, but a "virtual" ground exists at the center feeding location of a dipole. An unbalanced antenna requires a matching network to optimally deliver power to the RFID IC, which has a complex impedance. Such an antenna is beneficial in our application because the ground layer allows greater mounting flexibility, and better isolation against adjacent objects [27,33]. In contrast, dipole antennas placed on other objects (especially metal) will capacitively couple to them, changing the antenna characteristics such as the radiation pattern and input impedance.
KeyStub uses a traditional inset-fed patch antenna tuned to a center frequency of 915 MHz with an input impedance of 80Ω designed on an FR4 substrate (350 mm by 240 mm, = 4.40, tan = 0.02) with a thickness of 3.175mm. The 80Ω patch antenna impedance is chosen to match the resistance of the RFID IC (80-i420Ω) at 915MHz [32] for maximal power transfer. The PCB geometry is shown in Fig. 8. The ground plane and substrate dimensions were chosen to avoid distorting the antenna pattern due to truncating the ground plane prematurely.
The inset feed and patch dimensions were optimized using Ansys HFSS to achieve a return loss (i.e., 11 ) of -10dB or lower within the frequency band of 900-930MHz. From the optimization, the patch dimensions resulted in a length of 76.295mm and a width of 99.7mm. The inset feed dimensions are 12.2mm in depth and 4.056mm in width to achieve good impedance matching to the 80Ω microstrip line. The operating frequency of KeyStub can be scaled up/down following the same HFSS workflow. Using a single antenna with two ICs removes environmental interferences, such as the multi-path effect. In addition, the antenna patterns created by the different stub keys are invariant to location and environmental factors, which avoids detection ambiguity between different keys. Furthermore, by using a single patch antenna, KeyStub can be mounted anywhere within an environment, such as ceilings or orthogonal walls, due to the quasi-omnidirectional pattern that the inset-fed-patch antenna produces.
To use a single antenna with two RFID ICs, power must be combined and delivered to the antenna through both transmission lines. We customize the classical Wilkinson power combiner/divider to accomplish this task. The Wilkinson structure also provides isolation between the two RFID ICs due to the inherent nature of the combiner. The transmission line is 0.91mm in width and a quarter of the guided wavelength in length, i.e., 53.655mm. The transmission line width of 0.91mm provides an impedance of √ 2 = 113Ω. In addition, a resistor of 160Ωs is integrated into the Wilkinson device. The geometrical characteristics of the Wilkinson and resistor pads were optimized in HFSS to achieve the lowest possible insertion loss at the UHF RFID frequency of 915MHz.
We test the single-antenna design under the same setting as the aforementioned dual-antenna baseline. As shown in Fig. 7, in the baseline and both interference cases, the single-antenna design behaves similarly to the ideal case, with an average amplitude of 0.3 dB and a phase of 10 only. By using a single antenna, relative measurements can be conducted in heavy multi-path environments and do not require calibration. In addition, combining both RFID ICs into a single antenna does not interfere with normal RFID operations. In effect, the RFID reader simply follows the standard EPC gen2 protocol to query the two ICs sequentially and obtain the corresponding RSSI and phase values [59].
A design that simply uses fingers to induce measurable shifts in back-scattered signals is advantageous in terms of simplicity and cost, such as those in [39,51,65]. However, fingers do introduce a few limitations. The uncontrollable electromagnetic parameters of the finger, as expressed in [23], such as permittivity and conductivity, vary from person to person. It limits not only the versatility of KeyStub for different users but also the number of keys in order to accommodate the range of finger permittivity. In [1,8,35], Oyeka et al.
show that the permittivity at UHF frequencies can vary between 20 to as high as 50 whether the participant is a woman or a highly muscular man. In addition, they show that conductivity can also vary between 0.1 and 0.8 S/m within the same test group. The variability in the electromagnetic properties of the finger can greatly distort the back-scattered signals in unintended ways.
To demonstrate the impact of finger diversity, we ask a male and a female user to press the gap between different stubs and the main transmission line. A finger press connects a stub to the main transmission line and induces a mismatch loss, which can be detected by the RFID reader. Fig. 9 maps the relative amplitude and phase onto a constellation diagram. The female user shows some consistency to the expected results, whereas the male user shows no obvious similarity. The design does not leave much room for button press detection since the Dipole Antennas RFID IC 1 RFID IC 2 Transmission Lines 16 cm in Length Fig. 10. Dual Dipole Keypad Design in [51]. Fig. 11. Limitations of [51] in multi-path environment.
phase/magnitude features of some button presses are close together.
To further explain the limitations of using a bare finger, we evaluate the performance of the keypad design in [51] in a multi-path environment. The finger permittivity is swept between 20 and 50 to address differing conditions such as those discussed in [35]. A metal sheet similar to the setup discussed in Sec. 5 is used to create reproducible multi-path effects. The design uses two dipole antennas connected by two transmission lines which are mounted onto a cardboard sheet as shown in Fig. 10. Each dipole antenna houses an RFID IC. Gestures are finger moves along the two transmission lines, similar to unlocking a cellphone pin code. Each gesture is defined by the starting and ending locations of the swipe. The signature of the gesture is the relative measurement between the two ICs. This keypad design is predicated on the assumption that the two dipole antennas remain constant in all environments and with all finger permittivity. The experimental results of varying finger permittivity in a multi-path environment are shown in Fig. 11. The relative amplitude of the baseline response shows a consistent pattern that can be used to determine the starting and ending position of a swipe. With varying finger permittivity, the amplitudes deviate by up to 2dB, greatly obfuscating the starting location of the finger. Furthermore, when multi-path is present, up to 35 dB variations can be seen! Using two antennas creates an issue in multi-path environments, even when using relative measurements. To mitigate this, calibration is required at each new location.
It might be possible to mitigate the impact of finger variations by using gloves, as shown in [26]. However, it limits the ubiquity and usability of the design. Therefore, we adopt push buttons as the user interface, as they are inexpensive, natural to use, and common in everyday digital objects such as keypads and wireless remotes. Fig. 12 demonstrates the working principle of the 3D printed push buttons of KeyStub. A Stratasys 3D printer [44] is used to fabricate the buttons. ABS-M30 [43] printing material is selected to print the push buttons due to its strong mechanical and lightweight properties. Soft, small springs are mounted underneath each push button to achieve keypad-like quality and provide the necessary disconnection of the short when the button is not in use. Underneath each push button is a 5mm diameter copper disc used to short the selected stub button to the transmission line. We mount the copper disc using liquid adhesive.
The KeyStub key press states are determined by calculating the ratio of the complex response of the two RFID ICs. Although the responses of both ICs are measured alternately, the sampling interval is sufficiently small due to the high reading rate of commercial RFID readers. For example, the Impinj R420 [18] has a read rate of approximately 50 samples/second or 20 milliseconds per sample. Alternatively, the coherence time of the 915 MHz channel at normal mobility of 1m/s is around 58 milliseconds, much longer than the sampling period. Therefore, the channel remains stable for the two ICs, and we can safely assume the two ICs' channels are sampled simultaneously.
The relative measurement of RFID ICs allows for multi-path resistance and removes the requirement of calibration because external events such as blockage, change of location, or change of objects in the environment are canceled by the subtraction of noise that both ICs sense. Since both ICs essentially exist in the same environment, this allows for the removal of common mode noise, similar to that used in digital circuits such as operational amplifiers. A button press input in KeyStub is calculated and defined as follows:
In Tab. 1, the key index is read from left to right when viewing the keypad shown in Fig. 1A. A few key button definitions are shown in Fig. 13, where denotes the top stubs connected to the top chip (Chip 1), and denotes the bottom stubs connected to Chip 2 located on the bottom. As discussed in Sec. 4, the connected stubs modulate the back-scattered signal amplitude/phase in a deterministic manner. When a stub, such as "T1", is pressed, the stub attenuates, and phase shifts the UHF signal coming from Chip 1 while leaving Chip 2 relatively undisturbed. The power combiner/splitter design provides isolation and avoids the notorious coupling effects as shown in [54] between the two ICs. In addition, the combiner ensures that the back-scattered signal only radiates through the antenna rather than propagating to the other IC and getting reflected again. The difference in amplitude and phase between the signals reflected by the two ICs respectively are used to discriminate between the button press inputs.
Examples of the received amplitude and phase of both chips for four button presses are shown in Fig. 13. The top two figures in each sub-figure show the absolute amplitude and phase of the measured back-scattered signals while the key is repeatedly pressed and released. From the figures, it can be seen that the reader has a sufficiently high amplitude/phase resolution and sampling rate to discriminate between button press responses.
The bottom two figures show the relative measurement for amplitude and phase (i.e., and ) between IC2 and IC1. In the bottom left figure of Fig. 13a, oscillates between 0 and -2dB as the T1 open-circuited stub is connected and disconnected from Chip 1. In the bottom right figure of the same sub-figure, the relative phase, , also oscillates between -10 • and -30 • . As another example, in Fig. 13d shows much larger amplitude shifts as the short-circuited stub connected to Chip 2 is connected and disconnected, changing by roughly 10dB. of Fig. 13d also varies by approximately 50 • while the stub is connected.
Similar to Fig. 9, a constellation diagram of different keys is shown in Fig. 16. To push KeyStub's detection capacity to the extreme, we fabricate a keypad with 12 keys. The black markers represent the simulated responses of and for all 12 keys. The vertical and horizontal bars show the threshold level regions used for detecting the presence or absence of a button press input. Using both the relative amplitude and phase as detection features allows KeyStub to use the available constellation space more efficiently than simply using amplitude alone as was done in [51].
Since the open and short-circuited stubs vary the phase in a conjugated manner, the constellation is split into two main phase sections, 0-180 and 180-360 degrees. In addition, by using two RFID ICs, the relative amplitudes take positive or negative values, which create two main amplitude sections within the constellation diagram. The baseline input, or the absence of a stub, takes the location (0,0) in the constellation diagram and does not vary significantly, as shown in Fig. 7a.
Commercial UHF RFID readers randomly hop among 50 subcarriers within the 902-928 MHz band following the FCC regulation [5]. To account for the change of subcarrier wavelength, the phase offsets need to be compensated for consistent phase readings. Fortunately, the phase offsets induced by frequency hopping follow a closed-form model. Specifically, the phase of the -th subcarrier is:
where is the frequency of the -th subcarrier, is the distance between the tag and the reader, and is the initial phase. The phase offset is linear with respect to subcarrier frequency. Thus, to compensate for the phase, KeyStub collects phase readings when the keypad is idle, and estimates the slope of the linear phase offsets as ˆ = 2 . Then, it compensates the phase reading of the -th subcarrier as ˆ = -ˆ .
To detect key pressing events, KeyStub tracks the mean and variance [56] of the phase difference of the two RFID ICs as disucssed in [50].When no key is pressed, the phase difference of the two ICs remains around 0 • with very small variance, thanks to the single antenna design and power combiner. KeyStub detects key pressing if either of the following conditions is met: 1) the variance of the phase difference exceeds 25 2 0 , where 2 0 is the variance measured in the static channels; 2) the amplitude difference exceeds 1 dB or an amplitude variance of 2 0 . As was shown in Sec. 5, the keypad is resistant to environmental objects when not in use and will maintain a consistently low variance. The variance is sufficiently low so as not to trigger a key-pressing event accidentally.
Once a pressing action is detected, KeyStub collects the amplitude and phase readings of the two ICs and uses the differential amplitude and phase as features to compare on the constellation map, thereby recognizing a key. Fig. 14 shows the variance output of the relative phase measurement, , when T1 is pressed and released. The captured relative samples from the two ICs are processed by applying a moving average window, consisting of 8 samples, used to calculate the signal variance. When a button is not pressed, the variance remains low. The variance drastically changes, however, as the back-scattered phase is shifted with the connection of the T1 stub. The variance threshold is plotted for comparison. Once the variance threshold condition is met a counter is used to wait before sampling occurs in order to avoid the noisy sequence caused by button-press debouncing. A 100 sample wait counter is used before collecting samples for button press identification. After the waiting period is complete, samples and are collected and mapped to the constellation diagram for key recognition. The measured amplitude is mapped directly onto the constellation diagram, whereas the phase is first compared to the simulation using the nearest neighbor algorithm and then mapped to the constellation diagram to determine the selected button. This step is necessary because the Impinj RFID reader phase measurement returns either the true phase value or the true phase value with an additional [18]. The additional jump occurs randomly and is intrinsic to the Impinj hardware, and cannot be removed easily. Therefore, the phase with and without the jump is compared to the simulation model, and the sample with the lowest error is used as the true value.
We now systematically evaluate the effectiveness of the KeyStub design. Similar to state-of-the-art RFID sensing systems [39,51], we use the Impinj R420 RFID reader to sample the phase/amplitude of signals back-scattered by the RFID ICs. The samples are collected by a PC host [20] for processing as a human participant presses the KeyStub buttons under different experimental settings.
The experiments are conducted in a multi-path-rich environment where a human participant is located directly next to the keypad during each measurement setup. As discovered in Sec. 5.1, the human body orientation does not impact the relative amplitude and phase of the back-scattered signals therefore, standing positions were not restricted or choreographed throughout any of the measurements. The KeyStub user was able to stand on either side of the prototype and press keys using either hand or finger. Fig. 15 depicts three locations where the KeyStub keypad was evaluated i.e., a typical home office environment, a workplace laboratory, and an industrial Patch Antenna Wilkinson 3D Printed Buttons Bottom Stubs Top Stubs IC2 IC1 KeyStub KeyPad Experimental Environment 3.66m 4.00m Desk Case Closet <1m Home Office Layout Turn on the Motor! Turn on the AC! Laboratory Setting Industrial Setting Reader Reader 2m Reader 1m Fig. 15. Evaluation scenarios using the Reader and KeyStub.
center. Common objects were left in their respective environments. The minimum thresholds, timings, and button mappings detailed in Sec.7 are used for button recognition and detection in all environments. The Impinj R420 reader data logging and data processing algorithms are written in C# and MATLAB, respectively.
By default, the reader stays around 1m away from the keypad. The phase/amplitude values are sampled when the keys are pressed and released repeatedly. The baseline results of all 12 buttons measured within the home office environment are shown in Fig. 16.
The average amplitude and phase of all presses for each button are plotted on top of the simulated constellation locations for comparison. Due to some of the limitations discussed in Sec. 4.4 and due to the high attenuation of some of the keys, some gesture inputs do not function as expected and result in detection errors.
Fig. 17 shows the resulting confusion matrix [12] using all 12 keys. Key T2 and T3 cannot be reliably detected and recognized due to their overlapping responses. Similarly, B2 and B3 have overlapping responses. In addition, T6, B3, and B6 are hardly detected due to the same reasons as noted above. Based on these observations, we remove T3, T6, B3, and B6 and update the threshold definitions in the constellation diagram as shown in Fig. 18. The confusion matrix using the updated 8-key constellation diagram is shown in Fig. 19. From these observations, in order to increase the number of press button gestures, higher mechanical precision, and more distance between ICs and stubs are required. Using alternatives to the liquid adhesive to mount the copper discs could improve connection stability and reliability. Theoretically, using the stub button model, more than twelve buttons can be accommodated. However, a more delicate mechanical structure needs to be designed to achieve higher disc placement accuracy, stability, etc.
Scattering and reflections due to nearby line-of-sight (LOS) objects can create heavy multi-path effects, thus affecting the back-scattered signals' amplitude and phase. The secondary signal sources caused by multi-path can skew the detection of the keystrokes. In this experiment within the home office, we measure the detection accuracy of KeyStub without calibration in a multi-path-rich environment. A metal sheet (same as in Sec. 5) is located 0.5m away from the keypad, facing perpendicularly to the line of sight between the reader and the keypad. The metal reflector acts as a strong secondary signal source. Each of the eight keys is pressed repeatedly a minimum of 20 times. The experimental results are shown in Fig. 20. The detection accuracy drops to 96% compared to the baseline experiment. Despite this result, KeyStub still demonstrates high resilience in the presence of the close-by metal sheet, which creates stronger multi-paths than in everyday environments. Aside from a single strong reflector, we conduct measurements in an industrial center where RFID systems are more commonly used, as shown in Fig. 15. In this environment, many strong reflectors exist as apposed to a single strong reflector previously. The industrial environment includes workers operating large metallic equipment and heavy instruments scattered throughout the range. The results shown in Fig. 20 for "Clutter" show that KeyStub can operate in strong multi-path environments with an 97% accuracy. Even though many highly reflective objects exist in the environment, their multi-path contributions are removed similarly to the single reflector experiment due to the differential decoding methods thanks to the single antenna and dual RFIC design.
The foregoing experiments have demonstrated KeyStub's multi-path resistance to static, reflective objects. However, certain ambient environments may be more dynamic due to frequent human movement and relocating objects. The dynamic nature of the RF environment can affect the amplitude and phase in different ways compared to the static case.
In this experiment, we intentionally induce dynamic multi-paths by asking two participants to move around the keypad located in the home office as the keys are pressed. The experiments were conducted with one person or two people moving to introduce an increasing level of channel dynamics. The participants maintain a close distance of 1m or less to the keypad and reader antenna and do not cross the line of sight between the two antennas. The walking participants maintained a normal walking speed throughout the measurement (<1.35m/s). The experimental results without keypad calibration are shown in Fig. 21. We observe that the minimum average detection accuracy of 95% is maintained for both experiments. To increase the dynamic interference further, measurements were conducted in a laboratory/office setting (shown in Fig. 15) where up to 12 participants moved randomly about the space at different intervals as buttons were repeatedly pressed. The results of these experiments are shown in Fig. 21 with an accuracy of 94%. Although the minimum accuracy was reduced to 81% due to a single mechanically failing button, this still demonstrates that dynamic environments such as those created by people moving do not impact the reliability of the KeyStub system. Thanks to the single antenna design with dual RFID IC differential measurements, highly dynamic environments have minimal effect to the accuracy of the key detection. Since each RFID IC communicates through the same channel, multi-path effects, whether small or large, are removed through differential decoding.
To measure the link loss and its impact on KeyStub's detection performance, we vary the reader antenna distance from 1.0m to 2.5m in 0.5m steps. By increasing the reader-keypad distance, the back-scattered signal becomes weaker, and the input SNR is reduced. The results of this experiment conducted in the home office are shown in Fig. 22, where a minimum average detection accuracy of 96% is maintained for all reader antenna distances, which is larger than the current state-of-the-art design [51]. Although the accuracy is slightly higher at 2.5m than at 2m, the minimum accuracy is much lower. The discrepancy arises from stability issues of the 3D printed buttons as discussed in Sec. 8.1. With the reader noise floor of -83dBm, a maximum of 5m can be achieved theoretically with high detection accuracy. Measuring these distances within in the home office environment were not possible however, limiting the maximum experimental range to 2.5m. These results confirm that KeyStub can operate at adequate distances from the reader and hence be useful in ordinary, everyday environments. The range can be extended by improving the reader antenna effective-isotropic-radiated-power (EIRP), e.g., by increasing the reader's output power or antenna gain. In addition, the range can be improved by reducing KeyStub losses, such as removing the power combiner, using lower-loss dielectric substrates, or increasing the antenna gain.
Increasing the detection capacity. Increasing the detection capacity, i.e., the number of distinguishable keys, requires accurately measuring the amplitude and phase of the back-scattered symbol with high resolution. The Impinj R420 RFID reader used in our experiments limits the resolution at which we can discriminate between symbol amplitudes [18]. Currently, the resolution of the reader is 0.5dB, limiting the amplitude separation between the symbols. Furthermore, separating the amplitude and phase shift dependency of the symbol states will open the door for much higher encoding capacities, since the resolution of the reader is not as limited in phase. By using passive, resistive attenuators and lumped element phase shifters, this dependency could be removed and allow for more freedom in encoding symbols.
The stability of the symbol state is also important and requires high mechanical tolerances when fabricating the human interface. Our current prototype buttons involve low-profile 3D printed parts which are assembled by hand, limiting the precision. As a result, the amplitude/phase response deviates from our simulation model and constrains the detection capacity to 8 keys. When mass-produced, KeyStub can be made with precision machinery, ensuring a key pressing event can induce a more stable and precise amplitude/phase response, hence increasing the detection capacity. Lastly, circuit contaminants such as spurious microstrip radiation will interfere with the back-scattered signal symbol state and also distort the antenna pattern. Using multi-layer PCBs and strip-line transmission lines, spurious radiation can be reduced, thus minimizing interference. In addition to interference, the stub reflects energy back toward the IC instead of dissipating it, which inherently generates a lot of spurious radiation distorting the symbol state. By integrating resistive attenuators, reflections can be mitigated through heat dissipation in the resistors.
Extending the sensing range. By applying the same mechanisms as above, we can also create the same set of symbol states with lower attenuation values, which would increase the sensing distance significantly. In addition, using lower loss power combining techniques such as [10] will also improve the link budget by at least 6 dB!
This paper presents a UHF RFID-based system that can detect 8 key press buttons similar to a wireless remote control or keypad. Our system shows resilience in heavy multi-path environments such as those created by metallic objects and dynamic movements. The novel idea we present is the use of resonant stubs to generate a large number of gesture symbols by adjusting the back-scattered signal in a deterministic manner, thus not requiring the use of complicated swiping gestures. In conjunction with deterministic symbols, we perform relative measurements of the amplitude and phase of two RFID ICs through a single antenna which removes the need for calibration. The system details in this paper can be used in other frequency bands and allow designers the ability to construct and fabricate inexpensive, wireless, battery-free communication devices for a wide range of IoT and Human-Computer-Interaction applications.
Proc. ACM Interact. Mob. Wearable Ubiquitous Technol., Vol. 7, No. 4, Article 173. Publication date: December 2023.