Modern manufacturing can be broadly characterized as being driven by data, a characteristic that holds particular relevance in the automotive domain. Process parameters for installing safety-critical vehicle systems are measured and recorded, and live monitoring frequently enables manufacturers to rapidly address the sources of defects. In addition to final quality, motion information can be leveraged to improve manufacturing efficiency and understanding of the manufacturing plant flow. By enabling the manufacturing system to track parts, products, bins, vehicles, and workers, the workers can also be presented with information relevant to their current process location or notified of unusual processes in the production sched-ule in real time. Real-time locating systems (RTLS) provide this perception to the manufacturing system, including the location of employees and assets within the facility monitored live, with wide-ranging implications for safety, process monitoring, and logistics flow.
This paper characterizes the current state of RTLS technology for automotive manufacturing through a series of tests developed to meet the needs of the automotive industry and conducted with RTLS systems from four vendors in a simulated automotive manufacturing environment. Beginning with a summary of fundamental RTLS methods and the technologies through which they are implemented, it then presents an overview of the testing environment and details of all tests performed, in addition to commentary on their motivation in the context of automotive manufacturing. Following the description of the testing methodology, selected results are presented, and the testing implications are discussed. This work is focused primarily on the testing methodology, and a second work will deeply dive into the resulting data.
Table 1. Nomenclature AFCS : Automated Fastener Confirmation System BLE : Bluetooth Low Energy CE95 : 95% Confidence Interval CEP : Circular Error, Probable CVAC : Clemson Vehicle Assembly Center GPS : Global Positioning System NFER : Near-Field Electromagnetic Ranging PDoA : Phase Difference of Arrival PoE : Power over Ethernet RFID : Radio Frequency Identification RSSI : Received Signal Strength Indicator RTLS : Real-Time Location System TDoA : Time Difference of Arrival ToA : Time of Arrival UWB : Ultra-Wideband
The global manufacturing industries are increasingly adopting Real-Time Location Systems (RTLS) to track assets such as tools, materials, and products, as well as to enhance safety protocols [1]. The systems use radio waves in the ultra-high frequency band, or ultra-wideband (UWB), to provide real-time location data. The incorporation of UWB-based RTLS in manufacturing is becoming a critical resource for decision-makers in overseeing daily operations and conversion of real-world movement to digital measurements. These digital histories of movement are instrumental in establishing the emerging digital twin of production assets and continuously supplying data about the location of assets, directing mobile robots along the most effective route, monitoring and documenting each production step, and comparing the position of a tool with the digital 3D model of the product [2].
The topic of Real-Time Locating Systems sees a fair degree of complexity; considerations span fundamental theory, design, and application. At a high level, all such systems perform the same fundamental action: measurement of the location of an asset (locating tags here) in real-time. The particular means of achieving this can vary greatly, according to myriad factors. The systems reviewed for this report all use the ultra-high frequency band, also known as ultra-wideband. This section offers a brief overview of different techniques and technologies used in RTLS implementations.
UWB RTLS has found use throughout the manufacturing industry and played a role in increasing visibility and knowledge of the current state of logistics transportation tugs [3] and forklifts [4]; product carrying, loading/unloading, and warehousing [5]; and evaluation of productivity [6] and safety [7]. This data is increasingly being brought into a digital twin of the production floor to bring all of this movement data into one location to provide a real-time view of the current state of a manufacturing hall [4,8] and provide improved decision-making capabilities [9].
All solutions evaluated for this report are examples of tagbased RTLS, wherein a system of antennas or devices with multiple antennas (frequently termed "anchors") and tracking tags generates data, which is passed to a centralized IT system. This system processes the raw data, generating usable location information [10]. Several types of raw data can be measured to determine positions [11], briefly summarized below. Figure 2. An illustration of the geometric relationships used in triangulation 2.1.1. Fundamentals
Most methods rely on measuring distances from three or more antennas in separate locations to determine location in a typically two-dimensional plane. The method of combining these measurements varies but typically sees its fundamental basis in a geometric relationship. Assuming perfectly accurate and synchronized distance measurements, a location D = (x, y) whose position is defined via distance measurements to three known locations A = (x A , y A ), B = (x B , y B ), and C = (x C , y C ) can be calculated through evaluation of the following equations:
derived by modeling the location D as the intersection of three circles centered on A, B, and C, with radii corresponding to the measured distances. With the addition of a fourth fixed reference point, position determination in three dimensions can be achieved through an analogous application of spheres. This method, known as trilateration, sees somewhat more complexity in practice, where measurements might not align perfectly to one intersection point; circles may perhaps instead bound a region or not intersect at all, requiring approximations to be made, which vary according to the specific implementation of the technology. Another common method of determining location is triangulation, relying instead on the angle measured between antennas of known position and orientation and drawing fundamentally from geometric (and, in this instance, trigonometric) relationships. This method only requires two fixed references, relying on the properties of the triangle formed between them and the point of interest. Considering angles measured from a point A = (x A , y A ) and another B = (x B , y B ), as indicated in Figure 2, the coordinates of a point D = (x, y) can be calculated with the formulae
and
These expressions assume measurement of the angles from the same axis but can be modified through addition or subtraction to the angle to accommodate different orientations of the antennas. At this stage, it bears mention that no intrinsic property of these techniques for calculating location (nor, indeed, the specific technologies used to measure location) relates to measuring a tag's motion relative to fixed references. Evaluation of movement at the measurement level is performed simply through repeated, temporally distinct measures of position. Some systems implement predictive motion modeling techniques, mainly through the presumption of the tags' momentum. These predictions, however, are made following measurement and processing and reflect decisions made in a particular implementation of RTLS technology.
Determination of the distances to three defined anchor positions allows the calculation of the tag position through trilateration. Though generally a low-cost solution for RTLS, received signal strength indicator (RSSI) methodologies, which estimate the distance to a tag based on the strength of the received signal, rely heavily on a direct line of sight between anchors and tags.
Non-line-of-sight scenarios can result in significant changes in measured signal strength not reflective of differences in distance, reducing the efficacy of such methods in indoor settings.
RTLS can leverage several different technologies to maximize their effectiveness in different contexts. Some such technologies include Bluetooth low energy (BLE), global positioning systems (GPS), near-field electromagnetic ranging (NFER), and ultra-wideband (UWB).
Some RTLS applications use Bluetooth low energy (BLE) as a means of determining assets' locations [12]. An example of an RSSI application, Bluetooth suffers from the presence of obstructions in the environments where they see use, and consequently faces difficulties in determining absolute positions. Applications seeking to evaluate proximity have been shown to effectively make use of BLE technology [13].
Although fundamentally depending on some variation of a time of arrival (ToA) methodology, where distances are estimated according to the time a sent signal is received, applications of global positioning systems (GPS) are unique in the context of RTLS in that no fixed antennas need be installed in the area of interest. Instead, location is determined by evaluating the distance to a network of satellites. GPS can achieve a high level of accuracy but generally sees notably reduced performance indoors as a result of signal attenuation [14].
Near-Field Electromagnetic Ranging (NFER) approaches to RTLS applications represent perhaps one of the most technically fundamental methods of position measurement, relying on a long-established relationship between the electric and magnetic components of a radio wave [15]. Near a small antenna, a wave's electric and magnetic components are 90 degrees out of phase with one another. As range increases, the phase difference is reduced, theoretically approaching 0 degrees at a distance infinitely far from the antenna. This relationship, characterized by the equations
for c the speed of light, f the frequency of the emitted wave, and ∆ Φ the phase difference, enables the determination of distance through measurement of the phase difference between a wave's components: a phase difference of arrival (PDoA) approach. The introduction of additional antennas allows for distance measurement along several axes, ultimately allowing for triangulation to determine position. Of course, accuracy in measuring distances is diminished at longer ranges, where even a large change in r corresponds with slight changes in ∆ ϕ . Consequently, the effective range of these systems is limited. Applications of this type see a range of near 70 m in an unobstructed environment and can achieve location accuracy within 30 cm. In a "cluttered" environment, the performance of such a system can see adverse effects as significant as reducing its range to 30 m and accuracy to 1 m [16].
Ultra-wideband (UWB) refers to transmissions with frequencies ranging from 3.1 GHz to 10.6 GHz; while technically still radio waves (and thus principally similar to other RFID techniques), these frequencies are distinct from other radio transmissions, in addition to the 2.4 GHz band commonly used in Bluetooth or local network applications. Consequently, interference from other radio activity has a reduced impact on UWB systems. Systems of this type (such as those evaluated in this report), applying a TDoA methodology, have previously been demonstrated to perform effectively in industrial settings with obstructed lines of sight [17].
RTLS solutions offer critical contributions to the generation and flow of information in the systems to which they are deployed. Reflecting this significance, developing an accurate understanding of the use and performance of such systems has been the subject of a considerable volume of prior studies. Reviewing this existing body of literature, many prior publications can be broadly classified as either offering considerations of technology use cases, presenting lab-based evaluations of system performance, or describing realistic trials of the systems under consideration.
The first of the aforementioned categories of publication tends to offer little in the way of adding to the body of knowledge surrounding RTLS performance, but in synthesizing existing works, can serve to inform an understanding of what criteria are most important for characterization efforts [18]. Additionally, review papers of this sort commonly include a call for empirical evaluation of the systems and proposed applications discussed therein in industrial settings [11,19], acknowledging the unique demands and technical challenges associated with these environments.
Works in the second category offer a broadly applicable understanding of the performance of RTLS technologies, but commonly under idealized and closely controlled circumstances that do not reflect the demands of a manufacturing environment. Examples of such works might simply aim to assess the localization accuracy of an existing system experimentally [20] or seek to characterize the performance of a newly designed system [21]. While offering insight into the potential performance of such systems, and even if intending to account for some of the challenges of use in manufacturing [22], the potential application of this knowledge to implementation in manufacturing contexts is limited by the highly variable conditions of such environments.
Addressing this limitation, the third category of published works considers RTLS solutions in real manufacturing environments. By necessity, investigation in a manufacturing setting reflects the extreme demands present therein. Within these more demanding contexts, existing works have compared the performance of different technical and algorithmic approaches on system performance [23] in a specific environment. Other works have sought to describe performance at a relatively fundamental level [24], but with tests still performed in a realistic setting which is difficult or not controlled. The methodology presented in this paper seeks to combine the merits of such works and those discussed previously, offering a synthesis of the performance of four different UWB RTLS systems in scenarios ranging from basic characterization of system performance to recreation of specific automotive manufacturing conditions.
This work evaluates the performance of four RTLS systems under varied operating conditions, use cases, and environments they may encounter in a production environment. A better understanding of the tracking performance under these conditions will assist in setting the expectations for performance and common usage of UWB RTLS systems. This testing is not meant to rank UWB RTLS solution vendors, so all data and results are anonymized and aggregated.
Each of the four RTLS systems was installed in a standardized physical arrangement, and a series of system performance tests were established to investigate each solution. Each antenna was suspended from the ceiling to simulate an overhead position in a factory environment, and representatives of each respective company participated in installing and calibrating their setup. Each company was given an overview of the performance tests and was asked to remove any smoothing, correction, or any form of position alteration that might be included in a production-deployed system to reduce deviations and improve the data output with specific knowledge of the test process. This was to ensure that the performance measured was not based on process-specific calibration or post-processing but on the underlying system's ability. For example, in a production deployment, an RTLS tag may be locked to a straight line, and output positions are thus restricted to one axis only, which can improve output positional accuracy, such as when monitoring the location of vehicles moving down an assembly line.
Each set of anchors was connected to a private offline LAN and assigned dedicated address ranges. Each company representative installed a company-provided control server, which handled collecting and processing data points from each anchor and RTLS tag. The position information was output over a TCP/IP stream to a control node (Intel NUC) running an instance of the BMW Group's software for recording, visualization, and geo-fencing the tag data. The control node was provided and configured by BMW Manufacturing Co. for this testing.
All tests were conducted in Clemson University's Vehicle Assembly Center (CVAC), a 1-to-1 scale production simulation laboratory dedicated to manufacturing research and housing a full-scale multi-station assembly line. The goals of the environment are to mimic today's factories and enable applications of novel manufacturing technologies for future manufacturing without production pressures. The vehicle's motion can be fully controlled along the line's length and height and includes target position, speed, and acceleration, which is built into a single or multi-step program to control the movement per station. Manufacturing technologies are present lineside, including advanced fastening tools from Atlas Copco, Universal Robots UR10 and UR16 collaborative robots with Robotiq end effector 3D vision, and Siemens S7-1500 PLC systems. A reconfigurable flex space adjacent to the line serves as an additional manual/semi-manual assembly station and a sub-assembly kitting area. Computational support for the CVAC laboratory includes flexible edge computation with 2D computer vision, private LAN and WLAN (2.4, 5, and 6 GHz), onsite Dell EMC PowerEdge R740xd data and processing server, and fiber link to the Clemson Palmetto Cluster high-performance computing (HPC) resources. Frames with antennas mounted were suspended from the ceiling as shown in Figure 4, with five along the assembly skid line, four above the associate work line, and two along the edge of the flex assembly space (Figure 3). Antennas were standard among all four vendors and determined to allow for altering the density of antennas based on testing needs. Each location was hung at a height of 3 m above the floor, equivalent to 2.6 m above the walking surfaces of the vehicle carrier and platforms comprising the assembly line. Additionally, ten fixed positions were defined and marked throughout the space (referenced later as locations 1 through 10, respectively), and their locations were measured using surveying tools and recorded to assist with verifying the calibration of each RTLS output. Three of these positions were defined at the raised walking surface or 40 cm above the concrete floor to examine accuracy as tags move away from the 2D horizontal calibration plane. The remaining seven were defined at the tops of fixed tripods 1.6 meters above the supporting surface.
Tests were performed to evaluate different elements of the systems' performance under normal conditions and to simulate several unique conditions of interest in the application of RTLS. Each system was subjected to all of these tests, and the live location data was recorded. Three of the four systems only recorded position data in the XY plane.
The first test of each system consisted of placing tags set to an 8 Hz output rate in each of the previously described locations and recording data over a period of twenty-three minutes, allowing for roughly 10,000 position events to be collected by each of the tags. This offered sufficient data to assess both the temporal stability and accuracy of measured positions. The locations were primarily determined by spacing the tags between 2 and 3 meters apart, with modification as necessary to accommodate the fixed positions of equipment along the raised platform. Additionally, one tag was placed on the hood of the vehicle, one tag under the closed hood of the vehicle, one tag in the center of the first-row seating area, one tag on the vehicle carrier skid (on the rear right-hand side of the vehicle's exterior at 1.6 m above the work surface), and one in the center of the sub-assembly work area again at 1.6 m above the concrete floor. This combination of tag locations was regarded as not only spanning the totality of the testing environment but also evaluating conditions of interest to industrial application: namely, examining the accuracy and stability in close proximity to and obstructed by a vehicle body, where UWB signals might see some degree of attenuation or distortion. During data collection, all equipment inside the testing area was stationary, with no human or vehicle traffic through the environment. In the case that a system required two such tests due to insufficient available tags for evaluation, these were performed sequentially, with the environment only disrupted to move tags between locations.
Universal Robots UR10 and UR16e collaborative robots were used to generate spatially and temporally reproducible movements to evaluate the stability and repeatability of measurements of dynamic motions at multiple speeds. Each test condition saw the relevant movement repeated ten times sequentially at one of two different speeds, with an 8 Hz tag mounted directly to the robot's end effector or to an extension rigidly mounted to the robot. The UR10 testing was completed near the center of the calibrated area with a clear line of sight to antennas and approximately at waist height above the platform to avoid interference from surrounding surfaces or fixtures. This position provided a near-best-case scenario for the tracking task under consideration. As a supplement to the evaluation using the UR10, a Universal Robots UR16e was affixed to the assembly platform work surface between antenna positions 6 and 7 and used to generate precise, repeatable motions for system evaluation. This moved the robot to a much lower position, relative to the XY calibration plane of antennas, and moved the testing to one side of the sensed area, reducing the number of antennas within a clear line of sight, a scenario lending itself less readily to effective data collection, thereby contributing to a more complete characterization of each system. UR10 X-Y Arcs. The first of the UR10 movements was a 180degree arc of radius 1.3 m in the horizontal (X-Y) plane, performed first with angular speed and acceleration limited to 80 degrees/sec (approx. 1.8 m/s at tag) and 80 degrees/sec 2 , respectively, then to 180 degrees/sec (approx. 4 m/s at tag) and 180 degrees/sec 2 . Ten repetitions of this movement, including travel through the arc and an opposite movement returning to the starting position, were performed. This motion was repeated with the tag mounted to the end of a rigid extension on the robot, increasing the radius of each arc to 2 m and, accordingly, increasing the tag's maximum speed to approximately 2.8 m/s and 6.2 m/s for each angular velocity profile. The higher angular speed was required to simulate the tracking of conveyor systems and logistics vehicles. UR10 Linear. A tag was affixed to the robot's end effector, and a linear movement pattern was programmed to move the tag a distance of 2 m parallel to the laboratory x-axis. The robot ran through ten repetitions, returning to its start point. Similar to the rotation-based tests, speed and acceleration were restricted: first to 1000 mm/s and 100 mm/s 2 , respectively, and to 3000 mm/s and 3000 mm/s 2 in an additional test. For completeness, the text was conducted again with the tag affixed to a rigid attachment, but with the robot's movements still calculated according to the same tool center position as previously (the center of the end effector). This extended reach yielded a unique, slightly s-shaped movement through the testing space, which followed a nearly linear path but with slight deviations along each principal axis.
The slight deviation was viewed as beneficial to allow for measuring each system's ability to measure minor variations in the path over time. UR10 X-Z Arcs. Following testing in a horizontal plane, a similar series of movements was performed with an arc through the X-Z plane, moving both horizontally and vertically. As such, this test evaluates the same considerations as the X-Y arc tests (namely, the stability and accuracy of tracked movements) under sub-optimal conditions, including vertical motion not accounted for in the configuration of most systems. The tag was mounted at the same 1.3 m and 2 m radii as in the horizontal arc test, and the same two angular speed profiles were used for two iterations of this test. As previously, the UR10 was programmed to move through a 180-degree arc and return to its starting position ten times. UR10 Y-Z Arc (Extended). Extending the concept of the X-Z arc testing to its extreme, the final test condition using the UR10 was conducted only with tags attached to the end of the rigid extension to the robot (a 2 m radius). The robot was programmed to move through an arc in the Y-Z plane, allowing for additional vertical travel; the maximum height reached was identical to that of the extended X-Z arc test, with the robot and its extension oriented vertically. The minimum height reached, however, was significantly lower than that of the X-Y and X-Z tests, with the tag mounted to the rigid extension reaching a distance of only 0.25 m above the concrete floor. This experiment aimed to push the ability of each RTLS to accurately measure the path of the tag due to the majority of the motion occurring in the Z-axis, which for three of the four vendors was not explicitly calibrated and sensed.
UR16e XY and XZ Arcs. Supplementing testing conducted with the UR10, tests were replicated using a smaller UR16e robot mounted to the work surface. Effectively moving the robot below the plane evaluated previously, so farther from the calibration plane and out of the center of the tracked area. These tests were only conducted at a radius of 1 m due to the size limitation of the UR16e but with the same angular motion profiles as the UR10 robot. The resulting maximum tag velocities were approximately 1.4 and 2.8 m/s, respectively. The tests were otherwise kept consistent with the prior detailed sections.
In addition to characterizing the performance of the systems in a generally open environment, several tests were designed to evaluate their efficacy in the face of passive interference, that is, obstructing the line of sight from antennas to tags. Three such tests were conducted, two of which sought to investigate pseudorealistic scenarios for passive interference in an industrial setting, with the third test seeking to better quantify performance with respect to interference.
Aluminum Plate Test. The first interference test saw two tags, recording position data at frequencies of 8 Hz and 4 Hz placed on the floor near the center of the tracked area. Two sections of steel C channel were placed 40 cm apart with the tags between them, and a 9.5 mm aluminum plate was laid across and approximately 40 mm above the floor. Data was recorded for two minutes under this condition to evaluate whether positions were recorded with the plate in place to block UWB signals and the stability of measurements if taken.
Human Body Test. The second interference test considered interference by a human body on the tracking signal. Though interference of this sort is likely to be intermittent in most settings, it is perhaps just as likely to occur with considerable frequency: reflecting humans walking through a tracked space, reaching across their body, and other similar actions. This test consisted of an experimenter walking 5 m with a tag held out in front of them in an open hand. The experimenter then covered the tag with their other hand, ensuring that their hands fully enclosed it before walking an additional 5 m.
Aluminum Foil Test. The first two interference tests were implemented as a pass/fail metric: data was either successfully recorded or not. The final test sought to offer an opportunity for slightly more nuanced insights into performance in the face of interference. In this test, tags were wrapped between 0 and 5 layers of aluminum foil, each with a thickness of 0.02 mm. Under each condition, the linear motion test using the UR10 was repeated, with a corresponding attempt to record data throughout. Though still a discontinuous measure, this method aims to offer a more detailed consideration of how different amounts of signal attenuation impact the function of such systems and with consideration of the varied work environments RTLS are used in.
In keeping with the stated aim of evaluating the system under suboptimal conditions, Several tests were also performed to evaluate the system's resiliency as a whole: in the face of total and partial equipment failures. The failures were limited to floor-level equipment, meaning the company-provided processing PC was not power cycled as this unit will be commonly installed with a power backup and the knowledge that if the PC fails, the system ceases all data output so its recovery performance falls outside the aims of this work.
Antenna Cutout Test. The first of these tests saw antennas disconnected in a sequence designed to maintain a spatially broad distribution throughout the testing environment for the longest time possible, maintaining coverage while reducing the number of antennas. This sequence began with position 4, followed by 2, 7, 8, 3, 10, 6, 11, 1, 9, and 5, with 30 seconds in between each for the system to stabilize and allow for adequate data collection under each condition. This test was performed first with a static assembly line and robots, then with the vehicle body moving as detailed previously and the UR10 set to constantly move through the already-defined Y-Z arc.
Blackout Test. Following these antenna cutout tests, a test was performed to evaluate each system's startup time following a Figure 5. An example of the AFCS tool, with an RTLS tag affixed. loss of power. In this test, all antennas were disconnected, and following a wait of greater than 2 minutes, all antennas were simultaneously reconnected. The networking equipment used remained powered on, so any latency from power on time to tracking resumed time was related only to the antenna system.
Finally, a series of tests were devised to evaluate each system under conditions that might exist in an industrial environment.
Dynamic Test. In part, the first practical test sought to replicate the static testing described previously but in a closer approximation to a working environment. As such, this test was conducted with static tags arranged around the testing environment, in addition to an 8 Hz tag mounted inside of the vehicle body, a 1 Hz tag on the vehicle's hood, and another 8 Hz tag underneath the hood. The vehicle was then moved back and forth a distance of 12 m along the line at a rate of 100 mm/sec, being raised vertically at a speed of 14 mm/sec as it traveled in one direction to a maximum height of 185 mm, and lowered during opposite travel at the same rate. The vehicle achieved its full range of travel and returned to its starting position 5 times during the test, with data recorded throughout.
Tool-Mounted Test. Another test considered the use of tags in tracking the use of assembly tools. In this instance, a tag from each system was affixed to a tool, a part of an automated fastener confirmation system (AFCS) as shown in figure 5. The tool was then used in the completion of a simulated assembly task: tightening five fasteners, of which three were located underneath the vehicle's hood and two inside of its cabin. This test, of course, aims to evaluate each system in the course of completing a realistic vehicle assembly task. The resultant requirement that the tool be used both within and underneath the metal vehicle body, however, provides for an implicit investigation of passive interference on the RTLS signal.
Walking Tests. The final tests evaluated each system's tags as wearable devices on human users. 8 Hz tags were inserted in an experimenter's left breast pocket and affixed to a helmet worn by the experimenter. An additional 4 Hz tag was affixed to their right wrist. The experimenter was then tasked with walking through one of two different paths. One of these paths consisted of walking three laps around a section of the assembly line, followed by three laps around the assembly line as a whole, and following a series of diagonal paths through the interior of the space, before reversing along those same paths. Shown in Figure 6, this path spanned all of each system's area of coverage, excepting the offline assembly station The other path, indicated in Figure 7, aimed to characterize motion in the offline assembly station, in addition to outside of the tracked area: an important consideration for applied contexts, where implementation on a sufficiently large scale to span the area of interest in its entirety might prove prohibitively expensive, but occasional measurements outside of the defined area could prove beneficial.
This section presents the results of selected tests, aggregated across trials conducted with each system, in addition to commentary seeking to clarify and contextualize these results. The goal of this work is to detail the methodology and testing environment complete results will be included in a future work.
Analysis of data collected in the static test focused on the accuracy of positions recorded relative to the known position of tags placed in the testing environment and the consistency of measured positions (i.e., the precision of measurements). Three measures were applied to characterize performance on these fronts: for each location, a confidence interval reflecting the minimum radius from the actual position required to include 95% of measured points was calculated (CI95), in addition to a radius from the mean position required to include 50% of all recorded points (termed circular error, probable -CEP). The distance between the actual position and the average measured position (∆) was also calculated. These results are summarized in Table 2. Though generally reflecting fairly precise and accurate measurements, some outliers persist for particular points, serving to highlight the importance of validating the performance of RTLS systems throughout a space: as myriad factors can influence their performance, not the least significant of which are considerations of the physical environment surrounding tags. Table 2. A summary of selected static testing results, showing the necessary radii to capture 95% of points relative to the true position, 50% of points relative to the mean position, and the distance between these two positions: all described in meters.
#1 #2 #3 #4 #5 #6 #7 #8 CI95 0.17 0.16 0.17 0.15 0.17 0.24 0.89 0.40 CEP 0.03 0.02 0.04 0.03 0.02 0.06 0.06 0.03 ∆ 0.16 0.15 0.16 0.14 0.13 0.24 0.89 0.40
Evaluation of the data collected during the walking tests required a flexible approach. Because each test relied on the highly variable actions of a human experimenter, there is variation in the absolute position vs time to compare against. Reflective on this reality, analysis of this data took one of two forms. First, the number of expected data points was calculated based on the duration of each test. Aggregating the resulting data, the mean number of measured data points was compared to the mean number of expected data points for each tag location: head, chest, and wrist. These results are summarized in Table 3. The amount of data collected per tag varied per location, with the head-mounted tag recording the largest number of data points, as expected, followed by those mounted to the chest and finally to the wrist. This is due, in part, to the interference faced by the system in the course of conducting each walking task as the experimenter's body and environment obscured the direct line of sight to nearby antennas. The head-mounted tag was not only positioned the highest (and so less likely to be obscured by features of the environment or experimenter) but was consistently held in height and, to a degree, orientation, regardless of the wearer's movement. The other two tags saw varying degrees of obstruction according to which direction the wearer faced relative to system antennas and slightly changing height in the wrist-mounted unit due to the walking motion. This effect was even more pronounced for the out-of-bounds walking test, which included movement outside of the calibrated production line space. All locations saw a decrease in the number of recorded data points once outside of the intended area of coverage, where the head-mounted tag lost approximately 5% of expected data points, and the chest-and wrist-mounted tags' decreased a further 18% and 34%, respectively, versus the expected data. Assessing the stability of measurements reflected the aforementioned challenge: lacking a reference to a particular position. Because the head-mounted tag recorded the most points under each condition, it was selected to serve as a sort of "ground truth" for further analysis; the distances between position measurements taken at the chest or wrist within 0.25 s of those taken at the head were calculated for all measurements at the head. The results of these calculations are presented in the histograms presented in Figures 8 through 11.
In each instance, the distribution is positively skewed, albeit with a fair degree of variation between their maximum values and differing shapes of the distributions. Collectively, however, these results do serve to elucidate the challenges associated with using such systems for tracking human movement. Though certainly not impossible, careful consideration of one's interactions with the environment in the course of that movement should be made to address the potential for suboptimal data collection.
This work detailed the methodology of tests to characterize the performance of multiple RTLS solutions based on UWB technology supplied by four commercial vendors. Testing was designed to analyze the static and dynamic positional accuracy achieved by each solution, the area of coverage achieved by each system, the true frequency with which location data is recorded, the resiliency of each system to antennas being disconnected, and the time required for each system to restart following a loss of power. Additional tests were designed and performed with the intention of more completely characterizing the fundamental performance of the systems, in addition to affording insight into their behavior when subjected to the demands of a realistic manufacturing environment. The testing was designed under real-world conditions but within a controlled and repeatable environment and was the first study to the authors' knowledge to examine multiple systems under the same conditions and provide an understanding of the capabilities of modern UWB RTLS vendors. The testing methodology was detailed, and selected results were presented with future work in progress to expand upon the resulting data and takeaways. Additionally, this work allows comparisons to be made to the capabilities of other types of motion-tracking real-time location systems, such as from computer vision.
Each test considered herein was performed under the same conditions for each system, allowing for objective measurements to be made and ensuring analyses did not favor any one vendor over others. The results presented reflect these objective measurements and analyses and are in no way representative of opinions held by Clemson University, BMW Group, or any of their representatives on the systems' relative merits.