Search for: All records

We present CrazyJoystick, a flyable handheld joystick allowing seamless interaction methods to change between joystick and hand-tracking while displaying on-demand haptic feedback in extended reality (XR). Our system comprises a quadrotor that can autonomously approach the user when needed, addressing the limitations of conventional handheld and wearable devices that require continuous carrying throughout interactions. CrazyJoystick dynamically reallocates all thrust for haptic rendering during stationary states, eliminating the need to hover while delivering feedback. A customized cage allows users to grasp the device and interact with virtual objects, receiving 3.5 degree-of-freedom feedback. This novel transition method allows us to harvest the aerial mobility from multi-rotor based haptic devices, while having high force-to-weight ratios from being handheld during interaction. This paper describes the design and implementation of CrazyJoystick, evaluates its force and torque performance, and usability of the system in three VR applications. Our evaluation of torque rendering found that users can perceive the direction with an accuracy of 92.2%. User studies further indicated that the system significantly improves presence in VR environments. Participants found on-demand haptic feedback intuitive and enjoyable, emphasizing the potential of CrazyJoystick to redefine immersive interactions in XR through portable and adaptive feedback mechanisms. 

A Flying Light Speck, FLS, is a small drone configured with light sources to illuminate different colors and textures. A swarm of FLSs illuminates complex 3D multimedia shapes in a fixed volume, a 3D display. An FLS is a mechanical device. Its failure is the norm rather than an exception, causing a point of an illumination to go dark. In this paper, we use reliability groups with dark standby FLSs to minimize the duration of time a point remains dark. We introduce three techniques to prevent a dark standby FLS from obstructing the user’s field of view, FoV. All three move the FLS out of the user’s FoV. One technique, Suspend:Closest, maximizes the utility of a standby FLS while preventing it from obstructing the user’s FoV. 

This paper presents the design and implementation of a circular flight pattern for use by a 3D multimedia display, a Dronevision (DV). A DV uses drones configured with light sources, Flying Light Specks (FLSs), that are battery powered. The flight pattern enables a swarm of FLSs to enter an opening, granting them access to the charging coils to charge their batteries. We present two algorithms for an FLS to travel from its current coordinate to rendezvous with its assigned slot on the flight pattern, Shortest Distance (SD) and Fastest Rendezvous Time (FRT). In addition to quantifying the tradeoff associated with these algorithms, we present an implementation using a swarm of Crazyflie drones with Vicon localization. 

One may construct a 3D multimedia display using miniature drones configured with light sources, Flying Light Specks (FLSs). Swarms of FLSs localize to illuminate complex 3D shapes and animated sequences. This requires FLSs to measure their relative pose (distance and angle) accurately. A challenge is how to do this when the sensors used by FLSs have a blind range that prevents them from quantifying their relative pose. Our technique, Swazure, requires FLSs to cooperate to compensate for their sensor's blind range. It implements {\em physical data independence} by abstracting the physical characteristics of the sensors, making point cloud data independent of the sensor hardware. The size of an FLS relative to the minimum distance between points of a point cloud is an important parameter. It may result in potential obstructions that prevent Swazure from quantifying relative pose. We present two techniques, move obstructing and move source, to address this limitation. Our experimental results show the superiority of the Move Obstructing technique. 

Swarical, a Swarm-based hierarchical localization technique, enables miniature drones, Flying Light Specks (FLSs), to accurately and efficiently localize and illuminate complex 2D and 3D shapes. Its accuracy depends on the physical hardware (sensors) of FLSs used to track neighboring FLSs to localize themselves. It uses the specification of the sensors to convert mesh files into point clouds that enable a swarm of FLSs to localize at the highest accuracy afforded by their sensors. Swarical considers a heterogeneous mix of FLSs with different orientations for their tracking sensors, ensuring a line of sight between a localizing FLS and its anchor FLS. We present an implementation using Raspberry cameras and ArUco markers. A comparison of Swarical with a state of the art decentralized localization technique shows that it is as accurate and more than 2x faster. 

A Flying Light Speck, FLS, is a miniature sized drone configured with light sources to illuminate different colors and textures. A swarm of FLSs illuminates complex 3D multimedia shapes in a fixed volume, a 3D display. An FLS is a mechanical device. Its failure is the norm rather than an exception, causing a point of an illumination to go dark. In this paper, we use reliability groups with dark standby FLSs to minimize the duration of time a point remains dark. This study makes two novel contributions. First, it compares a centralized and a decentralized algorithm to form groups, demonstrating the superiority of the centralized technique. Second, it detects when the dark standby FLSs may obstruct the user's field of view and relocates them with minimal impact on their provided benefit. 

We present flight patterns for a collision-free passage of swarms of drones through one or more openings. The narrow openings provide drones with access to an infrastructure component such as charging stations to charge their depleted batteries and hangars for storage. The flight patterns are a staging area (queues) that match the rate at which an infrastructure component and its openings consume drones. They prevent collisions and may implement different policies that control the order in which drones pass through an opening. We illustrate the flight patterns with a 3D display that uses drones configured with light sources to illuminate shapes. 

Today's robotic laboratories for drones are housed in a large room. At times, they are the size of a warehouse. These spaces are typically equipped with permanent devices to localize the drones, e.g., Vicon Infrared cameras. Significant time is invested to fine-tune the localization apparatus to compute and control the position of the drones. One may use these laboratories to develop a 3D multimedia system with miniature sized drones configured with light sources. As an alternative, this brave new idea paper envisions shrinking these room-sized laboratories to the size of a cube or cuboid that sits on a desk and costs less than 10K dollars. The resulting Dronevision (DV) will be the size of a 1990s Television. In addition to light sources, its Flying Light Specks (FLSs) will be network-enabled drones with storage and processing capability to implement decentralized algorithms. The DV will include a localization technique to expedite development of 3D displays. It will act as a haptic interface for a user to interact with and manipulate the 3D virtual illuminations. It will empower an experimenter to design, implement, test, debug, and maintain software and hardware that realize novel algorithms in the comfort of their office without having to reserve a laboratory. In addition to enhancing productivity, it will improve safety of the experimenter by minimizing the likelihood of accidents. This paper introduces the concept of a DV, the research agenda one may pursue using this device, and our plans to realize one. 

Swarm-Merging, SwarMer, is a decentralized framework to localize Flying Light Specks (FLSs) to render 2D and 3D shapes. An FLS is a miniature sized drone equipped with one or more light sources to generate different colors and textures with adjustable brightness. It is battery powered, network enabled with storage and processing capability to implement a decentralized algorithm such as SwarMer. An FLS is unable to render a shape by itself. SwarMer uses the inter-FLS relationship effect of its organizational framework to compensate for the simplicity of each individual FLS, enabling a swarm of cooperating FLSs to render complex shapes. SwarMer is resilient to network packet loss, FLSs failing, and FLSs leaving to charge their battery. It is fast, highly accurate, and scales to remain effective when a shape consists of a large number of FLSs.