Search for: All records

Distributed cloud environments running data-intensive applications often slow down because of network congestion, uneven bandwidth, and data shuffling between nodes. Traditional host metrics such as CPU or memory do not capture these factors. Scheduling without considering network conditions causes poor placement, longer data transfers, and weaker job performance. This work presents a network-aware job scheduler that uses supervised learning to predict job completion time. The system collects real-time telemetry from all nodes, uses a trained model to estimate how long a job would take on each node, and ranks nodes to choose the best placement. The scheduler is evaluated on a geo-distributed Kubernetes cluster on the FABRIC testbed using network-intensive Spark workloads. Compared to the default Kubernetes scheduler, which uses only current resource availability, the supervised scheduler shows 34–54% higher accuracy in selecting the optimal node. The contribution is the demonstration of supervised learning for real-time, network-aware job scheduling on a multi-site cluster. 

This work aims to enable efficient digital rights management for volumetric video by introducing attribute-based selective coordinate encryption for point clouds. The method encrypts only a subset of coordinates to reduce computation and latency while maintaining security. Selective encryption makes point cloud frames distorted enough to block meaningful unauthorized viewing while still allowing basic visibility. The framework allows variation in the amount and type of encrypted coordinates (X, Y, Z, or combinations). Visual degradation is measured using standard point cloud quality metrics. Results show that encrypting only X coordinates reduces encryption time by 37% and decryption time by 46% compared to full encryption. Encrypting X and Y reduces these times by 20% and 36% while still degrading visual quality. Attribute-Based Encryption allows protected content to be cached and distributed without re-encryption, which reduces computation and latency. The current evaluation covers individual frames. Future work will cover full volumetric video streams and analyze caching gains during streaming with Attribute-Based Encryption. 

Inspired by prior work suggesting undetected errors were becoming a problem on the Internet, we set out to create a measurement system to detect errors that the TCP checksum missed. We designed a client-server framework in which the servers sent known files to clients. We then compared the received data with the original file to identify undetected errors introduced by the network. We deployed this measurement framework on various public testbeds. Over the course of 9 months, we transferred a total of 26 petabytes of data. Scaling the measurement framework to capture a large number of errors proved to be a challenge. This paper focuses on the challenges encountered during the deployment of the measurement system. We also present the interim results, which suggest that the error problems seen in prior works may be caused by two distinct processes: (1) errors that slip past TCP and (2) file system failures. The interim results also suggest that the measurement system needs to be adjusted to collect exabytes of measurement data, rather than the petabytes that prior studies predicted. 

Scientific data volume is growing, and the need for faster transfers is increasing. The community has used parallel transfer methods with multi-threaded and multi-source downloads to reduce transfer times. In multi-source transfers, a client downloads data from several replicated servers in parallel. Tools such as Aria2 and BitTorrent support this approach and show improved performance. This work introduces the Multi-Source Data Transfer Protocol, MDTP, which improves multi-source transfer performance further. MDTP divides a file request into smaller chunk requests and assigns the chunks across multiple servers. The system adapts chunk sizes based on each server’s performance and selects them so each round of requests finishes at roughly the same time. The chunk-size allocation problem is formulated as a variant of bin packing, where adaptive chunking fills the capacity “bins’’ of each server efficiently. Evaluation shows that MDTP reduces transfer time by 10–22% compared to Aria2. Comparisons with static chunking and BitTorrent show even larger gains. MDTP also distributes load proportionally across all replicas instead of relying only on the fastest one, which increases throughput. MDTP maintains high throughput even when latency increases or bandwidth to the fastest server drops. 

This paper presents an alternate method for encrypting video streams using attribute-based encryption, focusing on securing the data rather than the connection between streaming endpoints. This shift allows video segments to be encrypted once at the source and cached anywhere, removing the need for per-client encryption and decryption at intermediate caches. Access can be restricted or revoked by disabling users’ private keys instead of re-encrypting the entire video stream. The approach also removes the need for decryption and re-encryption at caches, since encrypted content can be stored directly. The work introduces ABEVS, a framework that applies attribute-based encryption to DRM-enabled video streaming. ABE is integrated into a DASH-based streaming client, and video content is encrypted and distributed through intermediate caches. The system is evaluated on CloudLab. Results show that the approach reduces computational load on caches and has minimal impact on cache hit rate and video quality compared to HTTPS-based streaming. 

In this work, we introduce PSMOA, a Policy Support Multi-objective Optimization Algorithm for decentralized data replication. PSMOA combines the NSGA-III algorithm with Entropy Weighted TOPSIS to assign dynamic weights to objectives based on system policies. The method optimizes replication time, cost, popularity, and load balancing. Simulations show that PSMOA outperforms NSGA-II and NSGA-III by producing solutions with lower replication costs and faster replication times while meeting different policy requirements. The results show that PSMOA improves data replication in complex, multi-organizational environments. 

Scientific communities are increasingly using geographically distributed computing platforms. Current methods of compute placement rely on centralized controllers such as Kubernetes to match tasks with resources. This centralized model does not work well in multi-organizational collaborations. Workflows also depend on manual configurations made for a single platform and cannot adapt to changing infrastructure. This work introduces a decentralized control plane for placing computations on distributed clusters using semantic names. Semantic names are assigned to computations so they can be matched with named Kubernetes service endpoints. This approach has two main benefits. It allows job placement to be independent of location so any cluster with enough resources can run the computation. It also supports dynamic placement without requiring knowledge of cluster locations or predefined configurations. 

Flood detection is difficult in rural areas with little or no monitoring infrastructure. Smaller streams and flood-prone regions often remain unmonitored, which leaves communities vulnerable. Commercial systems cost much and use proprietary designs, so many communities cannot use them. This work presents AquaCam, a low-cost and open-source flood detection system that uses a Raspberry Pi and a camera to measure stream water levels automatically. AquaCam captures images and trains a lightweight convolutional neural network (YOLOv8) with the collected data. The model learns to recognize water in natural backgrounds and measure water height. To test whether AquaCam can adapt to new environments, we evaluated the trained model at a different site with no retraining. The system still identified water levels accurately. This shows that the approach is practical and generalizable. AquaCam moves flood detection toward being affordable, accessible, and adaptable for the communities that need it. 

Inspired by earlier findings that undetected errors were increasing on the Internet, we built a measurement system to detect errors that the TCP checksum fails to catch. We created a client–server framework in which servers sent known files to clients, and the received data was compared to the originals to identify undetected network-introduced errors. The system was deployed on several public testbeds. Over nine months, we transferred 26 petabytes of data. Scaling the system to capture many errors proved difficult. This paper describes the deployment challenges and presents interim results showing that prior error reports may come from two different sources: errors that bypass TCP and file system failures. The results also suggest that the system must collect data at the exabyte scale rather than the petabyte scale expected by earlier studies. 

A cursory look at the Internet protocol stack shows error checking capability almost at every layer, and yet, a slowly growing set of results show that a surprising fraction of big data transfers over TCP/IP are failing. As we have dug into this problem, we have come to realize that nobody is paying much attention to the causes of transmission errors in the Internet. Rather, they have typically resorted to file-level retransmissions. Given the exponential growth in data sizes, this approach is not sustainable. Furthermore, while there has been considerable progress in understanding error codes and how to choose or create error codes that offer sturdy error protection, the Internet has not made use of this new science. We propose a set of new ideas that look at paths forward to reduce error rates and better protect big data. We also propose a new file transfer protocol that efficiently handles errors and minimizes retransmissions.