Almost all of science is transitioning from ''data-poor'' to ''data-rich'' in the 21st century, and large-scale observatories and shared-use instruments are playing essential roles in catalyzing scientific discoveries. With the increasing availability of shared-use instruments and observatories, observational data is becoming an important part of application workflows and contributor to scientific discoveries in a range of disciplines, and has enabled key scientific discoveries [1,2].
However, the corresponding growth in the number of users accessing these facilities coupled with the expansion in the scale and variety of the data, is making it challenging for these facilities to ensure their data can be accessed, integrated and analyzed in a timely manner, and is resulting in significant demands on their cyberinfrastructure (CI). At the same time, applications that use data from these observatories expect robust, highly available and • Design and prototyping of a push-based data delivery framework along with optimization strategies that leverage user data access patterns for data pre-fetching and caching.
• Experimentally evaluate the push-based data delivery framework using a simulated VDC platform and access traces from OOI and GAGE.
The remainder of this paper is organized as follows. Section 2 presents motivations for the presented work. Section 3 analyzes the user data access patterns using OOI and GAGE access traces. Section 4 describes the design push-based data delivery framework along with optimization strategies for data pre-fetching and caching. Section 5 presents a performance evaluation of the system. Section 6 concludes the paper.
Large-scale, shared-use scientific facilities, such as shared instruments, observatories and experimental platforms, have become key enablers for scientific discoveries in a range of disciplines. For example, the Event Horizon Telescope (EHT) generated the first images of a black hole event horizon [2], and the Laser Interferometer Gravitational-Wave Observatory (LIGO) enabled gravitational waves to be detected for the first time [1]. As a result, ensuring efficient, pervasive and democratic access to data from such facilities is essential and critical to amplifying their scientific impact.
However, rapidly growing data volumes and rates and the lack of sufficiently CI capabilities can limit effective access, and prevent scientists from using the data in a timely manner. For example, the LIGO project generates terabytes of data every day during its ''observing" mode [28], and the in-construction Square Kilometer Array (SKA) project is estimated to generate an exabyte of raw data every day though it will then be compressed to around 10 petabytes [29]. As noted at the 2019 NSF Workshops on Connecting Large Facilities and Cyberinfrastructure [4,30], many large facilities still struggle to provide broad data transfer and access services, especially in remote environments. Furthermore, using the wide-area network (WAN) is not the most effective approach for transferring large data volumes. For example, Dart et al. [31] note that it took three months to transfer 56 terabytes of climate model output data from the distributed Coupled Model Intercomparison Project (CMIP5) archive to the National Energy Research Supercomputing Center (NERSC). As most facilities do not provide co-located general-use computing and storage resources for end-users, users and application workflows typically have to retrieve the data over the network to local or national resources (such as those provided by Extreme Science and Engineering Discovery Environment (XSEDE)). Therefore, addressing data access challenges for large facilities is critical for both scientific facilities and their users.
Cyberinfrastructure (CI) aims to knits together the end-users, data sources, software services, and computational resources using high-performance networks [32], with the overarching goal of enabling advanced science and engineering application. As the scale and complexity of applications increase, general-purpose networks quickly become insufficient to support their requirements, which can include managing and transporting of terabyteor petabyte-scale dataset [33].
The Science Demilitarized Zone (Science DMZ) network model [34] is developed to optimize the transfer of large-scale scientific data. It achieves this goal by building a high-bandwidth network (e.g., 100 Gbps) among campuses, using dedicated Data Transfer Nodes (DTNs), by-passing traditional campus firewalls, and exploiting other techniques, as discussed in [22,23], to support high-throughput disk to disk data transfer [24].
Several recent-developed CI use the Science DMZ, such as the Pacific Research Platform (PRP) [19] and the Virtual Data Collaboratory (VDC) [20]. In this work, we leverage VDC, which architecture is presented in Fig. 1. VDC constructs a high-bandwidth VDC is a data DMZ-based regional network. Users can connect to VDC using Data Transfer Nodes (DTN). The Data Services Layer support data discovery, access and processing across a federated environment that integrates multiple data sources and CI. regional network connection. Its network services layer adopts the Science DMZ model to allow users on different campuses, data sources, and computational resources to connect to the VDC network using data transfer nodes (DTN). Its data service layer provides services, such as cataloging, curating, and querying, which enables users to discover data across observatories that are part for the VDC and integrate this data into application workflows running on remote computational resources.
DTNs within the VDC architecture are used to temporarily host data as moves across the network. Physically, the DTN is a purpose-built Linux server with significant storage and nontrivial computing power that is configured with the data transfer services such as Globus' gridFTP [35][36][37], and optimized for receiving WAN transfers at high speed [38,39]. For example, a DTN is capable for executing computation-intensive tasks such as machine-learning as proposed by the Chase-CI project [21]. In this work, we propose to deploy a cache/pre-fetching layer on the DTN to accelerate the data access performance for data from observatories. We also propose to leverage DTNs for data fusion and in-transit data processing.
Proxy caching is a well-known strategy for improving data access performance for users in geographically distributed scenarios. It achieves this goal by keeping content that is likely used in the near future at the locations close to its users [15,16]. Web data pre-fetching is a mechanism for predicting and fetching data prior to future requests [40][41][42]. Caching, combined with the pre-fetching, is an effective strategy for improving the user data access experience.
The proxy caching mechanism is typically implemented at three levels based on the cache location: client level, proxy level, and server level [15]. The proxy level caching is widely used in industry to keep the content close to users but not occupy users' local storage space. The Content Delivery Networks (CDN) [17] is one implementation of proxy level caching.
The role of the DTN within the Science DMZ network is equivalent to a proxy level cache within a web system. Therefore, by deploying a cache layer onto the DTNs in VDC essentially constructs a proxy server network that functions as the CDN to accelerate user data access performance [43] but without adding any additional hardware.
The cache replacement algorithm is crucial for web caching due to cache storage limitations [15,16,44]. Wong et al. categorize commonly used web cache replacement algorithms into:
(1) Recency-based (e.g., Least-Recently-Used [45]); (2) Frequencybased (e.g., Least-Frequently-Used [46]); (3) Size-based, which evicts the largest object first [47]; and (4) Function-based, which evicts objects according to their utility value [48]. The recencybased algorithm is the most commonly used for web caching because the web data has timeliness properties. Moreover, by leverages machine learning techniques, efforts such Ali et al. [49,50] further improve the Least-Recently-Used algorithm's performance.
Pre-fetching mechanisms enhance the caching performance by fetching data prior to the users' requests [40][41][42]. Pre-fetching algorithms are usually categorized as content-based and historybased. Content-based methods predict a user's future requests by analyzing data content. Xu et al. [51] propose to predict a user's future requests based on semantic preferences of previously retrieved Web documents. The history-based method conducts predictions based on a user's history records, and is the most commonly used approach for web pre-fetching. For example, existing studies [26,52] leverage data mining techniques to learn user access patterns using mining the correlations from a user's access records. Other work [27,53] has used Markov-based approaches to predict future requests by matching the user's current access sequence with history access sequences.
However, as existing literature [54] indicates, the factors determining which data objects will be re-accessed can vary significantly for different situations. Understanding user access patterns and particular environment characteristics are important for designing effective cache replacement and data pre-fetching mechanisms.
In order to exploit the locality of DTNs and the benefit of caching and pre-fetching, we propose to design a push-based data delivery framework along with a set of optimization strategies, including a hybrid pre-fetching model. To maximize the effectiveness of caching and pre-fetching we start by analyzing the access traces from real-world observatory and learn user data access patterns. We then develop a customized caching and pre-fetching mechanism that leverages this knowledge.
In this section, we analyze traces from the NSF Ocean Observatory Initiative (OOI) and the Geodetic Facility for the Advancement of Geoscience (GAGE) with the goal of understanding how users access and utilize the data from these observatories.
OOI [3,55,56] is a networked ocean research observatory with arrays of instrumented water column moorings and buoys, profilers, gliders, and autonomous underwater vehicles (AUVs) distributed across different open ocean and coastal regions. OOI provides ocean scientists, educators, and the public the means to collect sustained, time-series datasets to enable the examination of complex, interlinked physical, chemical, biological, and geological processes operating throughout the coastal regions and open ocean. GAGE [57] (operated by UNAVCO) is a non-profit universitygoverned consortium that facilitates geoscience research and education using geodesy. GAGE services include the operation of the Network of the Americas (NOTA), which integrates a number of existing networks, and engineering, instrumentation, and data services for user of terrestrial and satellite geodetic technologies.
The analyses presented below is conducted on two traces: a one-month (November 2018) OOI user access log that contains 17.9 million user requests, and a one-year (2018) GAGE log that contains 77.8 million user requests. Each entry in the logs includes a publicly available IP address and the request metadata (e.g., filename, access timestamp, etc.).
We use the GAGE trace to study how the impact of network performance exacerbates on data access. Since GAGE has a global user community, we analyze access performance across different continents. First, we estimate the user locations by reverse tracking their public IP address. Then, we compute the user distribution, the associated data transfer volume, and the average data transfer throughput as shown in Fig. 2. It is worth noting that while GAGE does have users from Antarctica who operate GPS receivers and work with data from the McMurdo station, their access records appear to come from other continents.
We observe that available network speed directly impacts the volume of user data transfers. North America, Oceania, and Europe have the highest average network throughput, and there is a positive correlation between their user percentages and the associated data transfer volumes. In contrast, there is a negative correlation for the rest of the continents. In particular, for Asian users, although they account for 37% users, they only account for 20% of the data transfer volumes and also have the lowest network performance (about 0.568 Mbps). This implies that low network performance limits user data access from the observatory.
Data from the observatories is spatial-temporal and contains geographic and timing information in its metadata. To query this data, users need to provide the name of the data object and an observation time range, i.e., the starting and ending timestamp.
Our analysis of the OOI and GAGE traces reveals two types of requests: human requests, which are interactive requests from humans browsing the data catalog and manually downloading data, and program requests, which are from workflows or scripts that routinely download data using automated APIs.
We distinguish between these types of requests based on their access patterns, which differ in terms of their access frequencies and the queried data sets. Human requests query a set of data within a short period and then disappear. In contrast, program requests regularly query the latest data at a consistent frequency.
We classify the users corresponding to these request types as human user and program user respectively.
To identify the request types, we maintain a running time window (e.g., one week) and analyze (i) each user's request frequency and (ii) patterns of repetition in the requests within that time window. If a user requests the same set of data objects more than once per day and this pattern repeats every day during the time window, we classify this user's future requests for this set of data objects as program requests. All other requests are classified as human requests.
Since access frequencies and the queried datasets differ between human users and program users, they require different optimization strategies. As a result, it is important to understand the relative percentages and associated data access volumes for each user type.
The user percentages and associated data transfer volumes for human users and program users for OOI and GAGE are presented in Table 1. More than 86.7% of users are human users for both the observatories making them the primary user type. However, they generated less than 9.9% of the data transfers, which implies that program users are the primary data consumers. As a result, we mainly focus our optimizations on requests by program users and address human users to the extent possible.
We analyze program requests based on the request metadata, i.e., access time, data object, and time range, and identify three access patterns: regular, overlapping and real-time request. These three request types are illustrated in Fig. 3 using a sample from the OOI trace on 23 November 2018. The horizontal axis is the time of the access request, and the vertical axis is the time range requested (i.e., the starting and ending timestamp of the requested data). Each vertical blue bar in the plot is a request, and the length of the bar represents the requested time range.
Regular requests query new data since the last request without any overlap. For example, in Fig. 3(a), a user makes a request every hour, and each request queries the most recent one-hour of data.
Regular request represent the most common request type. Observatories typically receive new data and update their databases at regular intervals which is determined by the characteristics of the instruments, the CI design, and other factors such as the data acquisition mechanism (e.g., satellite, cruise recovery). As a result, users develop programs that download the most recently updated data at these regular intervals.
Real-time requests are high-frequency (e.g., once per minute) regular requests. This type of request is shown in Fig. 3(b), where a user queries the past one-minute data every minute.
Real-time or near real-time data monitoring is typically used to develop science gateways [11] that monitor event occurrences, such as earthquake detection [5]. Since most observatories do not support subscribe-based data delivery, programs use high frequency pull-based requests to implement such monitoring. Real-time requests can result in large request traffic and significant load at the observatory. Furthermore, the data returned for each real-time request is typically small. As a result, these requests result in a large number of relatively small data transfers, which can potentially increase the system overheads and, in turn, reduce the quality of service provided by the observatory.
Overlapping requests are similar to the regular requests but with overlapping time-intervals across consecutive requests. These types of queries are illustrated in Fig. 3(c), where a user queries the past one-day data every hour, resulting in a 23-hour overlap between consecutive requests. While such queries are convenient from a user's perspective (e.g., the user is running an application every hour that needs the most recent 24 h of data), they can have a negative impact on the observatory as they result in redundant data transfers and unnecessary resource utilization.
Table 2 lists the percentage data transfer resulting from each of the three request types for OOI and GAGE. Note that for OOI overlapping requests dominate (i.e., 60.8%), while for the GAGE regular request dominate (i.e., 77.2%).
To quantitatively analyze the redundant data transfers resulting from overlapping requests, we divide the data transferred into fresh data, which is the percentage of data that is not part of previous request, and duplicate data, which is the percentage of redundant data. Table 2 shows that 90.4% of data transferred in response to overlapping requests is redundant data for OOI. Considering that 60.8% of OOI requests are overlapping, there is a significant amount of redundant data transfer observed in the OOI log.
Considering there is a large amount of redundant data transfer, adding a cache layer into the CI is an effective approach for reducing traffic and resource requirements at the observatory and improving performance. Furthermore. since a large percentage of the requests have predictable patterns, pre-fetching mechanisms can also be effective.
We observe a spatial-temporal correlation across human user requests in the OOI and GAGE traces indicating that human users tend to query data objects in a particular region as they navigate an observatory's data products. There are a couple of potential explanations for this. Scientist may be studying phenomena in a particular regions and as a result, query data from instruments and sensors in that region. Furthermore, observatory data portals typically use maps to help users navigate the observatory and explore desired instruments/sensors and associated data product and as a result, user request tend to spatially correlated. Consequently, it is possible to leverage this spatial-temporal correlation to predict human user requests.
Fig. 4 shows the queried data objects (using their instrument name and the instrument location) by three selected users from the one-day OOI trace used in Section 3.4. Since OOI deploys the same type of instrument at multiple locations [58], we use these two pieces of information to represent a specific data object. In the figure, we serialize the instrument name into an ID, which is represented in the y-axis. We then serialize the instrument locations, sort them by their proximity, and use their IDs to plot the x-axis. Thus, the dots in the same row represent the same type of instrument at different locations, while the dots in the same column represent different instruments in the same location. We observe a clear spatial correlation across requests in the plot, as users query multiple data objects of one region (vertical) and the same type of data object in nearby regions (horizontal). We also observe temporal correlation in our analysis. For example, in Figs. 3 where the consecutive blue bars, which represent the time interval of a request, have the same length. This implies that user (especially program users) tend to use a moving window while querying data. These observations confirm that we can potentially leverage this spatial-temporal correlation to predict user requests.
The proposed push-based data delivery framework exploits DTNs to cache queried data, manage the cached data, learn user access patterns, and proactively push data toward appropriate users.
The framework has a server-client architecture, as shown in Fig. 5. Clients run at user-side DTNs, pre-process user requests by searching for the requested data in caches at the local and peer DTNs, forwarding the request to the server if data is not found in these caches. The server runs at server-side DTNs that are local to the observatory. It handles requests arriving at the observatory, runs, executes the data pre-fetching and streaming mechanisms, and manages the placement of cached data.
Functionally, the architecture is composed of two main components: the cache layer and the data push engine. The cache layer spans the client and server side DTN to create a distributed interconnected cache network, and uses storage at the DTNs and caches data generated in response to user requests. The data push engine hosts the data pre-fetching and streaming mechanisms. It predicts user requests based on access patterns and can pre-fetch or stream data to users according based on these predictions.
The goal of pre-fetching in this work is to predict user accesses based on observed access patterns presented in Section 3, allowing appropriate data objects to be pushed to the user. This is achieved as follows.
First, we exploit the spatial-temporal correlations to design an association rule mining-based prediction model for predicting the human requests as well as other requests that cannot be classified, based on the work from Xiong et al. [26]. Next, we design a history-based prediction model to handle program requests using the frequency of these requests. Finally, we develop a data streaming mechanism to process real-time requests.
We use the Autoregressive Integrated Moving Average (ARIMA) model, a widely used technique for time series prediction [59,60], to predict the timestamp of the user's next request. This combination of models can optimize the prediction accuracy and pre-fetching performance.
We model user requests as a sequence R i = ⟨r 1 , r 2 , . . . , r n ⟩.
Each request tuple r i includes the request timestamp ts, the name of the data object (or data stream) d, and the requested time range (i.e., the vertical bar in Fig. 3) tr. Furthermore, we represent the request of user j as the tuple R j in Eq. ( 1), where
The history-based data pre-fetching model is used when a request from a program user is identified. Specifically, during the learning period, we monitor a user's request sequence R. If its request pattern repeats more than a threshold number of times, then we identify this user as a program user and mark this series of requests as predictable, and the framework starts pre-fetching data for the user in anticipation of future requests. The learning period and threshold are empirically set to one week and 3 times, respectively in our experiments.
We use the ARIMA model to predict the timestamp of the user's next request, i.e., ts i+1 . Our implementation of the model is based on existing literature [59,60]. Training ARIMA requires the n most recent data points (i.e., a historical time interval). The higher the value of n is, the more accurate the prediction result is, but the training time is longer. In our experimental evaluation, we empirically set n to 60, which results in a training time of a few seconds and acceptable accuracy. Since the access intervals for regular request's are typically in the order of hours, this ARIMA prediction cost is acceptable.
Once the next timestamp of the user's next request, ts i+1 , is predicted, the framework can start pre-fetching relevant data. We use a pre-fetching offset to determine how far in advance this data should be pre-fetched. For instance, given offset to 0.8, the framework pre-fetches the data at a timestamp at ts i +0.8 * (ts i+1ts i ), where ts i is the timestamp of the user's last request.
The pre-fetching offset allows the user to control the prefetched timestamp and allows for time to transfer the data. Furthermore, ARIMA predictions are impacted by the considerable variance across consecutive access times and it can take several cycles for ARIMA to adapt, and an appropriate offset provides sufficient time to buffer the ARIMA prediction latency and achieve a higher pre-fetching success rate.
The association rule mining-based prediction model is used for request from human users and in cases where requests do not have identified patterns. This model uses the association rule mining Frequent Pattern Growth (FP-Growth) algorithm [61] to construct the prediction model similar to existing work [26,[62][63][64]. FP-Growth will find an association among past requests to predict future requests for data objects, i.e., d i+1 . The time step for the next request is estimated based on the last two requests as follows: ts i+1 = ts i + (ts i -ts i-1 ) and tr i+1 = tr i .
The overall model is constructed as follows: (a) FP-tree construction: The frequent-pattern tree (FP-tree) is a compact structure that stores quantitative information about frequent patterns in a database. The algorithm first scans the training dataset and counts the number of times a data point
appears in the sequence of requests. This number is called support. Then, the algorithm finds the frequency 1-itemsets by comparing the itemset support with a predefined threshold. Finally, it re-scans the dataset and constructs the FP-tree.
(b) FP-Growth: Based on the FP-tree, which contains association rules between item sets and their corresponding confidence, the algorithm filters out association rules that have a confidence value lower than a predefined threshold, to form the complete set of frequent patterns.
Selecting appropriate threshold values for support and confidence is important to the model performance. In this work, we empirically set them to 30 and 0.5, respectively. This method's prediction accuracy is lower than the historybased method due to the randomness in human user requests as observed in our experiments. Therefore, once the model generates a list of related data objects for a user sorted by their probability of repetition, the model only pre-fetches the first n data objects within the time range tr that are identical to the user's last request. The choice of n represents a trade-off between moving more data objects to increases the success rate and the resulting larger overheads. We empirically set n to 3 in our experiments.
Real-time (or near real-time) data accesses to OOI and GAGE (we observed 25.70% and 6.10% real-time requests to OOI and GAGE respectively) are implemented as a sequence of high frequency requests, which results in significant traffic at the facilities. Like most such scientific observatories [3], these facilities do not support modern push-based models for delivering realtime streaming data. Furthermore, adding native support for data streaming can be a significant investment [30].
In this effort, we explore how these request for real-time (near-real-time) data can be supported more efficiently using in-network (using DTNs) resources, request prediction and data pre-fetching, and design a data streaming mechanism as part of the push-based data-delivery framework. This is done as follows: Once an incoming request is identified as a real-time request, the framework checks the observatory for data availability and pushes the most recent data back to the user for the duration of the request. Furthermore, the same request coming from multiple users can be combined into a single request to the observatory and redundant requests can be filtered out. Finally, this data can be cached at local DTNs to satisfy subsequent requests.
The cache layer of the push-based data delivery framework designed to leverage storage at the DTNs to place data from the observatories as close to the user as possible. The design is consists of the choice of the cache eviction policy at each of the local DTNs, and an overall data placement strategy across the network of DTNs that is aware of their geographical distribution.
The cache eviction policy at the local DTNs is based on the user access patterns. As observed in the analysis presented in Section 3, over 90.1% data transfer volume came from program requests that typically use a moving window to query data, and there is an overlap between consecutive requests.
Based on these observations, the least recently used (LRU) can be an effective cache eviction policy for the local DTN, which typically have a relatively large storage capacity (typical DTN configurations have tens to hundreds of GBs of memory and TBs of fast storage). Note that the LRU eviction policy can be replaced by other eviction policies as needed, and we compare the effectiveness of different policies in our experimental evaluation.
The goal of the data placement strategy is to place the data (and its replicas) at local DTNs that are close to potential users, and to keep the hot data, i.e., data with a high chance to be reaccessed, in the cache network as long as possible. The overall data placement strategy is composed of virtual groups and local data hubs.
A virtual group is a group of users that have common data interests and are geographically close to each other. We can place data objects of interest to a virtual group at a DTN that has the best network connective to the corresponding set of users.
In order to identify virtual groups with common data interests, we use K-Means to cluster past requests and then group users corresponding to the resulting clusters based on geographical proximity. We then map these user groups to appropriate DTNs which serve as the local data hub for the group. Note that this design assumes that users consistently access the observatory using institutional (e.g., university or research laboratory) resources and that the DTNs within the VDC align with these access locations.
The selection of a local data hub for a virtual group is based on three factors: network throughput, resource availability (e.g. storage) and user request frequency. As illustrated in Eq. ( 2), we select the DTN V dh that maximizes the weighted sum of the three values as the local data hub within the virtual group. In this equation, P ij represents the network throughput from DTN v i to v j and θ p is the weight for the throughput; U i represents the DTN device resource utilization (e.g., storage and CPU) with weight θ u ; and F i represents the request frequency of requests from the virtual group members that are connected to this DTN and has weight θ f . We empirically set θ p = 0.6, θ u = 0.2, and θ f = 0.2.
Note that clustering virtual groups and the selection of local data hubs are performed periodically allowing the framework adapting to users changing their groups and interests. If the local data hub changes, the previous data hub keeps the data that was already cached, and new data is cached at the new data hub to minimize reconfiguration costs.
The virtual group concept is illustrated in Fig. 6 and is inspired by [18]. As shown in the figure, the virtual groups are overlaid on the physical infrastructure composed of DTNs that serve as the users' access points. Users below on to one or more virtual groups (e.g., Virtual Group X and Virtual Group Y in the figure) based on their data interests, and each virtual group is divided into sub groups based on their geographical location and mapped to DTNs in the physical infrastructure, which is the local data hub for the sub group serving as its access point and local cache.
The overall operation of the framework is illustrated in Fig. 5. When the client DTN receives a request, it first searches the cache layer starting with cache at the user's local DTN. Any portion of the requested data found at the local cache is returned, the peer DTNs are searched for any remaining requested data. If data is found cached at peer DTN, it compares data transfer costs from that DTN to transfer costs from the observatory and decided whether to transfer data from the DTN. Any remaining data is then requested from the server, and is also recorded by the prefetching engine allowing it to update its models. Finally, the data is pushed towards the user.
We use a simulations along with OOI and GAGE request traces to evaluate the effectiveness and performance of the push-based data delivery framework presented in this paper under different conditions, and quantitatively compare it with two state-of-theart pre-fetching models [26,27].
We developed a simulator to emulate the VDC cyberinfrastructure as shown in Fig. 7. The simulated architecture is composed of seven geographically distributed DTNs interconnected via a wide-area network (WAN). We set DTN#1 as the VDC server that provides key data services including data cataloging, discovery, querying, etc.
The presented data delivery framework is deployed on the simulated VDC platform. The VDC server DTN is also server DTN for the framework and is the access point for the observatory. This DTN hosts the pre-fetching engine and manage data placement. The other DTNs (i.e., DTN#2 -#7) are the framework's client DTNs and collectively form the cache layer. The client DTNs can be configured to run different cache policies and use LRU by default. In our experiment we also evaluate the Least-Frequently Used (LFU) policy.
To emulate GAGE's average network throughput shown in Fig. 2, we limit the client DTNs' bandwidth from 40Gbps to 10Gbps. The network bandwidth between DTNs is shown in Fig. 8. Since the bandwidth is not homogeneous across DTNs, we also evaluate the impact of the data placement strategy and the on the virtual group and local data hub schemes. The simulation assumes that users connect to their local DTNs at 100 Gbps.
The simulation uses a task queue at the VDC server to process the user requests, and ten service processes. User requests arriving at the server DTN are queued in the task queue and wait for the next available service process. Limiting the service processes to ten allows us to evaluate the impact of increasing request traffic on the observatory data service. When user requests arrive faster than the service processes can process, it results in longer queuing time and larger processing latency.
We evaluate the Hybrid Pre-fetching Model (referred to as HPM) used by the push-based data delivery framework by comparing it to two state-of-the-art spatial-temporal pre-fetching models [26,27]. The first reference model (MD1) is by Li et al. [27] and uses the Markov method to perform predictions. The authors connect geospatial data coordinates to convert the user access history into an ''access path''. They observe that such paths follow Zipf's law, and thus, they can predict user requests using a basic Markov model. In our evaluation, we add the geospatial coordinate information to the OOI and GAGE datasets and use this model.
The second reference model (MD2) is by Xiong et al. [26] and is a data mining-based method that uses a regional mesh and association rules to learn the spatial correlation, and use ARIMA to predict the temporal correlation. This model applies the same prediction strategy to all user requests. In contrast, HPM distinguishes between request types (i.e., human or program requests) and uses the appropriate model for each request type.
To evaluate data delivery performance under different network conditions, we consider three scenarios: best, medium, and worst. The best case maintains the original DTN bandwidth, as shown in Fig. 8. The medium and worst cases cap the bandwidth at each DTN at 50% and 1%, respectively.
Request traffic represents the number of requests that the observatory receives within a unit of time. Due to limited computational capacity and bandwidth, the observatory can only process a limited number of requests concurrently, and as a result, as the request traffic increases, the number of pending requests in the task queue also increases, and latency and data transfer times are correspondingly higher. This impacts the performance of the data delivery framework.
In our simulations, we consider three request traffic scenarios: low, regular, and heavy. We define the traffic corresponding to the one month traces from the OOI and GAGE as the regular request traffic, and emulate different request traffic scenarios relative to this traffic. To emulate the heavy request traffic, we compress one month of requests into a one-week time interval. It means that the observatory will receive four times the number requests within a unit time as compared to regular request traffic. Similarly, we emulate low request traffic by expanding one month of request to span a two-month time interval.
As noted previously, the simulator is configured to use the LRU cache eviction policy by default, and our evaluations compares LRU with LFU. We also compare the performance for different cache sizes. Specifically, due to the difference in data sizes, we evaluate the OOI trace with cache sizes of 128 GB, 256 GB, 512 GB, 1 TB, and 10 TB, and the GAGE trace with cache sizes of 32 GB, 64 GB, 128 GB, 256 GB, and 10 TB. Note that the 10 TB cache is large enough to cache the entire data and represents best-case performance.
Given that OOI and GAGE requested are distributed across the globe, in our simulations, we cluster request based on the continent they originate from. We use the client DTNs (DTN#2 -#7) to represent the six continents (excluding Antarctica) and distribute users across these DTNs according to their locations. By default, the simulator uses the best network configuration and regular request traffic configuration.
In our experiments, we measure latency and throughput to quantify the data delivery performance to end-users. Latency is defined as the time between when the user submits a request to when the observatory starts processing it, and includes the time it spends in the observatory task queue. We compute throughput, by dividing the data size by the total data transfer time.
Furthermore, we use the recall metric to evaluate the performance of the pre-fetching mechanism, which is the percentage of the pre-fetched data that is accessed by the user. A higher recall value indicates that a smaller amount of pre-fetched data is wasted. Pre-fetched data may be wasteful if the prediction was incorrect, if the data is not delivered on time (i.e., before the user requests it), or if the cache configuration caused it to be evicted before it was accessed.
This experiment compares the framework's data delivery performance considering the cache configurations described in Section 5.1.4. Specifically, it compares the throughput and latency under the following conditions:
• W /O Cache: Represents the current observatory data delivery method that processes all requests by directly transferring data from the observatory to the user.
• Cache Only: Represents the baseline for our framework. It adds a cache layer using the DTNs, but does not use any optimization strategy.
• HPM, MD1, MD2: Represent our proposed framework that includes a cache layer and data placement optimization strategies, and is configured with three different pre-fetching models.
Our framework with pre-fetching achieves significant improvement in accelerating the data delivery performance, where the experiments' results are plotted in Figs. 9-12. For example, Fig. 9 shows that, in case of OOI, using the smallest cache size (128GB), HPM increases data transfer throughput by 2,689.8x, and reduces request latency by 34.8% as compared to the No Cache case. Only adding a cache layer (i.e., the Cache Only case), we observe a 739.6x improvement in throughput. This confirms our analysis presented in Section 3 that indicated significant overlap across request. It also validates the benefits of a cache layer as a simple yet effective method for improving the data delivery performance.
Using pre-fetching results in a 3.6x increase in throughput relative to the baseline. Moreover, HPM performs better than the reference models (MD1, MD2) in all cases. As seen in Figs. 9(c)-12(c), the pre-fetching recall values indicates that HPM achieves the best prediction accuracy. This is because there are over 90.1% program requests in the traces (Table 1) and the HPM's historybased prediction model can predict these requests with high accuracy. In contrast, the reference models (MD1 and MD2) treat all requests equally, which results in at least 66.7% wasted prefetching and redundant data movement. This validates the benefits of using user data access patterns to improve the data delivery performance.
Furthermore, MD2 has a higher recall value than MD1 for all cases. This indicates that the association rule-based prediction model performs better than the Markov-based method in predicting user data request for observatories. It also suggests that HPM's associated rule-based sub-prediction model performs well for non-program requests.
Figs. 9(b)-12(b) show that our framework decreases latency (e.g., 34.8% OOI, LRU, 128GB) as compared to the No Cache case. This is because the cache layer, and the pre-fetching and data streaming mechanisms reduce the number of data requests that are sent to the server DTN. Hence, it reduces the size of the task queue and allows each request to be processed faster. Specifically, Table 3 lists the normalized count of requests arriving at the server DTN for each configuration. HPM reduces the number of requests served by the observatory in all cases, especially as compared to the Cache Only case. This validates the benefits of using the data push, data pre-fetching and data streaming mechanisms for real-time requests, allowing users to mostly retrieve data from their local DTN.
The results show that the LRU cache eviction policy works better than the LFU eviction policy for small cache size configurations. LRU achieves 79.6% (OOI, 128GB) and 91.2% (GAGE, 32GB) higher throughput than LFU using 1TB and 256GB cache sizes respectively. In contrast, the throughput curve rises steeply when using LFU and achieves higher throughput than LRU with large cache sizes (i.e., 10TB, OOI). This implies that the recency of requests is more relevant than the frequency of request for observatories. Since DTN storage resources are limited and considering the typical scale of observatory data, performing well with a small cache size is important, and as a result, we suggest that using the LRU eviction policy is a better choice. We do not further evaluate advanced recency-based eviction models [49,50] in this paper and consider it as future work.
The goal of these experiments is to understand how prefetching improves the local data reuse. Fig. 13 plots the average percentage of requests that are served from the local cache for the four strategies using the last experiment for the LRU cache configurations. The color in the plots marks the data sources, where blue is the percentage of requests served using cached data, and the yellow is percentage of requests serviced using prefetched data. These plots illustrate that pre-fetching enables users to obtain more data from their local cache. For instance, using the smallest cache sizes (OOI 128GB, GAGE 32GB) and the HPM pre-fetching model, the percentage of local data access is 41.9% and 278.8% higher than the Cache Only case for OOI and GAGE, respectively.
As opposed to passively searching cached data, the prefetching mechanism proactively pushes data toward to user. It ensures that users can access more of their data locally regardless of whether the data is reused from the previous requests. For example, in Fig. 13b, GAGE has a smaller number of overlapping requests; however, the pre-fetching mechanism still enables a much higher percentage of local data accesses as compared to the baseline.
Furthermore, the pre-fetching mechanism can achieve nearoptimal performance with a small cache size. The best performance is obtained for a 10TB cache size as the entire datasets for both traces fit into the cache. HPM with the smallest cache size can achieve 79.2% and 78.0% of this best performance, for OOI and GAGE, respectively.
Based on these experiments, we conclude that our push-based data delivery framework can enable users to access more data from their local DTN cache by proactively pushing data toward users.
In this experiment, we use HPM with LRU and the GAGE trace to evaluate the impact of the data placement strategy on the data delivery performance. Table 4 summarizes the results. The first row presents the percentage of cached data that has been optimized by the data placement strategy using different cache configurations. We observe that the data placement strategy is more effective for a small cache size because it replicates hot data to the local data hub. We also measure the average throughput for retrieving data from the peer DTN cache. The data placement strategy improves throughput by 19.81% on average. The performance increases slightly as the cache size gets larger because the local data hub can keep the data replica longer. Overall, the data placement strategy improves data delivery performance by 2.46% on average. We anticipate that our data placement strategy is likely to impact overall data delivery performance even more significantly as the scale of the DTN network grows, the network gets more heterogeneous and complex, and more the amount data retrieved from peer DTN caches increases.
The goal of this experiment is to evaluate the ability of the push-based data delivery framework presented in this paper to tolerate different network conditions and request traffic levels. As presented in Section 5.1.3, we create three network conditions, i.e., best, medium and worst, where medium and worst are 50% and 1% of the best case respective, and three levels of request traffic. Moreover, we use a LRU cache of size 1TB and 256GB for the OOI and GAGE traces respectively. We compare the achieved throughput, which is presented in Table 5.
The columns of the table compares performance of each strategy for different network conditions. The performance remains constant for HPM, MD1 and MD2 for the best and medium network conditions. However, for the worst network condition, the throughput drops down by 34.9% and 31.1% relative to the best case for OOI and GAGE, respectively. These results illustrate that the pre-fetching mechanism tolerates network bandwidth variations as it can exploit the potentially idle network resource to transfer data in advance. We can also see that the network significantly impact the No Cache case as in this case data needs to downloaded directly from the observatory. This indicates that the data delivery methods currently implemented by observatories are not very resilient to unfavorable network conditions. Conversely, the Cache Only strategy is not significantly affected by the network conditions because in this case, data is mostly retrieved from the local DTN's cache.
The rows of the table present the average data transfer throughput for each cache strategy and for the low to high request traffic levels. The request traffic impacts all the cases except Cache Only as there is limited concurrent processing in this case. Since our simulations use a fixed number (ten) of service processes, heavier request traffic implies longer queuing time. Moreover, given sufficient processing capability, multiple concurrent data transfers decrease the shared bandwidth available to each task. As a result, the data transfer time increases, which can result in the pre-fetched data arriving too late and being wasted. We conclude that controlling the request traffic at the observatory is important. The push-based data delivery framework presented in this paper addresses this by using caching and pre-fetching mechanisms to reduce the request traffic.