An official website of the United States government
Flash caches are used to reduce peak backend load for throughput-constrained data center services, reducing the total number of backend servers required. Bulk storage systems are a large-scale example, backed by high-capacity but low-throughput hard disks, and using flash caches to provide a more cost-effective storage layer underlying everything from blobstores to data warehouses. However, flash caches must address the limited write endurance of flash by limiting the long-term average flash write rate to avoid premature wearout. To do so, most flash caches must use admission policies to filter cache insertions and maximize the workload-reduction value of each flash write. The Baleen flash cache uses coordinated ML admission and prefetching to reduce peak backend load. After learning painful lessons with our early ML policy attempts, we exploit a new cache residency model (which we call episodes) to guide model training. We focus on optimizing for an end-to-end system metric (Disk-head Time) that measures backend load more accurately than IO miss rate or byte miss rate. Evaluation using Meta traces from seven storage clusters shows that Baleen reduces Peak Disk-head Time (and hence the number of backend hard disks required) by 12% over state-of-the-art policies for a fixed flash write rate constraint. Baleen-TCO, which chooses an optimal flash write rate, reduces our estimated total cost of ownership (TCO) by 17%. Code and traces are available at https://www.pdl.cmu.edu/CILES/.
more »
« less
ReFlex4ARM: Supporting 100GbE Flash Storage Disaggregation on ARM SoC
Abstract—Flash Disaggregation enables to share flash storage across the data center, improving resource utilization and reduc- ing the total cost of ownership (TCO). Previous work on flash disaggregation utilized costly server processors leaving significant headroom for optimizing TCO. In this work, we develop a new flash disaggregation system based on a cost-effective and power- efficient ARM-based Smart NIC. This work introduces our archi- tecture and provides a comprehensive evaluation outperforming previous work in TCO by 2.57x.
more »
« less
- Award ID(s):
- 1841545
- PAR ID:
- 10136133
- Date Published:
- Journal Name:
- OCP Future Technology Symposium
- Format(s):
- Medium: X
- Sponsoring Org:
- National Science Foundation
More Like this
-
-
Compute and memory are tightly coupled within each server in traditional datacenters. Large-scale datacenter operators have identified this coupling as a root cause behind fleetwide resource underutilization and increasing Total Cost of Ownership (TCO). With the advent of ultra-fast networks and cache-coherent interfaces, memory disaggregation has emerged as a potential solution, whereby applications can leverage available memory even outside server boundaries. This paper summarizes the growing research landscape of memory disaggregation from a software perspective and introduces the challenges toward making it practical under current and future hardware trends. We also reflect on our seven-year journey in the SymbioticLab to build a comprehensive disaggregated memory system over ultra-fast networks. We conclude with some open challenges toward building next-generation memory disaggregation systems leveraging emerging cache-coherent interconnects.more » « less
-
Providing itemized energy consumption in a utility bill is becoming a priority, and perhaps a business practice in the near term. In recent times, a multitude of systems have been developed such as smart plugs, smart circuit breakers etc., for non-intrusive load monitoring (NILM). They are integrated either with the smart meters or at the plug-levels to footprint appliance-level energy consumption patterns in an entire home environment While deploying the existing technologies in a single home is feasible, scaling these technological advancements across thousands of homes in a region is not realized yet. This is primarily due to the cost, deployment complexity, and intrusive nature associated with these types of real deployment. Motivated by these shortcomings, in this paper we investigate the first step to address scalable disaggregation by proposing a disaggregation mechanism that works on a large dataset to accurately deconstruct the cumulative signals. We propose an iterative noise separation based approach to perform energy disaggregation using sparse coding based methodologies which work at the single ingress point of a home, i.e., at the meter level. We performed a ranked iterative signal removal methodology that effectively isolates appliances' individual signal waveform as noise on an aggregate energy datasets with moderate granularity (1 min). We performed experiments on real dataset and obtained approximately 94% energy disaggregation, i.e., disaggregated appliance-wise signal estimation accuracy.more » « less
-
To keep global surface warming below 1.5°C by 2100, the portfolio of cost-effective CDR technologies must expand. To evaluate the potential of macroalgae CDR, we developed a kelp aquaculture bio-techno-economic model in which large quantities of kelp would be farmed at an offshore site, transported to a deep water “sink site”, and then deposited below the sequestration horizon (1,000 m). We estimated the costs and associated emissions of nursery production, permitting, farm construction, ocean cultivation, biomass transport, and Monitoring, Reporting, and Verification (MRV) for a 1,000 acre (405 ha) “baseline” project located in the Gulf of Maine, USA. The baseline kelp CDR model applies current systems of kelp cultivation to deep water (100 m) exposed sites using best available modeling methods. We calculated the levelized unit costs of CO 2 eq sequestration (LCOC; $ tCO 2 eq -1 ). Under baseline assumptions, LCOC was $17,048 tCO 2 eq -1 . Despite annually sequestering 628 tCO 2 eq within kelp biomass at the sink site, the project was only able to net 244 C credits (tCO 2 eq) each year, a true sequestration “additionality” rate (AR) of 39% (i.e., the ratio of net C credits produced to gross C sequestered within kelp biomass). As a result of optimizing 18 key parameters for which we identified a range within the literature, LCOC fell to $1,257 tCO 2 eq -1 and AR increased to 91%, demonstrating that substantial cost reductions could be achieved through process improvement and decarbonization of production supply chains. Kelp CDR may be limited by high production costs and energy intensive operations, as well as MRV uncertainty. To resolve these challenges, R&D must (1) de-risk farm designs that maximize lease space, (2) automate the seeding and harvest processes, (3) leverage selective breeding to increase yields, (4) assess the cost-benefit of gametophyte nursery culture as both a platform for selective breeding and driver of operating cost reductions, (5) decarbonize equipment supply chains, energy usage, and ocean cultivation by sourcing electricity from renewables and employing low GHG impact materials with long lifespans, and (6) develop low-cost and accurate MRV techniques for ocean-based CDR.more » « less
-
This work explores a novel approach for improving the sodium-ion battery performance of coal char using flash pyrolysis and an ether-based electrolyte. Coal char is an ultra-low cost hard carbon with promising application as an anode material in sodium-ion batteries. During flash pyrolysis, char is heated at 1000 °C/s in a drop-tube furnace to create a highly-irregular structure. The larger d-spacing and smaller closed micropore diameter of flash-pyrolyzed char increases anode capacity compared to traditional slow-pyrolyzed char electrodes. The sodium-ion battery anode performance of flash-pyrolyzed char is further improved using an ether-based electrolyte in place of the traditional ester-based electrolyte. Performance improvements include greater initial Coulombic efficiency (58% in ester- vs. 64% in ether-based electrolyte) and improved specific capacity in an ether-based electrolyte. Overall, the combination of flash pyrolysis and ether-based electrolyte increases the sodium-ion battery discharge capacity of coal char by over 50%, from 72.5 mAh g−1 (slow-pyrolyzed char in ester-based electrolyte) to 109.4 mAh g−1 (flash-pyrolyzed char in ether-based electrolyte) (50 mA g−1 discharge rate). The results highlight improvements that can be realized through flash pyrolysis of coal char for battery applications and the numerous processing advantages of flash vs. slow pyrolysis.more » « less
- Free Publicly Accessible Full Text
-
Accepted Manuscript1.0
- Conference Paper:
- The DOI is not currently available.
