skip to main content
US FlagAn official website of the United States government
dot gov icon
Official websites use .gov
A .gov website belongs to an official government organization in the United States.
https lock icon
Secure .gov websites use HTTPS
A lock ( lock ) or https:// means you've safely connected to the .gov website. Share sensitive information only on official, secure websites.


Title: MPI4Spark Meets YARN: Enhancing MPI4Spark through YARN support for HPC
Award ID(s):
2018627
PAR ID:
10568458
Author(s) / Creator(s):
; ; ;
Publisher / Repository:
IEEE
Date Published:
ISBN:
979-8-3503-2445-7
Page Range / eLocation ID:
2265 to 2274
Format(s):
Medium: X
Location:
Sorrento, Italy
Sponsoring Org:
National Science Foundation
More Like this
  1. We show that a linear model is sufficient to accurately estimate the quantity of yarn that goes into a knitted item produced on an automated knitting machine. Knitted fabrics are complex structures, yet their diverse properties arise from the arrangement of a small number of discrete, additive operations. One can estimate the masses of each of these basic yarn additions using linear regression and, in turn, use these masses to estimate the overall quantity (and local distribution) of yarn within any knitted fabric. Our proposed linear model achieves low error on a range of fabrics and generalizes to different yarns and stitch sizes. This paves the way for applications where having a known yarn distribution is important for accuracy (e.g., simulation) or cost estimation (e.g., design). 
    more » « less
  2. Due to its speed and ease of use, Spark has become a popular tool amongst data scientists to analyze data in various sizes. Counter-intuitively, data processing workloads in industrial companies such as Google, Facebook, and Yahoo are dominated by short-running applications, which is due to the majority of applications being mostly consisted of simple SQL-like queries (Dean, 2004, Zaharia et al, 2008). Unfortunately, the current version of Spark is not optimized for such kinds of workloads. In this paper, we propose a novel framework, called Meteor, which can dramatically improve the performance for short-running applications. We extend Spark with three additional operating modes: one-thread, one-container, and distributed. The one-thread mode executes all tasks on just one thread; the one-container mode runs these tasks in one container by multi-threading; the distributed mode allocates all tasks over the whole cluster. A new framework for submitting applications is also designed, which utilizes a fine-grained Spark performance model to decide which of the three modes is the most efficient to invoke upon a new application submission. From our extensive experiments on Amazon EC2, one-thread mode is the optimal choice when the input size is small, otherwise the distributed mode is better. Overall, Meteor is up to 2 times faster than the original Spark for short applications. 
    more » « less
  3. We report a flexible and wearable bacteria-powered battery in which four functional yarns are placed in parallel for biological energy harvesting. A current collecting yarn is sandwiched between two conductive/hydrophilic active yarns including electricity-generating bacteria while a polymer-passivated cathodic yarn is located next to one of the active yarns to form a biological fuel cell configuration. The device uses Shewanella oneidensis MR-1 as a biocatalyst to produce a maximum power of 17μW/cm3 and current density 327μA/cm3, which are enough to power small-power applications. This yarn-structured biobattery can be potentially woven or knitted into an energy storage fabric to provide a higher power for smart textiles. Furthermore, sweat generated from the human body can be a potential fuel to support bacterial viability, providing the long-term operation of the battery. 
    more » « less
  4. We report a flexible and wearable bacteria-powered battery in which four functional yarns are placed in parallel for biological energy harvesting. A current collecting yarn is sandwiched between two conductive/hydrophilic active yarns including electricity-generating bacteria while a polymer-passivated cathodic yarn is located next to one of the active yarns to form a biological fuel cell configuration. The device uses Shewanella oneidensis MR-1 as a biocatalyst to produce a maximum power of 17µW/cm3 and current density 327µA/cm3, which are enough to power small-power applications. This yarn-structured biobattery can be potentially woven or knitted into an energy storage fabric to provide a higher power for smart textiles. Furthermore, sweat generated from the human body can be a potential fuel to support bacterial viability, providing the long-term operation of the battery. 
    more » « less