skip to main content


Title: MAARS: Machine learning-based Analytics for Automated Rover Systems
MAARS (Machine leaning-based Analytics for Automated Rover Systems) is an ongoing JPL effort to bring the latest self-driving technologies to Mars, Moon, and beyond. The ongoing AI revolution here on Earth is finally propagating to the red planet as the High Performance Spaceflight Computing (HPSC) and commercial off-the-shelf (COTS) system-on-a-chip (SoC), such as Qualcomm's Snapdragon, become available to rovers. In this three year project, we are developing, implementing, and benchmarking a wide range of autonomy algorithms that would significantly enhance the productivity and safety of planetary rover missions. This paper is to provide the latest snapshot of the project with broad and high-level description of every capability that we are developing, including scientific scene interpretation, vision-based traversability assessment, resource-aware path planning, information-theoretic path planning, on-board strategic path planning, and on-board optimal kinematic settling for accurate collision checking. All of the onboard software capabilities will be integrated into JPL's Athena test rover using ROS (Robot Operating System).  more » « less
Award ID(s):
1910397
NSF-PAR ID:
10318384
Author(s) / Creator(s):
; ; ; ; ; ; ; ; ; ; ; ; ; ; ; ; ; ; ; more » ; ; ; ; ; « less
Date Published:
Journal Name:
2020 IEEE Aerospace Conference
Format(s):
Medium: X
Sponsoring Org:
National Science Foundation
More Like this
  1. Obeid, Iyad ; Picone, Joseph ; Selesnick, Ivan (Ed.)
    The Neural Engineering Data Consortium (NEDC) is developing a large open source database of high-resolution digital pathology images known as the Temple University Digital Pathology Corpus (TUDP) [1]. Our long-term goal is to release one million images. We expect to release the first 100,000 image corpus by December 2020. The data is being acquired at the Department of Pathology at Temple University Hospital (TUH) using a Leica Biosystems Aperio AT2 scanner [2] and consists entirely of clinical pathology images. More information about the data and the project can be found in Shawki et al. [3]. We currently have a National Science Foundation (NSF) planning grant [4] to explore how best the community can leverage this resource. One goal of this poster presentation is to stimulate community-wide discussions about this project and determine how this valuable resource can best meet the needs of the public. The computing infrastructure required to support this database is extensive [5] and includes two HIPAA-secure computer networks, dual petabyte file servers, and Aperio’s eSlide Manager (eSM) software [6]. We currently have digitized over 50,000 slides from 2,846 patients and 2,942 clinical cases. There is an average of 12.4 slides per patient and 10.5 slides per case with one report per case. The data is organized by tissue type as shown below: Filenames: tudp/v1.0.0/svs/gastro/000001/00123456/2015_03_05/0s15_12345/0s15_12345_0a001_00123456_lvl0001_s000.svs tudp/v1.0.0/svs/gastro/000001/00123456/2015_03_05/0s15_12345/0s15_12345_00123456.docx Explanation: tudp: root directory of the corpus v1.0.0: version number of the release svs: the image data type gastro: the type of tissue 000001: six-digit sequence number used to control directory complexity 00123456: 8-digit patient MRN 2015_03_05: the date the specimen was captured 0s15_12345: the clinical case name 0s15_12345_0a001_00123456_lvl0001_s000.svs: the actual image filename consisting of a repeat of the case name, a site code (e.g., 0a001), the type and depth of the cut (e.g., lvl0001) and a token number (e.g., s000) 0s15_12345_00123456.docx: the filename for the corresponding case report We currently recognize fifteen tissue types in the first installment of the corpus. The raw image data is stored in Aperio’s “.svs” format, which is a multi-layered compressed JPEG format [3,7]. Pathology reports containing a summary of how a pathologist interpreted the slide are also provided in a flat text file format. A more complete summary of the demographics of this pilot corpus will be presented at the conference. Another goal of this poster presentation is to share our experiences with the larger community since many of these details have not been adequately documented in scientific publications. There are quite a few obstacles in collecting this data that have slowed down the process and need to be discussed publicly. Our backlog of slides dates back to 1997, meaning there are a lot that need to be sifted through and discarded for peeling or cracking. Additionally, during scanning a slide can get stuck, stalling a scan session for hours, resulting in a significant loss of productivity. Over the past two years, we have accumulated significant experience with how to scan a diverse inventory of slides using the Aperio AT2 high-volume scanner. We have been working closely with the vendor to resolve many problems associated with the use of this scanner for research purposes. This scanning project began in January of 2018 when the scanner was first installed. The scanning process was slow at first since there was a learning curve with how the scanner worked and how to obtain samples from the hospital. From its start date until May of 2019 ~20,000 slides we scanned. In the past 6 months from May to November we have tripled that number and how hold ~60,000 slides in our database. This dramatic increase in productivity was due to additional undergraduate staff members and an emphasis on efficient workflow. The Aperio AT2 scans 400 slides a day, requiring at least eight hours of scan time. The efficiency of these scans can vary greatly. When our team first started, approximately 5% of slides failed the scanning process due to focal point errors. We have been able to reduce that to 1% through a variety of means: (1) best practices regarding daily and monthly recalibrations, (2) tweaking the software such as the tissue finder parameter settings, and (3) experience with how to clean and prep slides so they scan properly. Nevertheless, this is not a completely automated process, making it very difficult to reach our production targets. With a staff of three undergraduate workers spending a total of 30 hours per week, we find it difficult to scan more than 2,000 slides per week using a single scanner (400 slides per night x 5 nights per week). The main limitation in achieving this level of production is the lack of a completely automated scanning process, it takes a couple of hours to sort, clean and load slides. We have streamlined all other aspects of the workflow required to database the scanned slides so that there are no additional bottlenecks. To bridge the gap between hospital operations and research, we are using Aperio’s eSM software. Our goal is to provide pathologists access to high quality digital images of their patients’ slides. eSM is a secure website that holds the images with their metadata labels, patient report, and path to where the image is located on our file server. Although eSM includes significant infrastructure to import slides into the database using barcodes, TUH does not currently support barcode use. Therefore, we manage the data using a mixture of Python scripts and manual import functions available in eSM. The database and associated tools are based on proprietary formats developed by Aperio, making this another important point of community-wide discussion on how best to disseminate such information. Our near-term goal for the TUDP Corpus is to release 100,000 slides by December 2020. We hope to continue data collection over the next decade until we reach one million slides. We are creating two pilot corpora using the first 50,000 slides we have collected. The first corpus consists of 500 slides with a marker stain and another 500 without it. This set was designed to let people debug their basic deep learning processing flow on these high-resolution images. We discuss our preliminary experiments on this corpus and the challenges in processing these high-resolution images using deep learning in [3]. We are able to achieve a mean sensitivity of 99.0% for slides with pen marks, and 98.9% for slides without marks, using a multistage deep learning algorithm. While this dataset was very useful in initial debugging, we are in the midst of creating a new, more challenging pilot corpus using actual tissue samples annotated by experts. The task will be to detect ductal carcinoma (DCIS) or invasive breast cancer tissue. There will be approximately 1,000 images per class in this corpus. Based on the number of features annotated, we can train on a two class problem of DCIS or benign, or increase the difficulty by increasing the classes to include DCIS, benign, stroma, pink tissue, non-neoplastic etc. Those interested in the corpus or in participating in community-wide discussions should join our listserv, nedc_tuh_dpath@googlegroups.com, to be kept informed of the latest developments in this project. You can learn more from our project website: https://www.isip.piconepress.com/projects/nsf_dpath. 
    more » « less
  2. Linking South and North America via a South Atlantic high-performance Research & Education Network (REN) with the nations of Africa’s researchers, students, and knowledge sharing communities has become an increasingly strategic priority. Africa offers research and education communities with unique biological, environmental, geological, anthropological, and cultural resources. Research challenges in atmospheric and geosciences, materials sciences, tropical diseases, biology, astronomy, and other disciplines will benefit by enhancing the technological and social connections between the research and education communities of the U.S., Brazil / Latin America, and Africa. For many years, we have seen the dramatic benefits of high-performance networking in all areas of science and engineering. The Americas Africa Research and eduCation Lightpaths (AARCLight) project (NSF OAC-1638990) provided support for a grant to plan, design, and define a strategy for high capacity research and education network connectivity between the U.S. and West Africa. The study indicated a high level of enthusiasm to engage in collaborative research between the U.S., Brazil, and the African communities. There is collaborative interest in sharing network infrastructure resources in the US at AMPATH in Miami, in Fortaleza and Sao Paulo, Brazil where RedClara and ANSP connect at SouthernLight, and in Cape Town, South Africa. There is strong evidence of multiple ongoing domain science projects between the U.S., Brazil, and Africa that would benefit from a new South Atlantic link. The results of this planning grant successfully supported the need to light a 100G pathway using the South Atlantic Cable System (SACS) connecting to AmLight-ExP in Fortaleza, Brazil, and via the West African Cable System (WACS) cable to the Cape Town, South Africa open exchange point. Based on these findings, AmLight-ExP , a high-performance R&E network supported by a consortium of participants and funding from the NSF is the steward of the SACS 100G link. With collaborative support from UbuntuNet Alliance, RNP, SANReN, and others, AmLight is taking steps to make this first South Atlantic R&E network path available to connect all three continents. This critical infrastructure establishes a new South Atlantic route to integrate with AmLight-ExP, adding resiliency to the global R&E network fabric by adding a new path to Africa and Europe from the southern hemisphere. The SACS cable, shown on Figure 1 as a purple dashed line between Fortaleza, Brazil, and Luanda, Angola, is the first east - west subsea cable in the South Atlantic. We will leverage network infrastructure in the southern hemisphere that is available to the R&E community including spectrum on Monet committed to the AmLight-ExP linking Miami, Fortaleza and São Paulo; a 100G Ethernet link on SACS; TENET’s capacity on WACS; the R&E exchange point in Cape Town-ZAOXI operated by SANReN (South African National Research Network) and TENET connected to WACS and the Ubuntunet Alliance Network connecting East Africa; and the South America eXchange R&E exchange point (SAX) in Fortaleza, operated by RNP and connected via AmLight-ExP via Monet to São Paulo and Miami. The paper will present 1) the key partners in the AmLight-SACS collaboration, 2) the activation plan, 3) how the network will be instrumented for performance measurements, and to capture data for network analytics, and 4) science drivers that will benefit from the use of a South Atlantic network route between the U.S., South America and West Africa. 
    more » « less
  3. Linking South and North America via a South Atlantic high-performance Research & Education Network (REN) with the nations of Africa’s researchers, students, and knowledge sharing communities has become an increasingly strategic priority. Africa offers research and education communities with unique biological, environmental, geological, anthropological, and cultural resources. Research challenges in atmospheric and geosciences, materials sciences, tropical diseases, biology, astronomy, and other disciplines will benefit by enhancing the technological and social connections between the research and education communities of the U.S., Brazil / Latin America, and Africa. For many years, we have seen the dramatic benefits of high-performance networking in all areas of science and engineering. The Americas Africa Research and eduCation Lightpaths (AARCLight) project (NSF OAC-1638990) provided support for a grant to plan, design, and define a strategy for high capacity research and education network connectivity between the U.S. and West Africa. The study indicated a high level of enthusiasm to engage in collaborative research between the U.S., Brazil, and the African communities. There is collaborative interest in sharing network infrastructure resources in the US at AMPATH in Miami, in Fortaleza and Sao Paulo, Brazil where RedClara and ANSP connect at SouthernLight, and in Cape Town, South Africa. There is strong evidence of multiple ongoing domain science projects between the U.S., Brazil, and Africa that would benefit from a new South Atlantic link. The results of this planning grant successfully supported the need to light a 100G pathway using the South Atlantic Cable System (SACS) connecting to AmLight-ExP in Fortaleza, Brazil, and via the West African Cable System (WACS) cable to the Cape Town, South Africa open exchange point. Based on these findings, AmLight-ExP , a high-performance R&E network supported by a consortium of participants and funding from the NSF is the steward of the SACS 100G link. With collaborative support from UbuntuNet Alliance, RNP, SANReN, and others, AmLight is taking steps to make this first South Atlantic R&E network path available to connect all three continents. This critical infrastructure establishes a new South Atlantic route to integrate with AmLight-ExP, adding resiliency to the global R&E network fabric by adding a new path to Africa and Europe from the southern hemisphere. The SACS cable, shown on Figure 1 as a purple dashed line between Fortaleza, Brazil, and Luanda, Angola, is the first east - west subsea cable in the South Atlantic. We will leverage network infrastructure in the southern hemisphere that is available to the R&E community including spectrum on Monet committed to the AmLight-ExP linking Miami, Fortaleza, and São Paulo; a 100G Ethernet link on SACS; TENET’s capacity on WACS; the R&E exchange point in Cape Town-ZAOXI operated by SANReN (South African National Research Network) and TENET connected to WACS and the Ubuntunet Alliance Network connecting East Africa; and the South America eXchange R&E exchange point (SAX) in Fortaleza, operated by RNP and connected via AmLight-ExP via Monet to São Paulo and Miami. The paper will present 1) the key partners in the AmLight-SACS collaboration, 2) the activation plan, 3) how the network will be instrumented for performance measurements, and to capture data for network analytics, and 4) science drivers that will benefit from the use of a South Atlantic network route between the U.S., South America, and West Africa. 
    more » « less
  4. null (Ed.)
    Mission-critical exploration of uncertain environments requires reliable and robust mechanisms for achieving information gain. Typical measures of information gain such as Shannon entropy and KL divergence are unable to distinguish between different bimodal probability distributions or introduce bias toward one mode of a bimodal probability distribution. The use of a standard deviation (SD) metric reduces bias while retaining the ability to distinguish between higher and lower risk distributions. Areas of high SD can be safely explored through observation with an autonomous Mars Helicopter allowing safer and faster path plans for ground-based rovers. First, this study presents a single-agent information-theoretic utility-based path planning method for a highly correlated uncertain environment. Then, an information-theoretic two-stage multiagent rapidly exploring random tree framework is presented, which guides Mars helicopter through regions of high SD to reduce uncertainty for the rover. In a Monte Carlo simulation, we compare our information-theoretic framework with a rover-only approach and a naive approach, in which the helicopter scouts ahead of the rover along its planned path. Finally, the model is demonstrated in a case study on the Jezero region of Mars. Results show that the information-theoretic helicopter improves the travel time for the rover on average when compared with the rover alone or with the helicopter scouting ahead along the rover’s initially planned route. 
    more » « less
  5. The topic of this paper is the airborne evaluation of ICESat-2 Advanced Topographic Laser Altimeter System (ATLAS) measurement capabilities and surface-height-determination over crevassed glacial terrain, with a focus on the geodetical accuracy of geophysical data collected from a helicopter. To obtain surface heights over crevassed and otherwise complex ice surface, ICESat-2 data are analyzed using the density-dimension algorithm for ice surfaces (DDA-ice), which yields surface heights at the nominal 0.7 m along-track spacing of ATLAS data. As the result of an ongoing surge, Negribreen, Svalbard, provided an ideal situation for the validation objectives in 2018 and 2019, because many different crevasse types and morphologically complex ice surfaces existed in close proximity. Airborne geophysical data, including laser altimeter data (profilometer data at 905 nm frequency), differential Global Positioning System (GPS), Inertial Measurement Unit (IMU) data, on-board-time-lapse imagery and photographs, were collected during two campaigns in summers of 2018 and 2019. Airborne experiment setup, geodetical correction and data processing steps are described here. To date, there is relatively little knowledge of the geodetical accuracy that can be obtained from kinematic data collection from a helicopter. Our study finds that (1) Kinematic GPS data collection with correction in post-processing yields higher accuracies than Real-Time-Kinematic (RTK) data collection. (2) Processing of only the rover data using the Natural Resources Canada Spatial Reference System Precise Point Positioning (CSRS-PPP) software is sufficiently accurate for the sub-satellite validation purpose. (3) Distances between ICESat-2 ground tracks and airborne ground tracks were generally better than 25 m, while distance between predicted and actual ICESat-2 ground track was on the order of 9 m, which allows direct comparison of ice-surface heights and spatial statistical characteristics of crevasses from the satellite and airborne measurements. (4) The Lasertech Universal Laser System (ULS), operated at up to 300 m above ground level, yields full return frequency (400 Hz) and 0.06–0.08 m on-ice along-track spacing of height measurements. (5) Cross-over differences of airborne laser altimeter data are −0.172 ± 2.564 m along straight paths, which implies a precision of approximately 2.6 m for ICESat-2 validation experiments in crevassed terrain. (6) In summary, the comparatively light-weight experiment setup of a suite of small survey equipment mounted on a Eurocopter (Helicopter AS-350) and kinematic GPS data analyzed in post-processing using CSRS-PPP leads to high accuracy repeats of the ICESat-2 tracks. The technical results (1)–(6) indicate that direct comparison of ice-surface heights and crevasse depths from the ICESat-2 and airborne laser altimeter data is warranted. Numerical evaluation of height comparisons utilizes spatial surface roughness measures. The final result of the validation is that ICESat-2 ATLAS data, analyzed with the DDA-ice, facilitate surface-height determination over crevassed terrain, in good agreement with airborne data, including spatial characteristics, such as surface roughness, crevasse spacing and depth, which are key informants on the deformation and dynamics of a glacier during surge. 
    more » « less