Artificial Intelligence and Advanced Flight Computing on the International Space Station

Background

Historically, space flight processors have significantly lagged ground-based processors in computing power. Recent developments in embedded processors for consumer electronics such as cell phones and other applications has renewed interest in flying COTS hardware for space missions as these applications require reduced Size, Weight, and Power (SWaP).

We benchmark a suite of flight software (FSW) applications on the Qualcomm Snapdragon 855 handheld Development Kit (HDK), a high performance embedded processor used in mobile phones. The Snapdragon 855 has several processing units including an 8 core ARM processor with clock speeds 1.7-2.8 GHz, GPU, DSP, Neural Processing Engine, and 6 GB RAM. The FSW applications tested include deep learning, instrument processing, planning and scheduling, and other benchmark applications.

We also benchmark the deep learning applications on the Intel Movidius Myriad X Vision Processing Unit (VPU). An earlier version of this processor was flown on ESA’s PhiSat-1.

The Snapdragon 855 and Movidius are onboard the International Space Station supported by the Spaceborne Computing-2 by Hewlett Packard Enterprise.

We also benchmark on Snapdragon Automotive Development Processors (ADP) running linux, NVIDIA Jetson Nano, LEON 4 based Sabertooth flight computer, and the Rad750 flight computer in ground testbeds.

Snapdragon 855 board (Image courtesy Qualcomm)

Intel Movidius Myriad (image courtesy Intel)

Technology

We have tested a range of deep learning applications on the Snapdragon Neural Processing Engine, GPU, ARM, and DSP, and on the Intel Movidius. These applications include Mars Classifiers using imagery from: MSL NAVCAM, MSL Target, MSL Engineering, and HiRise. In addition, we have Earth-based classifiers for Ice Storm detection from the SMICES smart instrument project and flood mapping from UAVSAR imagery.

Other Machine Learning techniques, including Naive Bayesian Classifiers, Support Vector Machine (SVM), and Random Decision Forest (RDF) methods, have also been deployed.

Our tests also include a range of instrument processing applications on the Snapdragon including NEAScout image co addition, SAR image formation, hyperspectral compression, decision trees (both ARM and GPU), Match filters, computer vision, and Sequential Maximum Angle Convex Cone spectral unmixing.

Onboard planning and scheduling applications testing on the Snapdragon include: CLASP coverage planning, M2020 Ground Surrogate planner, the Multimission Executive (Mexec) Europa Lander Prototype, SMICES storm targeting planning, and cloud avoidance pointing planning.

Impact

Our task to validate these algorithms on these potential and baseline flight processors has several goals:
1. to mature the applications by providing an implementation on the target flight processor(s);
2. to provide performance data for the applications on the processors; and
3. to provide some radiation impact data from the ISS flight.

Status

This effort began in Fall 2020. Hardware was delivered to the HPE/SBC-2 team in Fall 2020. SBC-2 was delivered to the ISS in Spring 2021 and experiments began in June 2021. SBC-02 and is expected to continue into Fall 2022.

Publications

Team

Jet Propulsion Laboratory, California Institute of Technology
 Faiz Mirza
 Jason Swope
 Dr. Emily Dunkel
 Evan Davis
 Dr. Zaid Towfic
 Dr. Damon Russell
 Dr. Joseph Sauvageau
 Dr. Douglas Sheldon
 Dr. Steve Chien
 Dr. Dennis Ogbe
 Lauren West

Ubotica
 Juan Romero-Cañas
 Dr. José Luis Espinosa-Aranda
 Léonie Buckley
 Elena Hervas-Martin
 Fintan Buckley

Hewlett Packard Enterprise
 Dr. Mark Fernandez
 Carrie Knox

We would also like to thank many of the application providers who also supported the testing of their applications.

Sponsors

JPL Foundry
NASA Science Mission Directorate