Federated Autonomous MEasurement (FAME)

Background

NASA’s New Observation System (NOS) program seeks to ”involve the coordination and integration of various instruments located at different vantage points from NASA and non-NASA sources, including in orbit, airborne and even in-situ sensors to create a more dynamic and complete picture of a natural physical process.” This can be seen as an evolution of prior programs in “Sensorwebs” which used data from a number of sources (space, air, in-situ, marine) to direct further sensing to most effectively track dynamic scientific phenomena such as wildfires, flooding, volcanic activity, algal blooms, and more. Concurrently, the European Space Agency (ESA) 𝚽-Lab is initiating a “virtual constellations” initiative with similar goals.

JPL is leading an effort to demonstrate these concepts leveraging the proliferation of commercial platforms in Low Earth Orbit. We describe the capabilities that we wish to demonstrate, initial progress in demonstrations, and future demonstration plans.

Problem

Many new spacecraft, operated by a wide range of operators are being deployed for Earth imaging. However, combining these assets in a timely and effective manner is challenging due to varied interfaces to request observations, and latencies in tasking and receiving data. Integrating data analysis steps into such timelines also be time consuming and require significant expert knowledge.

Technology

FAME leverages a number of technologies as listed below.

Onboard Analysis includes deep learning and spectral analysis onboard processing algorithms for a range of science applications. In these applications a major limiting factor is that most spacecraft have only visible spectral range imagers with a few extending in the Very Near Infra-red (VNIR) spectrum.

While flying off-line learned classifiers has significant (2004) heritage (EO-1), dramatic improvements in edge computing mean that much more sophisticated onboard analysis is now possible. For example, hyperspectral unmixing onboard ASE/EO-1 took hours to execute, now with dedicated AI/CNN hardware, CNN inference can be performed in 10s of seconds or faster. Deep learning applications include: cloud screening, surface water extent (flooding/hydrology), algal bloom/ocean color, and land use. Thermal analysis for detection of volcanic activity and wildfires is also an important application although without a higher wavelength (e.g. VSWIR or TIR) such analysis is susceptible to red-edge confusion.

Spectral signature detection approaches include spectral angle mapper, matched filters, and spectral unmixing both using deep learning / CNN hardware and traditional means such as Sequential Maximum Angle Convex Cone (SMACC). Spectral anomaly detection schemes include Reed-Xiaoli.

These key capabilities enable the spacecraft to interpret the acquired imagery onboard to glean knowledge from the imagery and therefore enable quicker action.

Dynamic Targeting is a capability in which a single spacecraft uses onboard analysis of acquired data to configure or direct sensing on the same spacecraft within the same overflight. In Low Earth Orbit with a ground track velocity of 7.5km/s at 500 km altitude a 45-50 degree lookahead requires a response timeline of 30-60 seconds. Such capability is already in operations on JAXA's TANSO-FTS instrument onboard the GoSAT2 mission and has also been proposed for hunting deep convective ice storms as part of the SMICES mission concept and also for study of planetary Boundary Layer (PBL) phenomena. A particularly intriguing concept is where a satellite directly receives data continuously broadcast by other satellites (as is common for weather satellites). Such a concept could allow for lookahead significantly in advance of the satellite orbit.

Cross-tasking aka Sensorweb is a capability where data from one sensor (satellite or other) is analyzed to drive tasking of another satellite. Sensorwebs have proven effective for volcanology, flooding, wildfires, and even using commercial assets. The NOS demonstration will demonstrate sensorweb technologies made even more effective by edge computing and intersatellite links that will make notifications more rapid.

Federated Scheduling is a capability in which service requests can be made at a service portal and a distributed scheduling protocol negotiates allocation of said service (e.g. an observation or series of observations) with multiple satellite providers transparent to the end user. For more details on this approach see the companion paper in these proceedings. Figure below highlights how Federated Scheduling enables greater access to space services by extending access seamlessly to multiple providers. The figure shows how automated federated scheduling allows for service requests that may correspond to multiple observations to be handled by the scheduler(s). The Figure highlights how federated scheduling automatically handles outcomes such as specific observations not executing (for reasons such as pre-emption, operations anomalies, or even weather) and automatically issues replacement requests.

Workflow Orchestration is a capability tightly linked to federated scheduling in which workflows (e.g. acquisitions, processing of said acquisitions, issuance of alerts, triggering additional observations contingent on results of said processing, combinations of observations periodic observations etc.) can be submitted to a service portal and the service portal automatically manages the workflow distributing the lower level activities (service requests) to appropriate satellites and computation (either onboard or ground) and ensures timely delivery of end user products. Figure 6 illustrates a science workflow for detecting volcanic activity, acquiring data from satellite observations, and generation of science measurements. Such a workflow can monitor reported activity from multiple sources, initiate observation requests (including aggregate requests as indicated earlier), robustly re-request if needed, and execute processing from direct measurements to derived quantities, delivering science data products to end users. These are the types of workflows used in [8,15,16,17] the advancement would be to enable specification in standard tools such as Jupyter notebooks [19].

Impact

FAME seeks to leverage the proliferation of Low Earth Orbiting (LEO) platforms to improve temporal and spatial coverage and reduce cost of observation services; specifically a number of infrastructure advances as follows:

1. edge computing (including “AI processors”) to immediately process acquired data into knowledge;
2. improved software environments onboard spacecraft
3. flight of common distributions of Linux as to "spacecraft virtual machines" in which software is deployed as a container with access to well defined spacecraft interfaces and even tested in the cloud on Virtual Machines; and
4. satellite communications link to enable 24/7 low data rate on demand communications to and from the ground to enable both (a) rapid dissemination of observed events or features and (b) rapid tasking from the ground.

These advances enable the following technical capabilities:

1. onboard analysis of acquired data, leveraging available onboard edge compute;
2. onboard rapid self-response, leveraging the knowledge generated from the onboard analysis to direct/reconfigure sensors on the same platform;
3. rapid notification of the ground or other assets, leveraging the knowledge from onboard analysis and 24/7 satellite communications link;
4. rapid cross-tasking of other assets, wherein insights derived on one spacecraft are used to drive tasking of another spacecraft; and
5. multi-agent systems technology to orchestrate user defined workflows across both flight and ground and also across multiple spacecraft operated by different providers.

Harnessing these capabilities can usher in a new era of Earth observation.

Status

FAME formally started in Summer 2025 has a phased rollout:

- on 7 spacecraft by Spring 2026;
- on 20 spacecraft by late 2027;
- with efforts to increase the scope of the demonstration with a stretch goal of being on 60 spacecraft by Summer 2028.

Publications

Team

Jet Propulsion Laboratory, California Institute of Technology, Pasadena, CA, USA
Steve Chien (PI), Itai Zilberstein, Alberto Candela, Domenico Barretta, Evan Davis, Federico Rossi

Ubotica, Dublin, Ireland
David Rijlaarsdam, Tom Hendrix, Aubrey Dunne

Open Cosmos, Harwell, UK
Oriol Cortes Grau, Alexandre Gol i Mestre, Manel Pedra Bovez, Oriol Aragon, Juan Puig Miquel

Loft Orbital, Golden, CO
Jad Mogannam, Mitchell Scher, Pieter van Duijn

Hyspace Technologies
Arvind Subramanian, Vishesh Vatsal, Adithya Kothandhapani

Sponsors

Earth Science Technology Office (ESTO), National Aeronautics and Space Administration