Multitask AI for onboard intelligence in Earth observation missions

There is a growing interest towards moving intelligence from the ground segment to onboard satellites. However, efficient use of the limited onboard computational resources is not trivial, especially in the case of AI systems employing neural networks for state-of-the-art inference on several problems. In this work, we study how a potential mission could address its AI computational payload to provide low-latency responses to multiple tasks with efficient use of resources and flexible design. We envision the use of a neural network composed of a backbone to extract features from multispectral input images at a relatively fine grained spatial resolution. This feature extractor is trained in a self-supervised manner to exploit large collections of unlabeled imagery by the mission prime designer. The features are then used by lightweight neural network heads, working in parallel, each specialized for one task. These heads can be designed independently by third-party contractors, with domain knowledge of the tasks. We show a lightweight neural network design for the backbone and a sample application head that is competitive with state-of-the-art methods at a fraction of the complexity and report interesting processing throughput results on a 7W System-on-Chip.