What Are the Different Types of AI Models in Earth Observation?
Example of Principal Components Analysis (PCA) applied to EarthDaily geospatial foundation model embeddings illustrate that three different vegetation classes (left: wheat in purple, meadows in green, and corn in orange) have clear separation in embedding space (right: point cloud), indicating the model has learned distinct features through pre-training that can be useful inputs for downstream model training.
How classical machine learning, task-specific deep learning and foundation models learn from satellite data, and what each one is good for.
Satellite imagery has become a staple input for AI. Models use it to classify crops, estimate soil moisture and flag changes on the ground, and how they learn to do that differs a great deal. Some are trained on a carefully chosen set of measurements and labeled examples. Others are pretrained on huge volumes of imagery first, and that pretraining becomes a starting point for many different applications.
EarthDaily’s analytics team, led by Senior Vice President - Analytics, Chris Rampersad, groups AI models into three broad approaches: classical machine learning, task-specific deep learning and foundation models. Chris calls the lines between them “blurry,” and with reason, since foundation models themselves generally use deep learning architectures. The most practical way to tell them apart is by how each one is trained and what that training lets a developer do next.
The three approaches at a glance
|
Feature |
Classical machine learning |
Task-specific deep learning |
Foundation models |
|
Examples |
Random forest, XGBoost |
Convolutional neural networks (CNNs), transformers |
Large, broadly pretrained models, often transformer-based |
|
How it learns |
For random forests and XGBoost, decision trees learn relationships in the inputs |
Learns complex representations by adjusting neural-network weights from labeled data |
Learns reusable representations through broad pretraining, then adapts to downstream tasks |
|
Labeled data needs |
Can work well with limited labeled data |
Task-specific supervised training can require substantial labeled data |
Pretraining can use unlabeled data; may reduce labeled-data needs for downstream task |
|
Computing power |
Generally low |
Moderate to high |
High for pretraining ; varies for downstream use |
|
Example EO use |
Land cover classification from spectral values and derived features |
Crop type classification |
Inputs for downstream models, similarity search, and change detection |
Classical machine learning: models for defined tasks
Random forest and XGBoost are the familiar names here. Both build decision trees to classify observations or predict numerical values using patterns learned in training. In Earth observation they are often used to classify land cover pixel by pixel from spectral values or derived features.
They are relatively straightforward to tune, comparatively light on computing power and cope well when training data is thin. Rick Chern of EarthDaily’s analytics team points out that with limited data, simpler models can sometimes generalize better, applying what they have learned to examples they never saw in training. So the right choice depends on two things: how much data is available, and what problem needs solving.
Task-specific deep learning: learning more complex relationships
Deep learning includes convolutional neural networks, or CNNs, which learn e.g. spatial patterns in imagery. In task-specific training, the model makes predictions, those predictions are checked against labeled examples, and its internal weights shift so it does better next time.
Chris is frank about the trade-off of task-specific deep learning models against classical machine learning and downstream usage of foundation models. Task-specific deep learning models are generally “harder to build” and “take more time,” and they often ground truth, i.e. labeled examples of observations matched to what is actually on the ground. For crop mapping, that means training examples from fields where the crop type is known. The model then applies what it has learned to classify crops in other fields.
CNNs remain an established benchmark in Earth observation. Some foundation models are starting to beat them on particular tasks.
Foundation models: learning before the application is built
Chris puts the purpose of a foundation model plainly: “to make it faster and easier to build high-performing machine learning models.” A foundation model is pretrained broadly before it is adapted to any specific task. In Earth observation, that pretraining typically means learning patterns from large volumes of unlabeled satellite imagery, and the resulting models are called geospatial foundation models.
Large language models, the best known foundation models, are the obvious comparison. Feed one enough text and it learns how words connect to one another and form sentences. Feed a geospatial foundation model enough imagery with spatial context and it starts to pick up the patterns between pixels. “A building is next to a road, a road is next to a boulevard with grass,” he says. Trained over a wide enough area, the model can learn that Asia looks different from North America, and what human activity looks like next to natural landscapes. How well those patterns transfer to new regions still is being determined.
Foundation models are harder to build than both classical machine learning and task-specific deep learning models. However, once built and pretrained, foundation models give developers a head start on each new application, and the hope is that they will need less ground truth as a result. Pre-trained models are also good at similarity searching. Find an airport in one place and the model can turn up others like it elsewhere.
Many foundation models use transformer architectures, as do plenty of other deep learning models, so broad pre-training and adaptability across applications are better markers. One foundation model can serve several applications, and adapting it depends on how far the new task sits from the original. Moving from crops to building change might take only retraining on different data, while other applications could need changes to the model itself. The EarthDaily team is still working out the best approach for each.
What comes out of a geospatial foundation model?
Run new satellite imagery through a geospatial foundation model and what comes out is a set of embeddings, numerical representations of what the model has learned about each location. Chris calls them “another form of data,” one that other models find easy to work with.
Picture an RGB image, with red, green and blue values for each pixel. EarthDaily imagery has 22 spectral bands. An embedding from EarthDaily’s geospatial foundation model can have 128 channels per pixel, for example. Together, those values represent patterns learned from imagery and its context, e.g., weather, rather than direct readings from individual spectral bands.
Those embeddings can then feed other models. EarthDaily’s soil moisture model, which produces a fresh estimate every day, draws on outputs from the company’s geospatial foundation model.
EarthDaily’s focus: learning how the land changes over time
Much of the geospatial foundation model research has gone into classification, working out whether a patch of ground is forest, water or urban land. Two years ago EarthDaily started building its model around time instead, because its market has always been about how the planet changes.
Weather forecasting is a useful parallel. AI models now play a leading role in weather prediction, resting on decades of archived observations and new measurements that arrive many times a day. Chris sees the same ingredients on the land surface, where a long historical record meets frequent observation. EarthDaily’s model learns from Sentinel-2 and Landsat time series alongside weather data, and it is designed to take in daily imagery from the EarthDaily Constellation.
An early experiment shows why cadence matters. The team used imagery from the VENµS satellite as a stand-in for near-daily coverage and predicted harvest dates 50 to 60 days out. With an image every one to two days, the reported median prediction error in harvest date was about seven days. When the input was cut to one image every two to four days, the error grew to about 12 days.
On existing benchmarks, EarthDaily’s model performs competitively against similarly sized leading geospatial foundation models from major research institutes on crop classification tasks, while performing better in selected crop yield forecasting benchmark tests. . Its architecture is designed around frequent observation, and the team expects that design to pay off as EarthDaily Constellation data comes online.
Each approach has its place in Earth observation, and the right one depends on the data at hand and the question being asked. EarthDaily’s geospatial foundation model research is focused on how the land changes over time, with the longer-term goal of using that understanding to support forecasting.
About the Author
.jpg?width=289&height=168&name=Data%20Use%20cases%20(1).jpg)