Introducing OlmoEarth embeddings: Custom embedding exports from OlmoEarth Studio for downstream analysis

The Allen Institute for AI (AI2) has officially expanded the capabilities of OlmoEarth Studio, its dedicated platform for developing Earth observation models. The latest update introduces the ability to compute and export embedding vectors—compact, high-dimensional numerical representations derived from the platform’s open-source foundation models. By providing these vectors, AI2 is offering the geospatial community a streamlined, cost-effective, and highly flexible entry point into advanced satellite data analysis.

With both the model weights and source code publicly accessible, researchers and developers can now inspect the underlying mechanics of how these embeddings are generated. This transparency ensures that users can confidently integrate OlmoEarth’s deep learning representations into their own workflows, ranging from rapid similarity searches to complex land-cover segmentation.

The Power of Embedding-Based Analysis

At its core, an embedding is a mathematical distillation of complex Earth observation data. Locations that share similar surface characteristics are mapped to nearby points in a high-dimensional vector space, while distinct terrains are pushed further apart. Because these embeddings are generated on-demand within the Studio environment, they provide a dynamic snapshot of the landscape that is far more versatile than static, pre-computed global archives.

"Embeddings are a fast, cost-effective entry point for leveraging OlmoEarth: they support a wide range of downstream tasks, from similarity search to segmentation to unsupervised exploration."

The output format—Cloud-Optimized GeoTIFFs (COGs)—is designed for interoperability. These lightweight files are easy to share and compatible with standard geospatial tools like QGIS, GDAL, and Python’s rasterio library.

Configuring Your Data in Studio

The workflow for generating embeddings mirrors the standard prediction process within OlmoEarth Studio. Users maintain granular control over the output through several key configuration parameters:

• Area of Interest: Users can upload custom polygons or draw them directly; the platform handles the complex imagery acquisition and tiling.
• Temporal Range: Embeddings can be generated across 1 to 12 monthly periods, allowing for the capture of seasonal dynamics rather than just annual averages.
• Encoder Variants: Depending on the required precision and compute budget, users can select from three model sizes:
• Nano: 128-dimensional, 1.4 million parameters.
• Tiny: 192-dimensional, 6.2 million parameters.
• Base: 768-dimensional, 89 million parameters.
• Spatial Resolution: Outputs can be tuned to 10, 20, 40, or 80 meters per pixel.
• Imagery Sources: Integration with Sentinel-2 L2A, Sentinel-1 RTC, or a fusion of both.

The resulting COGs store vectors as signed 8-bit integers (int8), optimizing storage while maintaining the integrity of the data. For those requiring floating-point precision, the olmoearth_pretrain library provides a simple dequantize_embeddings function to restore the original values.

Practical Applications: From Pixels to Insights

The utility of OlmoEarth embeddings is best demonstrated through four primary analytical workflows, all of which can be executed with minimal code and no specialized training data.

1. Similarity Search: Finding "More Like This"

By selecting a specific query pixel, users can compute the cosine similarity against all other pixels in a region. This creates a heatmap that highlights areas with identical surface characteristics. For example, when querying an urban center, the model naturally distinguishes between built-up infrastructure and surrounding agricultural land without ever being explicitly trained on those categories.

2. Few-Shot Segmentation

For tasks requiring discrete land-cover maps, embeddings act as a powerful feature set. A simple linear classifier—such as a logistic regression—can be trained on as few as 60 labeled pixels to produce a high-accuracy, wall-to-wall map. In tests conducted over coastal mangrove regions in Vietnam, this method achieved a weighted F1 score of 0.84, demonstrating that the foundation model has already internalized complex ecological distinctions during its pretraining phase.

3. Change Detection

Because Studio allows for the generation of embeddings at any temporal frequency, users can perform direct comparisons between time periods. By measuring the cosine distance between two monthly embedding rasters, researchers can instantly identify surface changes—such as the burn scars left by wildfires—without the need for manual labeling or complex change-detection algorithms.

4. Unsupervised Exploration

When ground truth is unavailable, Principal Component Analysis (PCA) offers a window into the model’s internal logic. By reducing the high-dimensional vectors to three dimensions and mapping them to an RGB color space, users can visualize the structural patterns the model has identified. This often reveals clear boundaries between different crop types, water bodies, and urban zones, providing an immediate, intuitive understanding of the landscape.

Bridging the Gap: From Export to Fine-Tuning

While the examples above rely on "frozen" embeddings—meaning the model weights remain static—this approach is often sufficient for most rapid analysis tasks. However, for applications demanding the highest possible performance, OlmoEarth Studio also supports Supervised Fine-Tuning (SFT). This allows users to train a task-specific "head" on top of the foundation model using their own labeled datasets, typically yielding superior results compared to linear probes on frozen features.

Key Considerations and Limitations

While the embeddings provide a robust foundation, users should remain mindful of input data quality. Factors such as persistent cloud cover, atmospheric artifacts, or gaps in satellite observations can influence the resulting vectors. As with any machine learning application, it is recommended to validate the embedding quality against specific use cases using the techniques outlined in the official documentation.

Getting Started

The release of custom embedding exports marks a significant milestone in making Earth observation more accessible to the broader research community. Whether you are performing large-scale ecological monitoring or investigating localized land-use changes, these embeddings provide the necessary building blocks to turn raw satellite imagery into actionable intelligence.

For those ready to begin, AI2 has provided a comprehensive embeddings tutorial and a Google Colab notebook, allowing users to experiment with the workflow without the need for local infrastructure setup.

• Technical Report: Read the full OlmoEarth paper
• Documentation: Access the Embedding API docs
• Platform Access: Learn more about OlmoEarth Studio

By democratizing access to these sophisticated representations, the OlmoEarth team is enabling a new wave of innovation in how we observe, analyze, and understand our changing planet.