This guide is part of Syncing STAC Catalogs with pystac-client, within the broader Mastering Geospatial Data Ingestion in Python reference.

Filtering STAC Items by Cloud Cover and Datetime

Every scene a search returns and does not use is transfer, time and quota spent for nothing. Both of the filters that matter most for optical imagery — acquisition window and cloud cover — can be evaluated by the server, provided it says it supports them.

Why Client-Side Filtering Is the Wrong Default

  • The transfer already happened. Discarding 90% of items after fetching them saves nothing but memory.
  • Pagination cost scales with the unfiltered count. A search matching 40 000 items pages 80 times whether or not you keep them.
  • An ignored filter looks like a working one. Servers that do not implement the filter extension return everything with HTTP 200.
  • Missing properties behave inconsistently. An item with no eo:cloud_cover may be included or excluded depending on the implementation, and the difference is invisible.

Version and Environment Compatibility

Component Version Note
pystac-client >=0.8 query shorthand and filter CQL2 pass-through
pystac >=1.10 Item property access, asset resolution
Python 3.10+ Union syntax and datetime.UTC
pip install "pystac-client>=0.8" "pystac>=1.10" "geopandas>=1.0"

Two Filter Dialects

Query extension versus CQL2 Two ways to express the same predicate. The query extension takes a compact mapping of property to operator and value and is widely implemented. CQL2 as JSON is the standardised successor, more expressive and less widely available. Which one to use is decided by the conformance classes the server lists, and a client should check rather than assume. query extension · widely implemented query={ "eo:cloud_cover": {"lt": 20} } conformance: item-search#query CQL2 · standardised, less common filter={"op": "and", "args": [ {"op":"<", "args":[…, 20]}, … ]} conformance: filter · filter-cql2-json Both are ignored silently by servers that implement neither, which is why the conformance check is the first call the client makes. Prefer the query extension for portability and fall back to filtering in memory only when the server supports nothing.

The search_scenes Recipe

from __future__ import annotations

import logging
from datetime import datetime, timezone

import pystac_client

logger = logging.getLogger(__name__)

QUERY_CONFORMANCE = "https://api.stacspec.org/v1.0.0/item-search#query"


def search_scenes(
    catalog_url: str,
    collection: str,
    bbox: tuple[float, float, float, float],
    start: datetime,
    end: datetime,
    max_cloud: float = 20.0,
    drop_missing_cloud: bool = True,
    limit: int = 500,
) -> list:
    """Return items matching the window and cloud budget, filtered server-side where possible."""
    client = pystac_client.Client.open(catalog_url)
    conformances = set(client.get_conformance_classes())
    server_side = any(c.startswith(QUERY_CONFORMANCE) for c in conformances)

    if not server_side:
        logger.warning("%s does not advertise the query extension — filtering in memory",
                       catalog_url)

    interval = f"{start.astimezone(timezone.utc).isoformat()}/{end.astimezone(timezone.utc).isoformat()}"
    search = client.search(
        collections=[collection],
        bbox=bbox,
        datetime=interval,
        query={"eo:cloud_cover": {"lt": max_cloud}} if server_side else None,
        limit=limit,
    )

    items = list(search.items())
    logger.info("%s: %d items in %s", collection, len(items), interval)

    def acceptable(item) -> bool:
        cloud = item.properties.get("eo:cloud_cover")
        if cloud is None:
            return not drop_missing_cloud
        return cloud < max_cloud

    kept = [item for item in items if acceptable(item)]
    if len(kept) != len(items):
        logger.info("client-side filter removed %d of %d items", len(items) - len(kept), len(items))
    return kept


def best_per_period(items: list, period: str = "%Y-%m") -> list:
    """One scene per period, chosen deterministically: least cloud, then newest, then id."""
    buckets: dict[str, list] = {}
    for item in items:
        key = item.datetime.strftime(period)
        buckets.setdefault(key, []).append(item)

    chosen = []
    for key in sorted(buckets):
        ranked = sorted(
            buckets[key],
            key=lambda i: (
                i.properties.get("eo:cloud_cover", 101.0),
                -i.datetime.timestamp(),
                i.id,
            ),
        )
        chosen.append(ranked[0])
    return chosen

Key Implementation Notes

  • The conformance check precedes the search. A client that assumes support silently downloads everything, which is the expensive failure this whole page exists to avoid.
  • The client-side filter runs regardless. It is a no-op when the server filtered correctly and a safety net when it did not, at negligible cost.
  • Missing cloud values are an explicit decision. SAR scenes and some processing levels legitimately have no cloud property; dropping them silently loses whole collections.
  • The datetime interval is timezone-aware and RFC 3339. A bare date is interpreted differently across implementations, usually as midnight UTC, which quietly excludes the last day.
  • Ranking uses a three-part key ending in the item id. Two scenes with identical cloud and timestamp then order the same way on every run, which is what makes the selection reproducible.
  • 101.0 is the sentinel for missing cloud in the ranking. It sorts below any real value, so an unknown-cloud scene is chosen only when nothing else is available.
Choosing one scene from a month of acquisitions Six acquisitions in one month plotted by cloud cover. Five range from 31 to 88 percent and one sits at 3 percent, which wins the month under the ranking rule. The deterministic tie-breaks on acquisition time and scene identifier mean the same choice is made every time the selection is recomputed. cloud cover per acquisition · one month 100% 0 80 31 88 3 49 67 selected Without an explicit rule the choice falls to result order, which no server guarantees and no re-run reproduces.

Datetime Windows That Do What You Meant

Three details about the interval catch people out repeatedly.

Closing the interval at the run's start time Two search windows over the same timeline. An open-ended interval keeps matching items published during the walk, so the result depends on how long the walk took. An interval closed at the run's start time has a fixed answer, and anything published afterwards is picked up by the next run's window. open-ended · the answer depends on the walk's duration items published mid-walk may or may not appear closed at run start · a fixed answer next run starts here, with an overlap

Open-ended intervals are written 2026-04-01T00:00:00Z/.. and are useful for “everything since”, but they interact badly with a paged walk over a growing collection: items added during the walk may or may not appear. For an incremental sync, close the interval at the run’s start time and let the next run pick up the remainder.

Inclusivity at the boundaries is not specified consistently. Treating both ends as inclusive and deduplicating on item id afterwards is safer than assuming, and costs one comparison.

Acquisition versus publication is the difference that produces missing scenes. The datetime property is acquisition; created and updated are catalogue times. A scene acquired inside your window but published after it will never appear in a datetime-filtered incremental sync — which is exactly the case the overlap window in incremental spatial loading with watermark timestamps exists to handle.

Troubleshooting Filtered Searches

Symptom Likely cause Fix
Filter returns everything Server ignores the unsupported parameter Check conformance; keep the client-side filter
Fewer scenes than expected Cloud property missing on valid items Decide drop_missing_cloud explicitly
Last day of the window missing Bare date interpreted as midnight Use full RFC 3339 timestamps with timezone
Different scene chosen each run Selection depends on result order Rank with an explicit three-part key
Search slow on a large window Wide interval with no bbox Narrow the bbox first; it prunes hardest
Scenes appear days late Publication lags acquisition Add an overlap window to the incremental sync

Integration Note

search_scenes returns items and downloads nothing, which keeps it testable against a saved response and cheap to run in a planning task. The download step consumes the chosen items, resolves their assets and fetches only the bands the pipeline needs, following syncing STAC catalogs with pystac-client. Recording the chosen item ids per period alongside the output is what makes a later “why this scene” question answerable.

Beyond Cloud Cover

Cloud percentage is the most-used quality filter and rarely the only one worth applying. Four others are commonly available and cheap to add once the filtering path exists.

Scene-level quality flags. Many collections expose a processing quality or validity indicator; a scene marked degraded will pass a cloud filter and still be unusable.

Sun elevation. Low-elevation acquisitions produce long shadows and poor illumination, which matters for index calculations even when the sky is clear. A minimum of around twenty degrees removes the worst of them.

Nodata or valid-pixel percentage. A scene clipped at the swath edge may be 90% empty over your area of interest while reporting excellent cloud cover, because the cloud figure describes the scene rather than your extent.

Platform or processing baseline. Mixing processing baselines within a time series introduces step changes that look like real signal; pinning the baseline, or at least recording it per scene, makes those explicable.

Each is a property predicate in the same filter, so adding them costs nothing at query time — and each removes a category of scene that would otherwise be discovered as an anomaly much further downstream.