This guide is part of Automated Vector & Raster Cleaning Workflows.

Validating Spatial Data Quality in Python Pipelines

The Problem: Spatial Data Fails Quietly

A tabular pipeline that receives bad data usually breaks: a string lands in a numeric column and something raises. A spatial pipeline receiving bad data usually does not break. An invalid polygon still has an area. A layer in the wrong reference system still joins — to nothing. A parcel dataset missing a third of its rows still renders, still exports, still passes every type check, and reaches a dashboard where somebody eventually notices that a district looks empty.

That asymmetry is the argument for making quality explicit rather than incidental. The checks that matter divide into four families, and each has a different failure signature and a different correct response.

Structural checks confirm the frame is what the pipeline expects at all: required columns present, dtypes right, a geometry column, a CRS. Failures here invalidate the whole batch — there is no meaningful partial result when the CRS is missing.

Geometric checks confirm each geometry is well formed: valid under GEOS, non-empty, of the expected type, within the CRS’s area of use. Failures are per-row, and the right response is usually quarantine rather than abort, as covered in quarantining invalid features to a dead-letter table.

Attribute checks confirm values are plausible: ranges, enumerations, uniqueness at the declared grain, referential integrity against lookup tables. Mostly per-row, occasionally structural when an entire column arrives null.

Relational checks confirm the dataset behaves correctly as a whole: a coverage tiles without gaps, identifiers are unique, join rates against reference layers fall in the expected band. These are the checks nobody writes and the ones that catch the expensive failures.

Prerequisites and Environment

pip install "geopandas>=1.0" "pandera>=0.20" "shapely>=2.0" "pyproj>=3.6" "pandas>=2.2"

pandera supplies the schema and check machinery; the spatial predicates come from Shapely’s vectorized API. Confirm the GEOS version, because validity behaviour differs across major releases:

import shapely

major = int(shapely.geos_version[0])
assert major >= 3 and shapely.geos_version >= (3, 10, 0), (
    f"GEOS {shapely.geos_version_string} predates make_valid; upgrade before relying on repair"
)

Version and Compatibility Matrix

Step 1 — Express the Contract as a Schema

A contract that lives in a docstring is documentation; one that lives in a schema object is a test. pandera handles the attribute side, and a thin wrapper adds the spatial predicates.

import geopandas as gpd
import pandera.pandas as pa
from pandera import Check, Column, DataFrameSchema

PARCEL_SCHEMA = DataFrameSchema(
    {
        "parcel_id": Column(str, nullable=False, unique=True),
        "zone_code": Column(str, Check.isin(["R1", "R2", "C1", "C2", "I1", "AG"])),
        "area_sqm": Column(float, Check.in_range(1.0, 5_000_000.0), nullable=False),
        "updated_at": Column("datetime64[ns]", nullable=True),
    },
    strict=False,   # extra columns are allowed; unmapped ones are handled upstream
    coerce=False,   # coercion belongs in the harmonization stage, not the gate
)

coerce=False is deliberate. A validation gate that silently fixes types is no longer a gate: it hides the very drift it exists to report. Type coercion belongs earlier, in attribute mapping and schema harmonization, where the conversions are logged.

Step 2 — Add the Spatial Invariants

from dataclasses import dataclass, field

import shapely


@dataclass
class SpatialCheckResult:
    passed: bool
    failures: dict[str, int] = field(default_factory=dict)
    detail: dict[str, list] = field(default_factory=dict)


def check_spatial_invariants(
    gdf: gpd.GeoDataFrame,
    expected_epsg: int,
    allowed_types: tuple[str, ...] = ("Polygon", "MultiPolygon"),
    id_column: str = "parcel_id",
) -> SpatialCheckResult:
    """Run the per-geometry invariants and return counts plus offending ids."""
    result = SpatialCheckResult(passed=True)

    if gdf.crs is None or gdf.crs.to_epsg() != expected_epsg:
        result.passed = False
        result.failures["crs"] = 1
        result.detail["crs"] = [str(gdf.crs)]
        return result  # structural: nothing else is meaningful yet

    geoms = gdf.geometry.values
    checks = {
        "invalid": ~shapely.is_valid(geoms),
        "empty": shapely.is_empty(geoms),
        "missing": shapely.is_missing(geoms),
        "wrong_type": ~gdf.geometry.geom_type.isin(allowed_types).to_numpy(),
    }

    for name, mask in checks.items():
        count = int(mask.sum())
        if count:
            result.passed = False
            result.failures[name] = count
            result.detail[name] = gdf.loc[mask, id_column].head(20).tolist()

    return result

Every predicate here runs over the whole array at once. The vectorized form matters at scale — shapely.is_valid over a million polygons runs in seconds, while the same check written as a row-wise apply takes minutes and produces identical results.

Step 3 — Add the Relational Checks

Per-row checks cannot see the failures that matter most: a batch that is complete but wrong as a whole.

def check_dataset_invariants(
    gdf: gpd.GeoDataFrame,
    reference: gpd.GeoDataFrame,
    expected_join_rate: float = 0.98,
    max_area_drift: float = 0.02,
    previous_area: float | None = None,
) -> dict[str, float]:
    """Whole-dataset checks: join rate against reference data, and area stability."""
    joined = gpd.sjoin(gdf, reference[["geometry"]], how="left", predicate="intersects")
    join_rate = joined["index_right"].notna().mean()

    metrics = {"join_rate": float(join_rate)}
    if join_rate < expected_join_rate:
        raise ValueError(f"join rate {join_rate:.3f} below expected {expected_join_rate}")

    total_area = float(gdf.geometry.area.sum())
    metrics["total_area"] = total_area
    if previous_area:
        drift = abs(total_area - previous_area) / previous_area
        metrics["area_drift"] = drift
        if drift > max_area_drift:
            raise ValueError(f"total area moved {drift:.1%} since the last run")

    return metrics

The join rate is the single most informative number in a spatial pipeline. A parcel layer that normally joins to its address table at 99.2% and today joins at 71% has not become invalid — it has almost certainly arrived in a different reference system, or with a shifted datum, exactly the failure described in CRS normalization across mixed datasets. No per-row check would have noticed.

Step 4 — Give Every Check a Severity

A check without a severity is an argument waiting to happen at 3 a.m. Three levels are enough.

Blocking. The batch does not proceed. Reserved for structural failures and for relational checks whose failure implies the data is wrong rather than merely unusual.

Quarantining. The offending rows are diverted and the batch continues, with the quarantine rate recorded and itself subject to a threshold. Most per-row geometric and attribute failures live here.

Observing. The result is recorded as a metric and nothing else happens. Drift measures and distribution checks start here and are promoted only once their normal range is known.

from enum import Enum


class Severity(str, Enum):
    BLOCK = "block"
    QUARANTINE = "quarantine"
    OBSERVE = "observe"


CHECK_SEVERITY = {
    "crs": Severity.BLOCK,
    "required_columns": Severity.BLOCK,
    "join_rate": Severity.BLOCK,
    "invalid": Severity.QUARANTINE,
    "empty": Severity.QUARANTINE,
    "wrong_type": Severity.QUARANTINE,
    "zone_code_unknown": Severity.QUARANTINE,
    "area_drift": Severity.OBSERVE,
    "vertex_count_p99": Severity.OBSERVE,
}

Keeping severity in a table rather than in the check bodies means it can be changed without touching the logic — and the change shows up in a diff, which is what makes “we downgraded that gate last month” a reviewable decision rather than an oral tradition.

Failure-Mode Reference

Integration Into ETL Pipelines

Quality gates belong between transform and load, where a failure protects the serving table rather than merely reporting on it — the placement argued in orchestrating spatial ETL pipelines. In Dagster they map naturally onto asset checks with a blocking severity; in Airflow they are ordinary tasks between the transform and the load; in Prefect they are functions that raise.

Whatever the tool, write the metrics somewhere queryable rather than only into logs. The value of a quarantine rate is entirely in its history: today’s 0.4% means nothing on its own and everything next to last month’s 0.05%.

Choosing Thresholds From Measured Baselines

Every threshold in a quality gate is a claim about what normal looks like, and a claim invented at the keyboard is almost always wrong in one of two expensive directions. Set it too tight and the gate fires nightly until somebody disables it. Set it too loose and it never fires at all, including on the night it should have.

The reliable procedure has three steps and takes about a week of elapsed time.

Run in observe mode first. Wire every check in, record every result, and let nothing block. The pipeline behaves exactly as before; the only change is that a table now accumulates measurements. A week of daily runs is usually enough to see the shape of the distribution, and a month is better for a source with weekly or monthly cycles.

Take the observed range, not the mean. A quarantine rate that sits between 0.05% and 0.4% over twenty runs has a normal band, and the threshold belongs above the top of that band with headroom — 1% here, not 0.5%. The mean is the wrong statistic because the interesting values are in the tail by construction.

Re-derive after every deliberate change. A new source, a changed transform, a widened extent: each invalidates the baseline. Thresholds that were derived once and never revisited are the ones that fire spuriously two years later, which is how a team learns to ignore them.

Two thresholds resist this treatment and should be absolute rather than empirical. The CRS check has no acceptable failure rate — one wrong reference system is one too many. And referential integrity on the primary key is binary: duplicate identifiers at the declared grain are always a defect, however common they become.

Sampling: When a Full Check Is Too Expensive

Most spatial checks are cheap enough to run on every row, and the vectorized predicates above are the reason. A few genuinely are not: pairwise overlap detection across a large coverage, precise area computation in a projected CRS for a global dataset, or any check that requires reading a raster’s full pixel array rather than its metadata.

For those, sampling is legitimate provided three conditions hold. The sample must be random rather than positional — the first thousand rows of a spatially sorted file cover one corner of the extent and tell you nothing about the rest. The sample size must be fixed in advance and recorded with the result, so a rate computed from 1 000 rows is not compared against one computed from 100 000. And a sampled check must be advisory only: it can raise suspicion, but blocking a batch on evidence from 1% of it is a decision no operator can act on confidently.

Where sampling is not enough, the better answer is usually to move the check rather than to weaken it. A pairwise overlap test that is unaffordable over a national coverage is affordable per tile, and running it inside the per-tile task distributes the cost across the same workers already processing the data.

What to Store, and For How Long

A quality gate produces three kinds of output, and they have different lifetimes.

Metrics — counts and rates per run — are small and should be kept indefinitely. They are the series that make trends visible, and their value grows with age. A row per run per check, with the run identifier, the value and the threshold in force at the time, is enough; the threshold column matters because it explains why an old value did not fire.

Quarantined rows are large and should have an explicit retention window. Long enough to investigate and replay — thirty days suits most pipelines — and no longer, because a dead-letter table that grows without bound eventually becomes the largest object in the warehouse and nobody dares delete it. The mechanics of the table itself are covered in quarantining invalid features to a dead-letter table.

Full failure detail — the offending geometries, the exception traces, the raw payloads — is the largest and shortest-lived. Keep a bounded sample per failure class rather than every instance: twenty examples of a validity failure teach you as much as twenty thousand, and cost four orders of magnitude less to store.

The one thing worth keeping forever, alongside the metrics, is the schema itself under version control. When someone asks in a year why a column was allowed to be null, the answer is in the diff.

Making Failures Actionable

A gate that fails with ValidationError: check failed has done half its job. The other half is telling whoever reads the alert what to do next, and that means three things in the message.

What broke, in domain terms. “Join rate against the address reference fell to 71%, expected above 98%” beats “sjoin assertion failed”. The first names the dataset relationship the reader can reason about.

How much. A count and a rate: “18 412 of 62 003 features (29.7%)”. The scale of a failure determines whether it is a quarantine or an incident, and the reader should not have to query for it.

Where to look. A handful of identifiers, and a pointer to where the rest live. Twenty parcel ids in the message and a dead-letter table name is enough to start an investigation without opening a notebook.

The same information belongs in the structured log record as fields rather than only in the human-readable string, so a later query can group failures by check, by source and by run without parsing text. That pairing — a readable message for a person, structured fields for a query — is the same pattern used throughout monitoring and observability for spatial pipelines, and it costs nothing to adopt at the point where the check already knows everything.

Related

• Writing Geometry Validity Assertions with pandera
• Detecting CRS Drift Between Pipeline Stages
• Quarantining Invalid Features to a Dead-Letter Table
• Geometry Repair with Shapely & GeoPandas
• Attribute Mapping & Schema Harmonization