This guide is part of Validating Spatial Data Quality in Pipelines, within the broader Automated Vector & Raster Cleaning Workflows reference.

Writing Geometry Validity Assertions with pandera

pandera validates attribute columns well and knows nothing about geometry. Bridging that gap takes a handful of vectorized predicates and one decision about how failures are surfaced.

Why a Geometry Check Belongs in the Schema

  • One place to read the contract. A reviewer opening the schema sees the attribute rules and the geometry rules together, rather than finding half of them in a validation module nobody remembers.
  • One failure report. Lazy validation returns a single frame of failure cases covering every check, which is both the alert payload and the quarantine input.
  • Cheap to run everywhere. The same schema validates a batch in the pipeline, a fixture in a unit test, and a sample in a notebook, with no branching.
  • Diffable. A change to what counts as acceptable geometry shows up in a pull request instead of in behaviour.

Version and Environment Compatibility

Component Version Note
pandera >=0.20 pandera.pandas namespace, lazy=True failure collection
GeoPandas >=1.0 Geometry column survives validate unchanged
Shapely >=2.0 Array predicates: is_valid, is_empty, get_num_coordinates
GEOS >=3.10 Consistent validity semantics across platforms 
pip install "pandera>=0.20" "geopandas>=1.0" "shapely>=2.0"

How the Checks Compose

From predicates to one failure-cases frame A geometry array feeds five vectorized predicates: validity, non-emptiness, allowed geometry type, vertex budget and bounds containment. Each returns a boolean mask. Pandera gathers the failing positions from every mask into a single failure-cases frame carrying the check name, the index and the offending value, which is then used both for the alert message and for routing rows to quarantine. geometry array N features is_valid not is_empty geom_type in allowed vertices <= budget within the CRS extent failure_cases check · index · failure_case one frame, every violation

The Schema Recipe

from __future__ import annotations

import geopandas as gpd
import numpy as np
import pandas as pd
import pandera.pandas as pa
import shapely
from pandera import Check, Column, DataFrameSchema

ALLOWED_TYPES = ("Polygon", "MultiPolygon")
MAX_VERTICES = 250_000
WGS84_BOUNDS = (-180.0, -90.0, 180.0, 90.0)


def _geoms(series: pd.Series) -> np.ndarray:
    """Underlying Shapely array for a geometry column, whatever its container."""
    return gpd.GeoSeries(series).values


def geometry_is_valid(series: pd.Series) -> pd.Series:
    return pd.Series(shapely.is_valid(_geoms(series)), index=series.index)


def geometry_is_not_empty(series: pd.Series) -> pd.Series:
    geoms = _geoms(series)
    ok = ~(shapely.is_empty(geoms) | shapely.is_missing(geoms))
    return pd.Series(ok, index=series.index)


def geometry_type_allowed(series: pd.Series) -> pd.Series:
    types = gpd.GeoSeries(series).geom_type
    return types.isin(ALLOWED_TYPES)


def vertex_budget(series: pd.Series) -> pd.Series:
    counts = shapely.get_num_coordinates(_geoms(series))
    return pd.Series(counts <= MAX_VERTICES, index=series.index)


def within_crs_extent(series: pd.Series) -> pd.Series:
    bounds = gpd.GeoSeries(series).bounds
    minx, miny, maxx, maxy = WGS84_BOUNDS
    ok = (
        (bounds["minx"] >= minx) & (bounds["maxx"] <= maxx)
        & (bounds["miny"] >= miny) & (bounds["maxy"] <= maxy)
    )
    return ok.fillna(False)


PARCEL_SCHEMA = DataFrameSchema(
    {
        "parcel_id": Column(str, nullable=False, unique=True),
        "area_sqm": Column(float, Check.gt(0), nullable=False),
        "geometry": Column(
            object,
            checks=[
                Check(geometry_is_valid, name="geometry_valid",
                      error="geometry fails GEOS validity"),
                Check(geometry_is_not_empty, name="geometry_present",
                      error="geometry is empty or missing"),
                Check(geometry_type_allowed, name="geometry_type",
                      error=f"geometry type not in {ALLOWED_TYPES}"),
                Check(vertex_budget, name="vertex_budget",
                      error=f"geometry exceeds {MAX_VERTICES} vertices"),
                Check(within_crs_extent, name="within_extent",
                      error="geometry falls outside the CRS area of use"),
            ],
            nullable=False,
        ),
    },
    strict=False,
    name="parcels",
)

Collecting Every Failure in One Pass

import logging

logger = logging.getLogger(__name__)


def validate_parcels(gdf: gpd.GeoDataFrame, id_column: str = "parcel_id"):
    """Validate lazily and return (clean_frame, failures_frame).

    Lazy validation means one pass reports every violating row for every check,
    which is what the quarantine path and the alert both need.
    """
    try:
        PARCEL_SCHEMA.validate(gdf, lazy=True)
        return gdf, gdf.iloc[0:0]
    except pa.errors.SchemaErrors as exc:
        cases = exc.failure_cases
        failing_index = cases["index"].dropna().unique()

        summary = cases.groupby("check")["index"].nunique().to_dict()
        logger.warning("validation failures by check: %s", summary)

        failures = gdf.loc[gdf.index.isin(failing_index)].copy()
        failures["failed_checks"] = (
            cases.dropna(subset=["index"])
            .groupby("index")["check"]
            .agg(lambda names: ",".join(sorted(set(names))))
            .reindex(failures.index)
        )
        clean = gdf.loc[~gdf.index.isin(failing_index)]
        logger.info("%d clean, %d quarantined of %d", len(clean), len(failures), len(gdf))
        return clean, failures

Key Implementation Notes

  • Every predicate returns a boolean Series aligned to the input index. Pandera uses that index to report failure positions, and an unaligned result produces a confusing report rather than an error.
  • name= on each Check is what appears in failure_cases. Without it the report shows a lambda repr, which is unusable in an alert.
  • within_crs_extent guards axis-order accidents. A latitude in a longitude column pushes the bounds outside the valid range, so a transposed layer fails here rather than in a join three stages later.
  • The vertex budget is a practical guard, not a correctness one. A single polygon with two million vertices is legal and will destroy the memory profile of every downstream operation; catching it at the boundary is cheaper than discovering it during a spatial join.
  • fillna(False) on the bounds check matters because a missing geometry produces NaN bounds, and NaN comparisons yield False for the range test but propagate as NA without it.
  • The clean and failing frames are returned separately rather than raising, so the caller decides between quarantine and abort using the severity table.
Fail-fast versus lazy validation on the same batch Two validation runs over the same sixty thousand row batch. The fail-fast run raises on row 84 and reports a single failure, so the next run finds the next failure and the cycle repeats. The lazy run completes every check and reports four hundred and twelve failing rows grouped into three checks, which is one investigation instead of many. fail fast raises at row 84 1 failure known · the rest still hidden fix, re-run, find the next one, repeat until the batch happens to pass lazy 412 rows across 3 checks geometry_valid 380 · vertex_budget 4 · type 28 one report, one quarantine write, one investigation with the shape visible

Troubleshooting Schema Failures

Symptom Likely cause Fix
SchemaError on the geometry dtype Column declared as a geometry dtype rather than object Declare Column(object, ...); the checks carry the semantics
Failure cases have no index Predicate returned a bare array instead of a Series Wrap the result with the input index, as the recipe does
Check name shows as a lambda name= omitted on the Check Name every check — it is the alert’s only identifier
Validation is slow Predicate uses .apply internally Route through the Shapely array API
Every row fails within_extent Frame is in a projected CRS, bounds are metres Parameterise the bounds by CRS instead of hard-coding WGS 84
One schema, three callers A single schema definition is used by three callers. The pipeline validates each nightly batch. The test suite validates fixtures, including deliberately broken ones, so the checks themselves are tested. An analyst validates an ad-hoc extract in a notebook. Because all three share one definition, a change to the contract cannot be applied in one place and forgotten in the others. PARCEL_SCHEMA one definition, version controlled the pipeline every nightly batch the test suite broken fixtures, on purpose a notebook ad-hoc extract, same rules

Integration Note

The clean frame continues to the load; the failures frame goes to the dead-letter path described in quarantining invalid features to a dead-letter table, carrying its failed_checks column so a replay can filter by failure class. Repair belongs downstream of the quarantine decision rather than inside the schema — the reasoning is in geometry repair with Shapely and GeoPandas, and the short version is that a gate which silently fixes data stops being a gate.

Testing the Checks Themselves

A validation schema is code, and code that has never failed in a test has never been proven to work. The cheapest insurance is a small directory of deliberately broken fixtures, one per check, asserted to fail with exactly the expected check name.

Build them from real geometry rather than from synthetic squares: a bowtie taken from a genuine parcel export, an empty geometry produced by a real repair, a multipolygon that arrived where a polygon was expected. Fixtures drawn from production carry the shapes that actually occur, and they double as regression tests when a library upgrade changes GEOS behaviour.

The positive case matters too. One fixture of known-good data, asserted to pass cleanly, catches the day someone tightens a threshold far enough to reject everything — a failure mode that is otherwise invisible until the nightly run quarantines an entire batch.

Parameterising the Schema per Layer

One hard-coded schema serves one layer. A pipeline that ingests parcels, buildings and road centrelines needs three, and copying the file three times guarantees they diverge.

Build the schema from a small configuration instead: allowed geometry types, the target EPSG code and its bounds, the vertex budget, and the attribute columns with their ranges. A factory function that takes that configuration and returns a DataFrameSchema keeps every layer’s rules in data while the check implementations stay in one place, tested once.

The configuration is also the natural home for the severity mapping, so a reviewer reading one file sees both what is required of a layer and what happens when it is not met. Changing a threshold then becomes a configuration diff rather than a code change, which is the difference between a decision that gets reviewed and one that gets merged unnoticed.