Skip to content

Chunked round-trip over DuckDB relations: intermittent deadlock in duckdb-python under thread churn #241

Description

@Mmoncadaisla

DuckDBHandle.supports_chunked = False disables the lazy (chunks=) round-trip for DuckDB relations. The blocker is upstream: a DuckDB query whose source is a Python-callback Arrow scan (any table registered through xarray-sql) intermittently deadlocks inside duckdb-python/CPython when other Python threads start or stop during execution.

Current state: chunked DuckDB round-trips fail fast instead of hanging; the eager path and every other handle operation run on one dedicated engine thread (the known-safe topology) and are unaffected. See DuckDBHandle in xarray_sql/lazyscan.py.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions