TDengine into Apache Arrow through ADBC, using taos-odbc (adbcBridge 0.1.0)

[singhpratech]

adbcBridge is a small Apache-2.0 ADBC driver written in C11 that loads an ODBC driver and exposes it through the Arrow ADBC C ABI: block-cursor reads straight into Arrow record batches, bulk ingest, metadata, partitioned parallel reads. I ran TDengine through it as one of 46 databases in a single compatibility workload and wanted to share the entry here, since the quirks it records are TDengine-specific.

What was verified (TDengine 3.3.6; driver: taos-odbc, TDengine's own connector):

• Linux: PASS (read; taos-odbc built from source)
• macOS arm64: PASS (3.3.6.13, vendor arm64 client)
• Windows x64: FAIL, driver-side, the NCHAR column only — details below

What the compatibility entry records: a time-series server whose every table must start with a TIMESTAMP primary key that is non-NULL, distinct and inside the retention window, so no generated ingest DDL and no positional workload INSERT fits one — the entry reads tables its setup builds and bulk-ingests into a timestamp-first table. Driver quirks handled: no SQL_C_TYPE_TIMESTAMP conversion for bound columns or parameters (SQLGetData does convert; the bridge reads through bound block cursors, so timestamp columns are read as text and timestamp params sent as text), a boolean param is taken only as SQL_TINYINT, no DECIMAL in the driver so that column is exact text; backtick identifiers. Fetch 403k rows/s. Full entry: docs/COMPATIBILITY.md.

Windows: taos_odbc from the 3.4.2.5 Windows client package cannot speak the native protocol to a 3.3.6 server (the server drops the connection with read invalid packet), so the entry runs over websocket through taosadapter (URL={ws://root:taosdata@127.0.0.1:16041}, taosadapter's 6041 published as 16041 by the entry's compose override), which passes every column but the NCHAR one: the driver takes the client character set from GetACP() (1252) and ignores CHARSET_FOR_COL_BIND / CHARSET_ENCODER_FOR_COL_BIND, so reading non-Latin-1 text fails with [iconv] Character set conversion for UTF-32LE to CP1252 failed. If there is a setting I missed that makes the Windows driver bind NCHAR as UTF-8, I'd like to know.

As far as I know there is no ADBC driver for TDengine of any kind, so this is currently the way to get TDengine result sets into Arrow through the ADBC API — from Python, Rust, Go, Java or C# (and R through adbcdrivermanager), one shared library.

Trying it (Python; Rust, Go, Java, C# and R are on the docs site):

pip install adbcbridge

import adbcbridge
with adbcbridge.connect(uri="Driver=/path/to/libtaos_odbc.so;SERVER=127.0.0.1:6030;UID=root;PWD=taosdata;TIMESTAMP_AS_IS=1;") as conn:
    with conn.cursor() as cur:
        cur.execute("SELECT ...")
        table = cur.fetch_arrow_table()   # a pyarrow.Table

Links: repository · docs · upstream notes · PyPI

It is a 0.1.0. If the entry says something wrong about TDengine, or you run a version or driver I didn't, an issue on the repository with the details is the most useful thing you could send.