Skip to content

add more examples, fix ducky v2 bindings bugs - #5

Merged
nicholasjng merged 3 commits into
masterfrom
capi-v2-bindings
Nov 21, 2026
Merged

nicholasjng merged 3 commits into
masterfrom
capi-v2-bindings

Conversation

@nicholasjng

Copy link
Copy Markdown
Owner

Also improves throughput and cuts memory usage on many workloads.

…audited defects

Loading a dataset stacked and gathered full-size copies because a result
could only be drained column by column. to_numpy_folds copies each row once,
into the array of its fold. Lists, dicts and NumPy arrays now bind as LIST and
STRUCT, so an embedding can be a query parameter.

The fixes come from an audit and were reproduced on this base first. Two
crashed or leaked (a table function whose factory returns a list, a connection
kept alive by its own UDF after execute()); the others returned wrong data
without an error, such as a retried to_numpy() after a failure, columns
dropped for duplicate names, and an aware time losing its offset.
…n one copy

Splits by row hash cannot keep a patient's rows together or keep a rare
label's share in every fold. Ranking rows within strata does both.

dataset() now reads through to_numpy_folds, which cuts load time by a third to
a half on 5M rows and, without a split, peak memory by 45%.

BREAKING CHANGE: split(seed=...) without key= now hashes the seed with the row
number. Adding it only shifted the same split by seed rows, so folds for a
given seed change.
…or JAX and MLX

The Titanic example fits in a few kilobytes and one linear layer, so it shows
little of DuckDB's share of the work on large data. These scale it up in three
directions: rows and network size, math that DuckDB reduces before the array
library sees it, and a text pipeline with vector search.
@nicholasjng
nicholasjng merged commit a65074d into master Nov 21, 2026
5 checks passed
@nicholasjng
nicholasjng deleted the capi-v2-bindings branch November 21, 2026 11:50
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant