diff --git a/datafusion/session/src/table.rs b/datafusion/session/src/table.rs index a6cddd10ce80a..cd3e2c980869a 100644 --- a/datafusion/session/src/table.rs +++ b/datafusion/session/src/table.rs @@ -359,8 +359,38 @@ pub trait TableProvider: Any + Debug + Sync + Send { /// Delete rows matching the filter predicates. /// - /// Returns an [`ExecutionPlan`] producing a single row with `count` (UInt64). - /// Empty `filters` deletes all rows. + /// Returns an [`ExecutionPlan`] producing one row in a single non-null + /// `UInt64` column named `count`, containing the number of deleted rows. + /// The default implementation returns a "not implemented" error. + /// + /// # Filters + /// + /// `filters` contains logical [`Expr`] predicates on the target table. + /// The planner collects them from filters and pushed-down table scan + /// predicates, splits `AND` conjunctions, removes table qualifiers, and + /// deduplicates them. For example, `t.id = 1 AND t.value > 15` becomes + /// separate `id = 1` and `value > 15` expressions. + /// + /// Delete a row only when every predicate evaluates to true. SQL + /// three-valued logic applies: a false or `NULL` predicate leaves the row + /// unchanged. Empty `filters` deletes all rows, as for `DELETE FROM t` + /// without a `WHERE` clause. + /// + /// # Execution + /// + /// This method is called during physical planning, including for `EXPLAIN`. + /// Perform mutations when the returned plan executes, rather than while + /// constructing it, so planning does not change the table. + /// + /// # Limitations + /// + /// The method receives no row limit ([#24998]). Subqueries in DML + /// expressions are not fully supported ([#24654]); in particular, a + /// subquery rewritten into a join can cause predicates to be lost before + /// this method is called. + /// + /// [#24998]: https://github.com/apache/datafusion/issues/24998 + /// [#24654]: https://github.com/apache/datafusion/issues/24654 // Hand-written `#[async_trait]` expansion to reduce compile time. See // fn delete_from<'life0, 'life1, 'async_trait>( @@ -381,8 +411,43 @@ pub trait TableProvider: Any + Debug + Sync + Send { /// Update rows matching the filter predicates. /// - /// Returns an [`ExecutionPlan`] producing a single row with `count` (UInt64). - /// Empty `filters` updates all rows. + /// Returns an [`ExecutionPlan`] producing one row in a single non-null + /// `UInt64` column named `count`, containing the number of affected rows. + /// The default implementation returns a "not implemented" error. + /// + /// # Filters + /// + /// `filters` follows the same conventions as [`Self::delete_from`]: + /// predicates are combined with `AND`, table qualifiers are removed, and + /// only rows for which every predicate is true are updated. A false or + /// `NULL` predicate leaves the row unchanged. Empty `filters` updates all + /// rows. + /// + /// # Assignments + /// + /// `assignments` contains `(column_name, Expr)` pairs from the `SET` + /// clause. The planner removes identity assignments and strips table + /// qualifiers from the expressions. Leave columns without an assignment + /// unchanged. + /// + /// Evaluate every assignment against the row values from before the + /// statement, and only for matching rows. For example, `SET a = b, b = a` + /// exchanges the two values; an expression such as `100 / divisor` must + /// not be evaluated on rows excluded by the filters. + /// + /// # Execution + /// + /// Like [`Self::delete_from`], this method is called during physical + /// planning, including for `EXPLAIN`. Perform mutations when the returned + /// plan executes so planning does not change the table. + /// + /// # Limitations + /// + /// Subqueries have the same limitations as in [`Self::delete_from`] + /// ([#24654]). `UPDATE ... FROM` is not supported ([#19950]). + /// + /// [#24654]: https://github.com/apache/datafusion/issues/24654 + /// [#19950]: https://github.com/apache/datafusion/issues/19950 // Hand-written `#[async_trait]` expansion to reduce compile time. See // fn update<'life0, 'life1, 'async_trait>( diff --git a/docs/source/library-user-guide/custom-table-providers.md b/docs/source/library-user-guide/custom-table-providers.md index dbbdf9d022716..30ffb2b5cf2f7 100644 --- a/docs/source/library-user-guide/custom-table-providers.md +++ b/docs/source/library-user-guide/custom-table-providers.md @@ -792,6 +792,19 @@ that a `FilterExec` is unnecessary for the `date` predicate, and the second ensures that only the relevant directories are scanned. The actual file reading happens later, in the stream produced by `execute()`. +## Row-Level DML: DELETE and UPDATE + +A custom table provider can support `DELETE` and `UPDATE` by implementing the optional [`TableProvider::delete_from()`] and [`TableProvider::update()`] methods. Their default implementations return a "not implemented" error. + +Each method builds an [ExecutionPlan] that changes the matching rows and returns the number of affected rows in a `count` column. Your provider decides how to apply and persist the changes. Perform the changes when the returned plan executes: these methods run during physical planning, including for `EXPLAIN`. + +The method documentation describes the filters, assignments, SQL semantics, and result schema that an implementation must support. [MemTable] provides an implementation of both methods over in-memory batches. + +For SQL syntax and examples, see the [DML section](../user-guide/sql/dml.md) of the user guide. + +[`tableprovider::delete_from()`]: https://docs.rs/datafusion/latest/datafusion/catalog/trait.TableProvider.html#method.delete_from +[`tableprovider::update()`]: https://docs.rs/datafusion/latest/datafusion/catalog/trait.TableProvider.html#method.update + ## Putting It All Together Here is a minimal but complete example of a custom table provider that generates