Skip to content

feat: expose server-reported fit and predict timings - #392

Merged
ggprior merged 2 commits into
mainfrom
georg/expose-timings
Sep 15, 2026
Merged

ggprior merged 2 commits into
mainfrom
georg/expose-timings

Conversation

@ggprior

@ggprior ggprior commented Sep 15, 2026

Copy link
Copy Markdown
Contributor

Adds server-reported timings to TabPFNClassifier and TabPFNRegressor, so users can tell time spent waiting for the server to start the work apart from time spent doing it.

model.fit(X_train, y_train)
model.predict(X_test)
model.get_timings()
# {"fit": {"elapsed_s", "queue_wait_s", "train_set_transform_s", "fit_s"},
#  "predict": {"test_set_transform_queue_wait_s", "test_set_transform_s",
#              "predict_queue_wait_s", "predict_s"}}
  • The estimators keep the values as fit_timings_ (set by fit()) and last_predict_timings. get_timings() returns both.
  • PredictionResult gains timings. ServiceClient.fit_with_result() returns a FitResult with the fitted id and the timings; ServiceClient.fit() still returns only the id.
  • A fit that is still pending after /fit takes its timings from the final status poll.
  • A regressor prediction split into several requests (output_type="full" above the row limit) reports the per-stage sum.
  • api_models.py gains FitTimings, PredictTimings and the optional timings fields, copied from the API schema.

Compatibility

  • Against a server that does not report timings, every value is None; nothing else changes.
  • Predict timings live in a private attribute behind a read-only property, because check_estimator requires predict() to leave the estimator's __dict__ unchanged.
  • Tests that patched InferenceClient.fit now patch InferenceClient.fit_with_result, which the estimators call.

🤖 Generated with Claude Code

https://claude.ai/code/session_01A99qZsvdCamLPCMD77vkDw

ggprior and others added 2 commits September 15, 2026 09:32
Users could only measure the total wall time of fit() and predict(), which
does not tell waiting for capacity apart from time spent on the work. The
API now reports that split, so the estimators keep it and return it from
get_timings().

ServiceClient.fit() keeps returning the fitted id; fit_with_result() adds
the timings. Against a server that does not report timings every value is
None.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01A99qZsvdCamLPCMD77vkDw
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01A99qZsvdCamLPCMD77vkDw
@ggprior
ggprior marked this pull request as ready for review September 15, 2026 09:34
@ggprior
ggprior requested a review from a team as a code owner September 15, 2026 09:34
@ggprior
ggprior requested a review from simo-prior September 15, 2026 09:34
@ggprior
ggprior added this pull request to the merge queue Sep 15, 2026
Merged via the queue into main with commit 5d986e8 Sep 15, 2026
11 checks passed
@ggprior
ggprior deleted the georg/expose-timings branch September 15, 2026 09:38
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants