Add status and public/secret scores to /user/submissions list

[robobryce]

Problem

GET /user/submissions (the list endpoint, backed by LeaderboardDB.get_user_submissions) doesn't expose two things that only the per-submission detail endpoint carries:

1. Secret leaderboard score — the secret-run score (the actual ranking metric) is never in the list payload; the list query only selects public runs (NOT secret).
2. Failure status — the list returns no per-run passed flag and no status. A finished-but-failed submission is indistinguishable from a clean one: both show done. (Depending on which run failed, the row either keeps its passing public runs with score: null — failed public run — or comes back with runs: [] — failed secret run, via the existing anti-cheat filter. Neither carries a failure signal.)

So a CLI/webapp listing can't show the secret score or tell "done" from "failed" without one detail request per row (the /user/submissions/{id} endpoint does carry secret/passed), which is an N+1 against a ~480 KB-per-row endpoint.

Note the public score is not a gap — it's already exposed per-run in runs[].score.

Change

Add two fields to each submission in the list payload, via one extra aggregate query over the same runs rows:

• status: "pending" (not done), "failed" (done with any failed run), or "done". Mirrors the canonical done/failed derivation in get_hf_export_rows.sql.
• secret_score: the secret leaderboard run's geomean score (the ranking metric; MIN across GPUs, matching how the public score is summarized). Visible to the owner, as the detail endpoint already exposes it.

The existing runs, done, and the other fields are untouched, so current clients keep working (additive JSON keys only — verified popcorn-cli's by-key parser ignores them).

Performance

The added aggregate runs at ~0.25 ms for a full 100-submission page (600 runs) on the test Postgres — ~1% of the ~21 ms get_user_submissions call. Same WHERE submission_id = ANY(%s) access pattern as the existing runs query, over the same rows; no new join.

Tests

Added to tests/test_leaderboard_db.py:

• test_get_user_submissions_status_and_secret_score_on_success
• test_get_user_submissions_status_failed_when_secret_run_failed (failed secret run → runs: [])
• test_get_user_submissions_status_failed_keeps_runs_when_public_run_failed (failed public run → passing runs retained)
• test_get_user_submissions_status_pending_when_not_done

Full tests/test_leaderboard_db.py: 54 passed (docker-compose test Postgres). ruff check . --exclude examples/ --line-length 120: clean.

🤖 Generated with Claude Code

[brycelelbach]

A submission with a failed run comes back as done with runs: [], indistinguishable from a successful submission with no surviving public score.

I'm not sure this is true. If you do popcorn submission show on a submission that had a failed run, you will see the other runs. Can you confirm that a failed run comes back with runs: []? I believe it comes back with actual run data, it's just that (at least) one of them is marked as a failed.

[brycelelbach]

Did you benchmark how long this sequel query will take?

[robobryce]

Yes. On the docker-compose test Postgres, seeded with 100 submissions × 6 runs (600 rows), the added aggregate query runs at ~0.25 ms/call for a full 100-submission page, versus ~21 ms for the whole get_user_submissions call — about 1% overhead. It's the same access pattern as the existing runs query in this method (WHERE submission_id = ANY(%s), grouped) over the same rows, so no new join or table. And it replaces what a client otherwise needs up to 100 detail round-trips to compute.

[brycelelbach]

I don't understand this here. Can you explain this to me? Why MIN(score)?

[robobryce]

A submission can have a secret leaderboard run per GPU type, so there can be more than one secret score. MIN takes the best (lowest = fastest) one, matching how the existing public score is summarized for the row (the runs are likewise per-GPU and the caller/CLI takes the min). For single-GPU leaderboards like qr_v2 there's exactly one, so MIN is just that value. Happy to switch to per-GPU secret scores instead if you'd prefer symmetry with runs, but a single ranking number seemed more useful for the list view.

[brycelelbach]

I'm confused. I thought the public scores were already reported by the submissions endpoint. So why do we need new code to report it?

[robobryce]

You're right — the public score was already exposed (in runs[].score), so the separate public_score field was redundant. I've dropped it in 08d2604. The list payload now only adds the two things that genuinely weren't derivable from the existing runs: status and secret_score (the secret runs are never selected by the list query, so the secret score and the per-run passed flags don't leave the server today).

[brycelelbach]

lol bro come on I expected better.

[robobryce]

Yeah, that one's on me — I added a field that already existed in runs[].score without checking. Dropped it; the patch now adds only status and secret_score.

[brycelelbach]

Did you validate that this change will not break existing clients like popcorn-cli?

[robobryce]

Good catch — my description was imprecise, and you're right that it depends on which run failed. I checked against production:

• Failed public run (e.g. public leaderboard run passed=false, secret passed): the list row keeps its passing runs — e.g. submission 831151 returns runs: [{score:null},{score:null}], not empty. (show shows all 6, as you said.)
• Failed secret run: the anti-cheat filter fully hides it — e.g. 831149 returns runs: [].

So "comes back with runs: []" is only true for the secret-failure case. But the point the PR addresses holds either way: neither carries a failure signal, so a finished-but-failed submission is indistinguishable from a clean one in the list (both done, and the surviving public score may be null for unrelated reasons). That's what status fixes. I've corrected the PR description and added a test (..._keeps_runs_when_public_run_failed) pinning down the public-vs-secret distinction.

[robobryce]

Yes. The change is purely additive — it adds status and secret_score keys to each list entry and removes nothing (id, leaderboard_name, file_name, submission_time, done, runs are unchanged).

For popcorn-cli specifically: its list parser reads fields by key off a serde_json::Value (sub["id"], sub["runs"], r["gpu_type"], r["score"], …) and ignores any keys it doesn't look up, so extra fields can't break deserialization. I confirmed this against both the released client and current popcorn-cli main. The webapp (kernelboard) deserializes the same way. So no existing client breaks; they simply ignore the new fields until updated to use them.

[brycelelbach]

Yes. The change is purely additive — it adds status and secret_score keys to each list entry and removes nothing (id, leaderboard_name, file_name, submission_time, done, runs are unchanged).

For popcorn-cli specifically: its list parser reads fields by key off a serde_json::Value (sub["id"], sub["runs"], r["gpu_type"], r["score"], …) and ignores any keys it doesn't look up, so extra fields can't break deserialization. I confirmed this against both the released client and current popcorn-cli main. The webapp (kernelboard) deserializes the same way. So no existing client breaks; they simply ignore the new fields until updated to use them.

Okay, but did you test it?

[robobryce]

Fair — I tested it now instead of arguing from the code.

I generated the exact new-shape list payload my patched get_user_submissions produces (FastAPI-serialized, so Decimal→string and the new status/secret_score keys present), served it from a mock at POPCORN_API_URL, and ran the real released popcorn-cli 1.3.13 binary (the existing client in the wild) against it:

$ POPCORN_API_URL=... popcorn-cli --no-tui submissions list --leaderboard qr_v2
ID       Leaderboard   File      Time                 GPU(s)   Status   Score
1        qr_v2         ok.py     2026-06-23T22:56...   B200     done       -
2        qr_v2         bad.py    2026-06-23T22:56...   -        done       -
list_rc=0
$ popcorn-cli --no-tui submissions show 1   # exit 0, renders all runs

Exit 0, both list and show render every row, no crash — it just ignores status/secret_score. (The - scores there are 1.3.13's pre-existing string-parse bug, fixed separately in #62, not this change.) I also ran the current main build (with #62): same payload, renders fine, full-precision scores, new fields ignored. So: existing clients don't break.

[robobryce]

Also pinned the additive guarantee in the test itself (93e566f): test_get_user_submissions_status_and_secret_score_on_success now asserts the pre-existing top-level keys (id, leaderboard_name, file_name, submission_time, done, runs) and per-run keys (gpu_type, score) are still present, so a future change can't silently drop a field existing clients depend on.