Skip to content

The Laurens deficit is the rig, not the town: add the GSV arm and score all 11 challengers on both (#151) #153

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
jonfroehlich merged 5 commits into from
Sep 4, 2026
Merged

The Laurens deficit is the rig, not the town: add the GSV arm and score all 11 challengers on both (#151) #153

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
5 commits
Select commit Hold shift + click to select a range
8d61982
Rename laurens -> laurens_mapillary; it is now one of two arms
jonfroehlich Aug 31, 2026
62cc220
Add the Laurens GSV arm bundle -- the discriminator for #151
jonfroehlich Aug 31, 2026
e9dfd86
Laurens GSV arm GT: recall 0.505 -- #151 splits the difference
jonfroehlich Aug 31, 2026
5111834
Score all 11 challengers on laurens_mapillary; the AP test was a popu…
jonfroehlich Sep 1, 2026
d388f17
Register laurens_gsv as a held-out second arm: the Laurens deficit is…
jonfroehlich Sep 1, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1,257 changes: 1,257 additions & 0 deletions analysis_out/crop_window_eval_annapolis.json
View file
Open in desktop

Large diffs are not rendered by default.

1,261 changes: 1,261 additions & 0 deletions analysis_out/crop_window_eval_paterson.json
View file
Open in desktop

Large diffs are not rendered by default.

1,181 changes: 1,181 additions & 0 deletions analysis_out/crop_window_eval_sao_paulo.json
View file
Open in desktop

Large diffs are not rendered by default.

38 changes: 19 additions & 19 deletions analysis_out/op/confidence_calibration.csv
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -132,25 +132,25 @@ gainesville,0.8,0.85,21,2,0.9130434782608695,0.7320359570190598,0.97582050459276
gainesville,0.85,0.9,34,0,1.0,0.8984820937803899,1.0
gainesville,0.9,0.95,29,0,1.0,0.8830264055344442,1.0
gainesville,0.95,1.0,18,0,1.0,0.8241154494176252,1.0
laurens,0.05,0.1,22,85,0.205607476635514,0.13988321742546095,0.2917381232985993
laurens,0.1,0.15,11,33,0.25,0.14574066080685744,0.39440849808414946
laurens,0.15,0.2,10,17,0.37037037037037035,0.21532287067936856,0.5577109537655369
laurens,0.2,0.25,9,8,0.5294117647058824,0.30962897311549487,0.738351988039119
laurens,0.25,0.3,12,5,0.7058823529411765,0.468664666737861,0.8672020613444361
laurens,0.3,0.35,7,2,0.7777777777777778,0.4525833436728054,0.9367762376877727
laurens,0.35,0.4,10,1,0.9090909090909091,0.6226353745137962,0.9837682477371741
laurens,0.4,0.45,10,2,0.8333333333333334,0.5519636426153274,0.9530358523851776
laurens,0.45,0.5,6,0,1.0,0.6096569663469354,0.9999999999999999
laurens,0.5,0.55,1,1,0.5,0.09452865480086614,0.9054713451991339
laurens,0.55,0.6,9,3,0.75,0.46768966087934005,0.9110599603710386
laurens,0.6,0.65,11,2,0.8461538461538461,0.5776482949136899,0.9567427368172621
laurens,0.65,0.7,13,3,0.8125,0.5699064013154177,0.9340852122641324
laurens,0.7,0.75,11,2,0.8461538461538461,0.5776482949136899,0.9567427368172621
laurens,0.75,0.8,9,0,1.0,0.7008472464490407,1.0
laurens,0.8,0.85,20,0,1.0,0.8388698745050667,1.0
laurens,0.85,0.9,14,1,0.9333333333333333,0.7018292244757601,0.9881334113938051
laurens,0.9,0.95,10,0,1.0,0.7224598312333834,1.0
laurens,0.95,1.0,0,0,,,
laurens_mapillary,0.05,0.1,22,85,0.205607476635514,0.13988321742546095,0.2917381232985993
laurens_mapillary,0.1,0.15,11,33,0.25,0.14574066080685744,0.39440849808414946
laurens_mapillary,0.15,0.2,10,17,0.37037037037037035,0.21532287067936856,0.5577109537655369
laurens_mapillary,0.2,0.25,9,8,0.5294117647058824,0.30962897311549487,0.738351988039119
laurens_mapillary,0.25,0.3,12,5,0.7058823529411765,0.468664666737861,0.8672020613444361
laurens_mapillary,0.3,0.35,7,2,0.7777777777777778,0.4525833436728054,0.9367762376877727
laurens_mapillary,0.35,0.4,10,1,0.9090909090909091,0.6226353745137962,0.9837682477371741
laurens_mapillary,0.4,0.45,10,2,0.8333333333333334,0.5519636426153274,0.9530358523851776
laurens_mapillary,0.45,0.5,6,0,1.0,0.6096569663469354,0.9999999999999999
laurens_mapillary,0.5,0.55,1,1,0.5,0.09452865480086614,0.9054713451991339
laurens_mapillary,0.55,0.6,9,3,0.75,0.46768966087934005,0.9110599603710386
laurens_mapillary,0.6,0.65,11,2,0.8461538461538461,0.5776482949136899,0.9567427368172621
laurens_mapillary,0.65,0.7,13,3,0.8125,0.5699064013154177,0.9340852122641324
laurens_mapillary,0.7,0.75,11,2,0.8461538461538461,0.5776482949136899,0.9567427368172621
laurens_mapillary,0.75,0.8,9,0,1.0,0.7008472464490407,1.0
laurens_mapillary,0.8,0.85,20,0,1.0,0.8388698745050667,1.0
laurens_mapillary,0.85,0.9,14,1,0.9333333333333333,0.7018292244757601,0.9881334113938051
laurens_mapillary,0.9,0.95,10,0,1.0,0.7224598312333834,1.0
laurens_mapillary,0.95,1.0,0,0,,,
budapest_district5,0.05,0.1,7,107,0.06140350877192982,0.030058899274456374,0.12134434032853611
budapest_district5,0.1,0.15,7,52,0.11864406779661017,0.058675082779140006,0.22523875773415053
budapest_district5,0.15,0.2,11,39,0.22,0.12753786224302288,0.35241815689459555
Expand Down
4 changes: 2 additions & 2 deletions analysis_out/op/confidence_calibration.json
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -1632,7 +1632,7 @@
]
}
],
"laurens": [
"laurens_mapillary": [
{
"lo": 0.05,
"hi": 0.1,
Expand Down Expand Up @@ -2562,7 +2562,7 @@
"annapolis",
"paterson",
"gainesville",
"laurens"
"laurens_mapillary"
],
"bins": [
{
Expand Down
84 changes: 42 additions & 42 deletions analysis_out/op/low_floor_sweep.csv
View file
Open in desktop

Large diffs are not rendered by default.

6 changes: 3 additions & 3 deletions analysis_out/op/recall_by_distance.csv
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -20,9 +20,9 @@ paterson,far (>25 m),88,0.4318181818181818,0.5227272727272727,8
gainesville,near (<12.5 m),99,0.7777777777777778,0.8383838383838383,6
gainesville,mid (12.5-25 m),123,0.7398373983739838,0.8617886178861789,15
gainesville,far (>25 m),50,0.3,0.42,6
laurens,near (<12.5 m),180,0.4388888888888889,0.5722222222222222,24
laurens,mid (12.5-25 m),60,0.3,0.4666666666666667,10
laurens,far (>25 m),9,0.0,0.0,0
laurens_mapillary,near (<12.5 m),180,0.4388888888888889,0.5722222222222222,24
laurens_mapillary,mid (12.5-25 m),60,0.3,0.4666666666666667,10
laurens_mapillary,far (>25 m),9,0.0,0.0,0
budapest_district5,near (<12.5 m),150,0.58,0.6933333333333334,17
budapest_district5,mid (12.5-25 m),88,0.4431818181818182,0.6363636363636364,17
budapest_district5,far (>25 m),50,0.48,0.58,5
Expand Down
2 changes: 1 addition & 1 deletion analysis_out/op/storage_floor.csv
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -6,7 +6,7 @@ morgantown,267,7,12,0.8876404494382022,0.8614232209737828,0.8164794007490637,0.7
annapolis,294,9,10,0.9013605442176871,0.8707482993197279,0.8367346938775511,0.7380952380952381,265,256,246,217
paterson,395,1,9,0.7569620253164557,0.7544303797468355,0.7316455696202532,0.6810126582278481,299,298,289,269
gainesville,272,6,14,0.8897058823529411,0.8676470588235294,0.8161764705882353,0.6727941176470589,242,236,222,183
laurens,249,22,21,0.7831325301204819,0.6947791164658634,0.6104417670682731,0.3895582329317269,195,173,152,97
laurens_mapillary,249,22,21,0.7831325301204819,0.6947791164658634,0.6104417670682731,0.3895582329317269,195,173,152,97
budapest_district5,300,7,18,0.7566666666666667,0.7333333333333333,0.6733333333333333,0.51,227,220,202,153
sao_paulo,281,2,10,0.8683274021352313,0.8612099644128114,0.8256227758007118,0.6512455516014235,244,242,232,183
manual_gold,3919,19,48,0.920387854044399,0.9155396784894105,0.9032916560347027,0.8494513906608829,3607,3588,3540,3329
Expand Down
20 changes: 10 additions & 10 deletions analysis_out/op/tp_origin_by_bin.csv
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -69,16 +69,16 @@ gainesville,0.6,31,0
gainesville,0.7,35,2
gainesville,0.8,55,0
gainesville,0.9,47,0
laurens,0.0,0,22
laurens,0.1,0,21
laurens,0.2,0,21
laurens,0.3,0,17
laurens,0.4,0,16
laurens,0.5,9,1
laurens,0.6,24,0
laurens,0.7,20,0
laurens,0.8,34,0
laurens,0.9,10,0
laurens_mapillary,0.0,0,22
laurens_mapillary,0.1,0,21
laurens_mapillary,0.2,0,21
laurens_mapillary,0.3,0,17
laurens_mapillary,0.4,0,16
laurens_mapillary,0.5,9,1
laurens_mapillary,0.6,24,0
laurens_mapillary,0.7,20,0
laurens_mapillary,0.8,34,0
laurens_mapillary,0.9,10,0
budapest_district5,0.0,0,7
budapest_district5,0.1,0,18
budapest_district5,0.2,0,9
Expand Down
1 change: 0 additions & 1 deletion analysis_out/op_cache/laurens.json
View file
Open in desktop

This file was deleted.

1 change: 1 addition & 0 deletions analysis_out/op_cache/laurens_mapillary.json
View file
Open in desktop

Large diffs are not rendered by default.

Loading
Loading