Repository navigation
Commit 99532f8
tracker: count people, not beacons — visitor rollup + scripted cap (#263)
"Human visits" was every beacon from a non-crawler user agent: the page
view plus four scroll depths, every click and every form submit, three to
four per page view. On four properties it read 51,531 in a week while the
raw event table held 420 distinct visitor ids in a day, which is also what
datafa.st reported for the same sites. Nothing outside the 24h raw table
kept a visitor id, so weekly uniques could not be stated at all.
- tracker_visitor_daily_stats: one row per (project, UTC day, visitor id)
with event and pageview counts and which side of the human/bot line the
visitor ended the day on. tracker_touch_visitor upserts it per beacon in
one round trip and applies the scripted cap: more than 500 events or 200
page views from one visitor in a day flips it to bot for the day, and the
ingest route counts every later beacon from it under bot:scripted. A
stock-UA headless browser was the largest "human" source and nothing in
the classifier could see it.
- tracker_visitor_totals / tracker_visitor_daily_series: exact distinct
visitors over a window (and the window before) and per day.
- Stats page, /dashboard cards and /dashboard/analytics lead with "Human
visitors" and "Page views" from the rollup; the old figure stays as
"Human events" with a definition that says what it is. A failed rollup
read leaves the tile out rather than showing 0.
- /api/tracker/v1/stats totals: visitors and pageviews come from the
rollup (null when unreadable, never 0), events is the old number. The
CLI line prints all three. totalsFromSeries no longer returns beacons as
visitors, so `crawlproof dashboard` cost-per-visitor is finally per
visitor.
Migration applied to production 2026-09-21 via the Supabase MCP; rows
begin that day and the UI captions any window that reaches further back.
Co-authored-by: Claude Fable 5.1 <noreply@anthropic.com>1 parent e28f93b commit 99532f8
13 files changed
Lines changed: 951 additions & 70 deletions
File tree
- app
- (app)/dashboard
- analytics
- projects/[id]/stats
- api/track
- lib
- dashboard
- tracker
- supabase/migrations
- tests
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
30 | 30 | | |
31 | 31 | | |
32 | 32 | | |
| 33 | + | |
| 34 | + | |
33 | 35 | | |
34 | 36 | | |
| 37 | + | |
| 38 | + | |
| 39 | + | |
| 40 | + | |
| 41 | + | |
| 42 | + | |
| 43 | + | |
35 | 44 | | |
36 | 45 | | |
37 | 46 | | |
| |||
71 | 80 | | |
72 | 81 | | |
73 | 82 | | |
74 | | - | |
75 | | - | |
| 83 | + | |
| 84 | + | |
| 85 | + | |
| 86 | + | |
76 | 87 | | |
77 | 88 | | |
78 | 89 | | |
79 | | - | |
| 90 | + | |
| 91 | + | |
| 92 | + | |
80 | 93 | | |
81 | | - | |
| 94 | + | |
82 | 95 | | |
83 | 96 | | |
84 | 97 | | |
| |||
217 | 230 | | |
218 | 231 | | |
219 | 232 | | |
| 233 | + | |
| 234 | + | |
| 235 | + | |
| 236 | + | |
| 237 | + | |
| 238 | + | |
| 239 | + | |
| 240 | + | |
| 241 | + | |
| 242 | + | |
| 243 | + | |
| 244 | + | |
| 245 | + | |
| 246 | + | |
| 247 | + | |
| 248 | + | |
220 | 249 | | |
221 | 250 | | |
222 | 251 | | |
| |||
264 | 293 | | |
265 | 294 | | |
266 | 295 | | |
| 296 | + | |
| 297 | + | |
| 298 | + | |
267 | 299 | | |
268 | 300 | | |
269 | 301 | | |
| |||
444 | 476 | | |
445 | 477 | | |
446 | 478 | | |
447 | | - | |
| 479 | + | |
| 480 | + | |
| 481 | + | |
| 482 | + | |
| 483 | + | |
| 484 | + | |
| 485 | + | |
| 486 | + | |
| 487 | + | |
448 | 488 | | |
449 | 489 | | |
450 | 490 | | |
451 | | - | |
| 491 | + | |
452 | 492 | | |
453 | 493 | | |
454 | 494 | | |
| |||
470 | 510 | | |
471 | 511 | | |
472 | 512 | | |
| 513 | + | |
| 514 | + | |
| 515 | + | |
473 | 516 | | |
474 | 517 | | |
475 | 518 | | |
| |||
485 | 528 | | |
486 | 529 | | |
487 | 530 | | |
488 | | - | |
| 531 | + | |
489 | 532 | | |
490 | 533 | | |
491 | 534 | | |
492 | 535 | | |
| 536 | + | |
| 537 | + | |
| 538 | + | |
| 539 | + | |
| 540 | + | |
| 541 | + | |
| 542 | + | |
| 543 | + | |
493 | 544 | | |
494 | | - | |
| 545 | + | |
495 | 546 | | |
496 | 547 | | |
497 | 548 | | |
| |||
633 | 684 | | |
634 | 685 | | |
635 | 686 | | |
| 687 | + | |
| 688 | + | |
| 689 | + | |
636 | 690 | | |
637 | | - | |
| 691 | + | |
638 | 692 | | |
639 | 693 | | |
640 | 694 | | |
| |||
650 | 704 | | |
651 | 705 | | |
652 | 706 | | |
653 | | - | |
| 707 | + | |
654 | 708 | | |
655 | 709 | | |
656 | 710 | | |
| |||
676 | 730 | | |
677 | 731 | | |
678 | 732 | | |
| 733 | + | |
| 734 | + | |
| 735 | + | |
679 | 736 | | |
680 | 737 | | |
681 | 738 | | |
| |||
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
3 | 3 | | |
4 | 4 | | |
5 | 5 | | |
6 | | - | |
| 6 | + | |
| 7 | + | |
7 | 8 | | |
8 | 9 | | |
9 | 10 | | |
| |||
121 | 122 | | |
122 | 123 | | |
123 | 124 | | |
124 | | - | |
| 125 | + | |
125 | 126 | | |
126 | 127 | | |
127 | 128 | | |
128 | 129 | | |
| 130 | + | |
129 | 131 | | |
130 | 132 | | |
131 | 133 | | |
| |||
266 | 268 | | |
267 | 269 | | |
268 | 270 | | |
269 | | - | |
| 271 | + | |
270 | 272 | | |
271 | 273 | | |
272 | 274 | | |
273 | | - | |
| 275 | + | |
| 276 | + | |
| 277 | + | |
274 | 278 | | |
275 | 279 | | |
276 | 280 | | |
277 | | - | |
| 281 | + | |
278 | 282 | | |
279 | 283 | | |
280 | 284 | | |
281 | | - | |
| 285 | + | |
282 | 286 | | |
283 | 287 | | |
284 | 288 | | |
285 | | - | |
| 289 | + | |
| 290 | + | |
| 291 | + | |
286 | 292 | | |
287 | 293 | | |
288 | 294 | | |
| |||
436 | 442 | | |
437 | 443 | | |
438 | 444 | | |
| 445 | + | |
| 446 | + | |
| 447 | + | |
| 448 | + | |
| 449 | + | |
| 450 | + | |
| 451 | + | |
| 452 | + | |
| 453 | + | |
| 454 | + | |
| 455 | + | |
| 456 | + | |
| 457 | + | |
| 458 | + | |
| 459 | + | |
| 460 | + | |
| 461 | + | |
| 462 | + | |
| 463 | + | |
| 464 | + | |
| 465 | + | |
| 466 | + | |
| 467 | + | |
| 468 | + | |
| 469 | + | |
| 470 | + | |
439 | 471 | | |
440 | 472 | | |
441 | 473 | | |
| |||
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
8 | 8 | | |
9 | 9 | | |
10 | 10 | | |
11 | | - | |
| 11 | + | |
12 | 12 | | |
13 | 13 | | |
| 14 | + | |
| 15 | + | |
| 16 | + | |
| 17 | + | |
| 18 | + | |
14 | 19 | | |
15 | 20 | | |
16 | 21 | | |
| |||
65 | 70 | | |
66 | 71 | | |
67 | 72 | | |
68 | | - | |
69 | | - | |
70 | | - | |
71 | | - | |
72 | | - | |
73 | | - | |
74 | | - | |
| 73 | + | |
| 74 | + | |
| 75 | + | |
| 76 | + | |
| 77 | + | |
| 78 | + | |
| 79 | + | |
75 | 80 | | |
76 | | - | |
77 | | - | |
78 | | - | |
79 | | - | |
80 | | - | |
81 | | - | |
| 81 | + | |
| 82 | + | |
| 83 | + | |
| 84 | + | |
| 85 | + | |
| 86 | + | |
| 87 | + | |
| 88 | + | |
82 | 89 | | |
83 | 90 | | |
84 | 91 | | |
85 | 92 | | |
86 | 93 | | |
87 | 94 | | |
| 95 | + | |
88 | 96 | | |
| 97 | + | |
| 98 | + | |
89 | 99 | | |
90 | 100 | | |
91 | 101 | | |
92 | 102 | | |
| 103 | + | |
| 104 | + | |
| 105 | + | |
| 106 | + | |
| 107 | + | |
93 | 108 | | |
94 | 109 | | |
95 | 110 | | |
| |||
265 | 280 | | |
266 | 281 | | |
267 | 282 | | |
268 | | - | |
| 283 | + | |
269 | 284 | | |
270 | 285 | | |
271 | 286 | | |
| |||
276 | 291 | | |
277 | 292 | | |
278 | 293 | | |
| 294 | + | |
| 295 | + | |
| 296 | + | |
279 | 297 | | |
280 | 298 | | |
281 | 299 | | |
| |||
0 commit comments