Skip to content

chore(antithesis): DiffEq, count total contexts emitted - #2157

Closed
blt wants to merge 1 commit into
mainfrom
blt/chore_antithesis_diffeq_count_total_contexts_emitted
Closed

chore(antithesis): DiffEq, count total contexts emitted#2157
blt wants to merge 1 commit into
mainfrom
blt/chore_antithesis_diffeq_count_total_contexts_emitted

Conversation

@blt

@blt blt commented Jul 17, 2026

Copy link
Copy Markdown
Contributor

Summary

Change Type

  • Bug fix
  • New feature
  • Non-functional (chore, refactoring, docs)
  • Performance

How did you test this PR?

References

@dd-octo-sts dd-octo-sts Bot added area/docs Reference documentation. area/test All things testing: unit/integration, correctness, SMP regression, etc. labels Jul 17, 2026

blt commented Jul 17, 2026

Copy link
Copy Markdown
Contributor Author

@datadog-datadog-prod-us1-2

datadog-datadog-prod-us1-2 Bot commented Jul 17, 2026

Copy link
Copy Markdown
Contributor

Pipelines  Tests

⚠️ Warnings

🚦 34 Pipeline jobs failed

DataDog/saluki | test-integration   View in Datadog   GitLab

DataDog/saluki | Ubuntu base image check   View in Datadog   GitLab

DataDog/saluki | build-adp-baseline-image   View in Datadog   GitLab

View all 34 failed jobs.

ℹ️ Info

🔄 Datadog auto-retried 1 job - 0 passed on retry View in Datadog

Useful? React with 👍 / 👎

This comment will be updated automatically if new data arrives.
🔗 Commit SHA: 54fb12c | Docs | Datadog PR Page | Give us feedback!

@pr-commenter

pr-commenter Bot commented Jul 17, 2026

Copy link
Copy Markdown

Binary Size Analysis (Agent Data Plane)

Baseline: e4cba81 · Comparison: 54fb12c · diff
Analysis Configuration: stripped binaries · Pass/Fail Threshold: +5%
Sizes: 40.86 MiB (baseline) vs 40.86 MiB (comparison)
Size Change: +0 B (+0.00%)

✅ Binary size difference within threshold

Changes by Module
Module File Size Symbols
anon.44a0faa3240fa409baa379409d152aec.1.llvm.15436866739357029521 +130 B 1
anon.44a0faa3240fa409baa379409d152aec.1.llvm.18163430659783575214 -130 B 1
anon.44a0faa3240fa409baa379409d152aec.4.llvm.15436866739357029521 +114 B 1
anon.44a0faa3240fa409baa379409d152aec.4.llvm.18163430659783575214 -114 B 1
anon.44a0faa3240fa409baa379409d152aec.3.llvm.15436866739357029521 +109 B 1
anon.44a0faa3240fa409baa379409d152aec.3.llvm.18163430659783575214 -109 B 1
anon.44a0faa3240fa409baa379409d152aec.0.llvm.15436866739357029521 +97 B 1
anon.44a0faa3240fa409baa379409d152aec.0.llvm.18163430659783575214 -97 B 1
anon.44a0faa3240fa409baa379409d152aec.2.llvm.15436866739357029521 +95 B 1
anon.44a0faa3240fa409baa379409d152aec.2.llvm.18163430659783575214 -95 B 1
Detailed Symbol Changes
    FILE SIZE        VM SIZE    
 --------------  -------------- 
  [NEW]    +130  [NEW]     +40    anon.44a0faa3240fa409baa379409d152aec.1.llvm.15436866739357029521
  [NEW]    +114  [NEW]     +24    anon.44a0faa3240fa409baa379409d152aec.4.llvm.15436866739357029521
  [NEW]    +109  [NEW]     +19    anon.44a0faa3240fa409baa379409d152aec.3.llvm.15436866739357029521
  [NEW]     +97  [NEW]      +7    anon.44a0faa3240fa409baa379409d152aec.0.llvm.15436866739357029521
  [NEW]     +95  [NEW]      +5    anon.44a0faa3240fa409baa379409d152aec.2.llvm.15436866739357029521
  [DEL]     -95  [DEL]      -5    anon.44a0faa3240fa409baa379409d152aec.2.llvm.18163430659783575214
  [DEL]     -97  [DEL]      -7    anon.44a0faa3240fa409baa379409d152aec.0.llvm.18163430659783575214
  [DEL]    -109  [DEL]     -19    anon.44a0faa3240fa409baa379409d152aec.3.llvm.18163430659783575214
  [DEL]    -114  [DEL]     -24    anon.44a0faa3240fa409baa379409d152aec.4.llvm.18163430659783575214
  [DEL]    -130  [DEL]     -40    anon.44a0faa3240fa409baa379409d152aec.1.llvm.18163430659783575214
  [ = ]       0  [ = ]       0    TOTAL

@blt
blt changed the base branch from blt/chore_antithesis_update_endpoint_submission to graphite-base/2157 July 17, 2026 22:57
@blt
blt force-pushed the blt/chore_antithesis_diffeq_count_total_contexts_emitted branch from 93c9df1 to 662a005 Compare July 17, 2026 22:58
@blt
blt force-pushed the graphite-base/2157 branch from 0f927d3 to ab87b38 Compare July 17, 2026 22:58
@graphite-app
graphite-app Bot changed the base branch from graphite-base/2157 to main July 17, 2026 22:58
@blt
blt force-pushed the blt/chore_antithesis_diffeq_count_total_contexts_emitted branch from 662a005 to 4ce3352 Compare July 17, 2026 22:58
@pr-commenter

pr-commenter Bot commented Jul 17, 2026

Copy link
Copy Markdown

Regression Detector (Agent Data Plane)

Run ID: b3856e89-af4f-410e-9e86-ac28a55742de
Baseline: e4cba812 · Comparison: 54fb12c4 · diff

Optimization Goals: ✅ No significant changes detected

Fine details of change detection per experiment (3)

Experiments configured erratic: true are tagged (ignored) and skipped when determining which experiments regressed or improved. Experiments which are detected as erratic at runtime are tagged (erratic) to flag that the run's sample dispersion was high, but their regression / improvement signal still counts.

experiment goal Δ mean % links
quality_gates_rss_dsd_low memory ⚪ -0.17 metrics profiles logs
quality_gates_rss_dsd_medium memory ⚪ -0.17 metrics profiles logs
quality_gates_rss_idle memory ⚪ -0.54 metrics profiles logs
Bounds Checks: ✅ Passed (3)
experiment check replicates observed links
quality_gates_rss_dsd_low memory_usage 10/10 ✅ 45.4 MiB ≤ 50 MiB metrics profiles logs
quality_gates_rss_dsd_medium memory_usage 10/10 ✅ 67.1 MiB ≤ 75 MiB metrics profiles logs
quality_gates_rss_idle memory_usage 10/10 ✅ 31.3 MiB ≤ 40 MiB metrics profiles logs
Explanation

A change is flagged as a regression when |Δ mean %| > 5.00% in the regressing direction for its optimization goal AND SMP marks the experiment as a regression (is_regression: true). Improvements use the matching criteria for the improving direction. Experiments configured erratic: true (tagged (ignored)) are skipped outright; experiments detected as erratic at runtime (tagged (erratic)) still count, since that flag describes sample dispersion rather than directional certainty. The Δ mean % cell is colored accordingly: 🟢 = improvement, 🔴 = regression, ⚪ = neutral. Reduction in CPU or memory is an improvement; reduction in ingress throughput is a regression.

This commit adjusts the differential scenario to also count the
total number of contexts emitted in a run by both lanes. Previously
the symdiff of ADP-on / ADP-off would pass even if, say, the ratio
of actually emitted contexts was 10:1 or whatever. We now say that
this ratio closes to 1:1 over time. I have needed to fiddle with the
faults present in the differential scenario just because one lane being
network faulted while the other isn't doesn't make much sense.
@blt
blt force-pushed the blt/chore_antithesis_diffeq_count_total_contexts_emitted branch from 4ce3352 to 54fb12c Compare July 20, 2026 17:58
@blt blt mentioned this pull request Jul 20, 2026
4 tasks
@blt blt closed this Jul 29, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

area/docs Reference documentation. area/test All things testing: unit/integration, correctness, SMP regression, etc.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant