Skip to content
Merged
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion training_rules.adoc
Original file line number Diff line number Diff line change
Expand Up @@ -185,7 +185,7 @@ If any run logs more than one seed, a warning is raised by the package checker.
OPEN: Any random number generation may be used. The seed is not expected to be logged.

=== Numerical formats
CLOSED: The numerical formats fp64, fp32, tf32, fp16, fp8, nvfp4, mxfp4, bfloat16, Graphcore FLOAT 16.16, int8, uint8, int4, and uint4 are pre-approved for use. Additional formats require explicit approval. Any numerically safe techniques needed to compensate for different precision may be used, provided they leave the real-valued mathematical function of the model, i.e. the function the model would compute at infinite precision, unchanged.
CLOSED: The numerical formats fp64, fp32, tf32, fp16, fp8, mxfp6, nvfp4, mxfp4, bfloat16, Graphcore FLOAT 16.16, int8, uint8, int4, and uint4 are pre-approved for use. Additional formats require explicit approval. Any numerically safe techniques needed to compensate for different precision may be used, provided they leave the real-valued mathematical function of the model, i.e. the function the model would compute at infinite precision, unchanged.

Some examples of numerically safe techniques that are allowed include but are not limited to:

Expand Down
Loading