Repository navigation
Conversation
Give additional disclosure with the results to help public understand what was done
|
MLCommons CLA bot All contributors have signed the MLCommons CLA ✍️ ✅ |
|
|
||
| Results need to be informative and provide basic information about how they were run and convergence without needing to run additional scripts or consulting conmfigurational files. Specifically, the results the following must be disclosed as a part of each result | ||
|
|
||
| * Smallest (most compact) numerical precision used to accelerate training |
There was a problem hiding this comment.
Since that does not paint a real picture of the results - shouldn't we require more information about the numerical precision recipe used for training? Like which inputs, activations, weights, gradients, optimized states use which precision; and what techniques are used to stabilize convergence?
|
|
||
| * Smallest (most compact) numerical precision used to accelerate training | ||
|
|
||
| * Parallelization techniques employed, such as CP8, PP4, or TP1 |
There was a problem hiding this comment.
Documenting this information is error prone and adds additional overhead for submitters. Can we make these MLLOG tags so the results file shows this information?
|
|
||
| * RCP checker output | ||
|
|
||
| Additionally, configuration files must be named so that they can easily be matched with output files. |
There was a problem hiding this comment.
Or, the submitter needs to ensure that there's a clear way to figure out which config file was used for which submission.
|
Added directly into the logging repo |
2 participants
Give additional disclosure with the results to help public understand what was done