[From AlexC in GC]
The reference implementation uses a value of 10 for this parameter. However the implementation in GC-Optimum passes None, with no possibility for the user to set this parameter
- create training command line option
--max-weight-norm it can default to None for backwards compatibility
- in
IPUTrainer.create_optimizer pass this argument to LAMB optimizer as part of optimizer_kwargs
[From AlexC in GC]
The reference implementation uses a value of 10 for this parameter. However the implementation in GC-Optimum passes
None, with no possibility for the user to set this parameter--max-weight-normit can default toNonefor backwards compatibilityIPUTrainer.create_optimizerpass this argument to LAMB optimizer as part ofoptimizer_kwargs