Repository navigation
[QUESTION] How to time the code #871
Description
Hi Megatron team!
I wanted to time some of Megatron's code blocks, but didn't find a good way to do it.
But I was surprised to find code like this:
if config.timers is not None:
config.timers('layernorm-grads-all-reduce', log_level=1).start(
barrier=config.barrier_with_L1_time
)
_allreduce_layernorm_grads(model, config)
if config.timers is not None:
config.timers('layernorm-grads-all-reduce').stop()
Can I us config.timers() to time the code? If yes, how can i see the output from config.timers.
Thanks!
Activity