DDP Communication Hooks

若API未标明支持情况,则代表该API的支持情况待验证。

API名称

是否支持

限制与说明

torch.distributed.GradBucket

     

torch.distributed.GradBucket.index

     

torch.distributed.GradBucket.buffer

     

torch.distributed.GradBucket.gradients

     

torch.distributed.GradBucket.is_last

     

torch.distributed.GradBucket.set_buffer

     

torch.distributed.GradBucket.parameters

     

torch.distributed.algorithms.ddp_comm_hooks.default_hooks.allreduce_hook

     

torch.distributed.algorithms.ddp_comm_hooks.default_hooks.fp16_compress_hook

     

torch.distributed.algorithms.ddp_comm_hooks.default_hooks.bf16_compress_hook

     

torch.distributed.algorithms.ddp_comm_hooks.default_hooks.fp16_compress_wrapper

     

torch.distributed.algorithms.ddp_comm_hooks.default_hooks.bf16_compress_wrapper

     

torch.distributed.algorithms.ddp_comm_hooks.powerSGD_hook.PowerSGDState

     

torch.distributed.algorithms.ddp_comm_hooks.powerSGD_hook.powerSGD_hook

     

torch.distributed.algorithms.ddp_comm_hooks.powerSGD_hook.batched_powerSGD_hook

     

torch.distributed.algorithms.ddp_comm_hooks.debugging_hooks.noop_hook