[train_engine] fsdp support mesh

[Mddct]

• HYBRID_SHARD: Apply FULL_SHARD within a node, and replicate parameters across nodes. This results in reduced communication volume as expensive all-gathers and reduce-scatters are only done within a node, which can be more performant for medium -sized models.
• torch 2 官方目前推荐使用mesh 管理各种init group，
  • https://pytorch.org/tutorials/recipes/distributed_device_mesh.html
  • https://static.sched.com/hosted_files/pytorch2023/d1/%5BPTC%2023%5D%20Composable%20PyTorch%20Distributed%20with%20PT2.pdf

[xingchensong]

fsdp