Implementation of Dirac Initialization Method

[Saanidhyavats]

I was thinking to add https://pytorch.org/docs/stable/_modules/torch/nn/init.html#dirac_ initialization method. It can be a useful feature for MLX.

[Saanidhyavats]

What do you think about it @awni ?

[awni]

It's a bit niche.. so I'm not sure it's worth adding. I see that it's in PyTorch..but I also don't know who uses that anymore. What we can do is leave the issue open.. and if it gets attention we can consider adding it reactively. I think it's better than adding code that will never / rarely be used.

[Saanidhyavats]

Okay, sounds good