[Backend Support] tinygrad

[DavidLandup0]

Thoughts on supporting tinygrad as a backend?
GitHub: https://github.com/tinygrad/tinygrad

Some of the reasons would include:

• Highly torch-like, so implementing the backend wouldn't be difficult syntactically
• Simple to add accelerators and supports accelerators such as WebGPU, AMD, etc. which aren't common. It isn't as CUDA/cuBLAS-dependent as PyTorch or TensorFlow. This could help bring accelerated DL and inference to a higher number of users with consumer grade hardware and web apps.
• Lightweight with minimal dependencies

Some of the difficulties would potentially include:

• Fewer core ops meaning some common ops in torch, numpy, jax and tensorflow would need a bit of extra implementation to add
• Adding another supported backend inherently increases the maintenance cost of the library

If approved, I'm volunteering to implement this.

[SamanehSaadat]

@fchollet I was wondering what you think about this. Do you think we should support tinygrad backend in Keras?

[fchollet]

The question has come up before. The answer is, we're open to adding it in the future, but not today. Adding a new backend depends on:

• Popularity of the backend package
• Willingness of contributors to help ship the new backend
• Support/approval from the backend package authors

Our next backend target is still MLX. We can add tinygrad at some point in the future, but it will depend on making progress on the points above.

MLX enjoys a level of popularity comparable to Keras-Hub: https://pypistats.org/packages/mlx
Tinygrad is not very popular yet: https://pypistats.org/packages/tinygrad