More Mooncake Design Docs

[willtebbutt]

Moving forwards, I'm going to allocate 1h every morning to writing / reviewing docs for Mooncake. I'll do this until I can't find anything to spend the time on.

This isn't ready for review yet. I've mainly opened it so that I don't just have a local copy.

[codecov]

Codecov Report

All modified and coverable lines are covered by tests ✅

[github-actions]

Performance Ratio:
Ratio of time to compute gradient and time to compute function.
Warning: results are very approximate! See here for more context.

┌────────────────────────────┬──────────┬──────────┬─────────┬─────────────┬─────────┐
│                      Label │   Primal │ Mooncake │  Zygote │ ReverseDiff │  Enzyme │
│                     String │   String │   String │  String │      String │  String │
├────────────────────────────┼──────────┼──────────┼─────────┼─────────────┼─────────┤
│                   sum_1000 │ 100.0 ns │     83.8 │     1.0 │        5.61 │    8.21 │
│                  _sum_1000 │ 941.0 ns │     6.64 │  1480.0 │        33.0 │    1.08 │
│               sum_sin_1000 │  6.57 μs │     2.29 │     1.7 │        10.9 │    1.98 │
│              _sum_sin_1000 │   5.3 μs │     2.69 │   287.0 │        13.6 │    2.41 │
│                   kron_sum │ 312.0 μs │     63.2 │    3.97 │       218.0 │   956.0 │
│              kron_view_sum │ 375.0 μs │     63.0 │    8.56 │       208.0 │    87.1 │
│      naive_map_sin_cos_exp │  2.13 μs │     2.48 │ missing │        7.49 │    2.41 │
│            map_sin_cos_exp │  2.12 μs │     2.77 │    1.44 │        6.16 │     2.9 │
│      broadcast_sin_cos_exp │  2.27 μs │     2.59 │     3.7 │        1.46 │    2.26 │
│                 simple_mlp │ 413.0 μs │     5.03 │    1.32 │        7.32 │    3.48 │
│                     gp_lml │ 203.0 μs │     12.4 │    4.43 │     missing │    6.59 │
│ turing_broadcast_benchmark │  2.03 ms │     3.15 │ missing │        25.0 │ missing │
│         large_single_block │ 390.0 ns │     4.42 │  4760.0 │        30.0 │    2.18 │
└────────────────────────────┴──────────┴──────────┴─────────┴─────────────┴─────────┘