Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

TODO for nnue.md #127

Open
6 of 12 tasks
Sopel97 opened this issue Jun 8, 2021 · 0 comments
Open
6 of 12 tasks

TODO for nnue.md #127

Sopel97 opened this issue Jun 8, 2021 · 0 comments
Labels
documentation Improvements or additions to documentation enhancement New feature or request

Comments

@Sopel97
Copy link
Member

Sopel97 commented Jun 8, 2021

  • some results regarding net size (https://docs.google.com/document/d/1gTlrr02qSNKiXNZ_SuO4-RjK4MXBiFlLE6jvNqqMkAY/edit#heading=h.p7t1qfux64dc exps 84, 85, 86, 89)
  • about data used to train the nets (filtering, mirroring and stuff)
  • fix header links in doc
  • make headings understandable without context
  • consistent formatting (for feature set names, net arch names, etc...)
  • backprop math
  • linear layer with block sparse outputs forward (no results yet but it's an interesting direction)
  • linear layer with sparse inputs forward (put some data about sparsity after CReLU)
  • ways to reduce the network size with Half* (less king buckets, perspective mirror)
  • either write something about weight pruning or remove the mention that it will be described later in the document
  • revise implementation for sparse input FC (pad nnz indices so that only one loop is needed. use maddubs -> madd(ones, ...) implementation that processes 4 inputs at a time, with 8 bit weights). See https://pastebin.com/zDa7PbJh
  • a better nnz indices implementation for the above, see On sparsity and the best way to compute indices of non zero elements. syzygy1/Cfish#204
@Sopel97 Sopel97 added documentation Improvements or additions to documentation enhancement New feature or request labels Jun 8, 2021
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
documentation Improvements or additions to documentation enhancement New feature or request
Projects
None yet
Development

No branches or pull requests

1 participant