Skip to content
This repository has been archived by the owner on Oct 31, 2023. It is now read-only.

About the projection and prediction head dimension configs #39

Open
Dylan-H-Wang opened this issue Aug 30, 2022 · 0 comments
Open

About the projection and prediction head dimension configs #39

Dylan-H-Wang opened this issue Aug 30, 2022 · 0 comments

Comments

@Dylan-H-Wang
Copy link

Dylan-H-Wang commented Aug 30, 2022

Hi,

Thank you for the amazing works and I am inspired a lot from it!

One thing I want to ask is how you designed the structure of your projection and predications heads, i.e., how to decide the number of layers and hidden and output dimensions? P.S. I notice that you have already mentioned the bottleneck structure of prediction MLP is helpful.

Thank you for the helps and time!

Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant