You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
#11 has notice the problem about the expansion_factor on tokens.
But, in the official implemtention, and also in the papers, the dimension for tokens mlp is actually half of the hidden dim, and is not relevant to the num of patches.
The most intuitive evidence is that, for the same image resolution, i.e. the same num of patches, the S/M/L/B models has different MLP dimension Ds.
The text was updated successfully, but these errors were encountered:
#11 has notice the problem about the expansion_factor on tokens.
But, in the official implemtention, and also in the papers, the dimension for tokens mlp is actually half of the hidden dim, and is not relevant to the num of patches.
The most intuitive evidence is that, for the same image resolution, i.e. the same num of patches, the S/M/L/B models has different
MLP dimension Ds
.The text was updated successfully, but these errors were encountered: