Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

model_card_data not available on CrossEncoder #3009

Open
x-tabdeveloping opened this issue Oct 21, 2024 · 2 comments
Open

model_card_data not available on CrossEncoder #3009

x-tabdeveloping opened this issue Oct 21, 2024 · 2 comments

Comments

@x-tabdeveloping
Copy link

CrossEncoder objects currently do not have the model_card_data attribute, but SentenceTransformer objects do.

from sentence_transformers import SentenceTransformer, CrossEncoder

ce = CrossEncoder("cross-encoder/ms-marco-TinyBERT-L-2-v2")
trf = SentenceTransformer("all-MiniLM-L6-v2")

assert hasattr(trf, "model_card_data")
assert not hasattr(ce, "model_card_data")

Having access to model card information on cross-encoders would be a great convenience for the MTEB project as we produce model metadata automatically from the model_card_data attribute when loading published models.
Adding this would also make the two objects' behaviours more consistent.

@tomaarsen
Copy link
Collaborator

Hello @x-tabdeveloping!

Good to see you here.
CrossEncoder models will soon be given the model_card_data attribute, when I get around to improving CrossEncoder training in Sentence Transformers (i.e. multi-gpu, bf16, automatic model card generation, etc.). I'm hoping to have started & finished this feature in a few months.

When I've added that support, I'll be in touch with the MTEB team as they can take advantage of it.

  • Tom Aarsen

@x-tabdeveloping
Copy link
Author

Awesome @tomaarsen !! Thanks for getting back to me so quickly.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants