Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

关于论文中clip-based模型咨询 #1

Open
whatever-wlb opened this issue Oct 24, 2024 · 1 comment
Open

关于论文中clip-based模型咨询 #1

whatever-wlb opened this issue Oct 24, 2024 · 1 comment

Comments

@whatever-wlb
Copy link

作者您好,看到您的论文非常有启发,但是有如下疑问:
实验中的clip-based模型使用的是BGE相关模型,此模型应该是只支持文本embedding,
是使用bge替换了clip中的语言模型自行重训后的跨模态模型进行的实验吗

@JCruan519
Copy link
Collaborator

@whatever-wlb
你好!本文的工作仅聚焦于评估,并没有对BGE进行重新训练。
简单来说,在BGE的评估流程中,获取视觉和语言的embedding,随后计算相似度来决定是否插入图像。

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants