Feat/954 llama cpp

[bikash119]

This PR adds llama-cpp support to create embeddings.

  from distilabel.embeddings import LlamaCppEmbeddings

  embeddings = LlamaCppEmbeddings(model="second-state/all-MiniLM-L6-v2-Q2_K.gguf")

  embeddings.load()

  results = embeddings.encode(inputs=["distilabel is awesome!", "and Argilla!"])
  # [
  #   [-0.05447685346007347, -0.01623094454407692, ...],
  #   [4.4889533455716446e-05, 0.044016145169734955, ...],
  # ]

[bikash119]

@davidberenstein1957 : I have added a working version for llama-cpp support to generate embeddings. Will add more parameters as we have here

[bikash119]

• Create embeddings using model from local disk
• Create embeddings using huggingface hub model
• Normalize embeddings by passing normalize_embeddings=True to LlamaCppEmbeddings class

[bikash119]

@davidberenstein1957 : May I request you to review the changes. In addition, I couldn't add the documentation to Component Library or to API Reference. I need a little help to add / update documentation.

[davidberenstein1957]

Looking good already. I left some initial comments. Have you seen this already #999?

[davidberenstein1957]

Hi Bikash.That would be great. Do you think we can inherent the same / similar logic using a mixin? Perhaps we can tackle a similar approach for vLLM too? Not entirely sure about potential edge cases though but feel free to explore it. Also it might be cleaner to create a separate PR/issue and tackle them there. Regards,DavidOn 1 Oct 2024, at 00:29, bikash119 ***@***.***> wrote: @bikash119 commented on this pull request. In src/distilabel/embeddings/llamacpp.py:

+ if self.hub_repository_id is not None:

+ try: + from huggingface_hub.utils import validate_repo_id + + validate_repo_id(self.hub_repository_id) + except ImportError as ie: + raise ImportError( + "Llama.from_pretrained requires the huggingface-hub package. " + "You can install it with `pip install huggingface-hub`." + ) from ie + try: + self._logger.info( + f"Attempting to load model from Hugging Face Hub: {self.hub_repository_id}" + ) + self._model = _LlamaCpp.from_pretrained( + repo_id=self.hub_repository_id, + filename=self.model, + verbose=self.verbose, + embedding=True, + kwargs=self.extra_kwargs, + ) + self._logger.info("Model loaded successfully from Hugging Face Hub") + except Exception as e: + self._logger.error( + f"Failed to load model from Hugging Face Hub: {str(e)}" + ) + raise @davidberenstein1957 : Should I capture this logic in original LLM class too? cc : @gabrielmbmb —Reply to this email directly, view it on GitHub, or unsubscribe.You are receiving this because you were mentioned.Message ID: ***@***.***>

[bikash119]

@davidberenstein1957 : Ok , let me create a different PR to handle the scenario. I will keep the this PR untouched and once we have the scenario handled using mixins, will remove the token handling code from LlamaCppEmbeddings class.

[bikash119]

Hi @davidberenstein1957 : I have made the changes to make model download from hub reusable through mixin. Please share your feedback. If this looks good, will create another PR to handle the model download in llamacpp class of llm too.

[davidberenstein1957]

Don't forget to update the docs and API references :)

[bikash119]

Don't forget to update the docs and API references :)

Have included the LlamaCppEmbeddings class to __init__.py under embeddings folder. On executing mkdocs serve I could see the documenation in Component Gallery and API reference.

[bikash119]

hi @davidberenstein1957 : Thank you for your time to review my work. I am tried to incorporate your feedback and did a few improvements.

• Added example for user to use both public and private model. For private models, the user has to configure export HF_TOKEN=hf... .
• Added an option to test --cpu-only to test embedding generation on cpu.
• Added examples for public , private, cpu based generation.
  Please share your feedback.

[davidberenstein1957]

Thanks for the updates. Left some minor comments. :)

[davidberenstein1957]

aren't some of the attributes already present in the parent class?