Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[NPU] Example & Quickstart updates #12650

Merged
merged 29 commits into from
Jan 7, 2025
Merged

[NPU] Example & Quickstart updates #12650

merged 29 commits into from
Jan 7, 2025

Conversation

Oscilloscope98
Copy link
Contributor

@Oscilloscope98 Oscilloscope98 commented Jan 3, 2025 •
edited
Loading

Description

Example

  • Remove models with optimize_model=False
  • Refer NPU example main readme to Quickstart
  • Unify verified tables link between Quickstart and examples (add Qwen2-7B-Instruct to examples)
  • Move embedding model into a separate folder
  • Move save/load examples under NPU/HF-Transformers-AutoModels
  • Update troubleshooting regarding accuracy tuning
  • Polish arguments (e.g. low-bit)
    • Remove disable-transpose-value-cache
  • Add sample output in English for CPP examples
  • Add link for CPP folder in main example readme
  • Update main example readme model table linking to examples

Preview: https://github.com/Oscilloscope98/ipex-llm/tree/npu-example-quickstart-updates/python/llm/example/NPU/HF-Transformers-AutoModels

Quickstart

  • Add Save/Load section
  • Improve accuracy tuning section
  • Unify verified tables link between Quickstart and examples
  • Add install cmake for CPP Prerequisites

Preview: https://github.com/Oscilloscope98/ipex-llm/blob/npu-example-quickstart-updates/docs/mddocs/Quickstart/npu_quickstart.md

@Oscilloscope98
Remove model with optimize_model=False in NPU verified models tables,…
633a52d 
… and remove related example
@Oscilloscope98
Remove experimental in run optimized model section title
6a6396b
@Oscilloscope98
Unify model table order & example cmd
9cd894d
@Oscilloscope98
Move embedding example to separate folder & update quickstart example…
6b0acfe 
… link
@Oscilloscope98
Add Quickstart reference in main NPU readme
9a613fd
@Oscilloscope98
Small fix
95b668e
@Oscilloscope98
Small fix
ed49fb5
@Oscilloscope98
Move save/load examples under NPU/HF-Transformers-AutoModels
885a336
@Oscilloscope98
Add low-bit and polish arguments for LLM Python examples
7421361
@Oscilloscope98
Small fix
50d30c6
@Oscilloscope98
Add low-bit and polish arguments for Multi-Model examples
a285861
@Oscilloscope98
Polish argument for Embedding models
e1f2339
@Oscilloscope98
Polish argument for LLM CPP examples
9d7d8b3
@Oscilloscope98
Add low-bit and polish argument for Save-Load examples
b9e5831
@Oscilloscope98
Add accuracy tuning tips for examples
0e82867
@Oscilloscope98
Update NPU qucikstart accuracy tuning with low-bit optimizations
de61245
@Oscilloscope98
Add save/load section to qucikstart
890cdd9
@Oscilloscope98
Update CPP example sample output to EN
e65934c
@Oscilloscope98
Add installation regarding cmake for CPP examples
a8e5426
@Oscilloscope98 Oscilloscope98 changed the title [WIP] NPU Example & Quickstart updates NPU Example & Quickstart updates Jan 6, 2025
@Oscilloscope98 Oscilloscope98 marked this pull request as ready for review January 6, 2025 09:46
@Oscilloscope98 Oscilloscope98 changed the title NPU Example & Quickstart updates [NPU] Example & Quickstart updates Jan 6, 2025
plusbang
plusbang approved these changes Jan 7, 2025
View reviewed changes
Copy link
Contributor

@plusbang plusbang left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@Oscilloscope98
Copy link
Contributor Author

Merge for now. Will raise another PR for any further comments :)

@Oscilloscope98 Oscilloscope98 merged commit 381d448 into intel:main Jan 7, 2025
1 check passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@plusbang plusbang plusbang approved these changes

@jason-dai jason-dai Awaiting requested review from jason-dai

Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
2 participants
@Oscilloscope98 @plusbang