vpux-compiler error occured when using qwen2.5-7B in large content or prompt

[dockerg]

First error is occured when max-context-len >= 10000

The second error is occured when max-prompt-len >=6000

Device : Nuc 14 pro
CPU: Ultra5 125H
Memory: 96G whole
NPU driver: 32.0.100.3104
Others:

[plusbang]

Hi, @dockerg , max-context-len>=10000 is not supported yet, maybe you could try 2K.

[dockerg]

how about "DDR" memory allocate issue?

[plusbang]

how about "DDR" memory allocate issue?

Does this issue exist for smaller value of max-prompt-len (such as 1024)? Too long prompt is also not supported yet.

[dockerg]

how about "DDR" memory allocate issue?

Does this issue exist for smaller value of max-prompt-len (such as 1024)? Too long prompt is also not supported yet.

It occured when max-prompt-len > 4000;
What limit the prompt? Hardware or npu-driver or Ipex-llm not support it?

[plusbang]

It occured when max-prompt-len > 4000; What limit the prompt? Hardware or npu-driver or Ipex-llm not support it?

The issue relates to hardware and npu driver.