[DO NOT MERGE] patch wvSpltK_fused_moe from https://github.com/amd-hhashemi/vllm/tre…

[carlushuang]

This PR originally from https://github.com/amd-hhashemi/vllm/tree/wvSpltK_fused_moe, commit: 2f842d4

# this PR can pass below test
from vllm import LLM, SamplingParams
sampling_params = SamplingParams(temperature=0, top_k=1)
llm = LLM("/mnt/nvme0n1/models/AI-ModelScope/Mixtral-8x7B-Instruct-v0___1", tensor_parallel_size=1,dtype="float16")
print(llm.generate("What is batch inference?", sampling_params))

[github-actions]

This pull request has been automatically marked as stale because it has not had any activity within 90 days. It will be automatically closed if no further activity occurs within 30 days. Leave a comment if you feel this pull request should remain open. Thank you!