Print Hyperparamter

[hschaeufler]

Is it possible to return the used config for hyperparameters like temp and top_p if no values are defined for them?

For example as follows would be nice:

model_path = "results/llama3_1_8B_instruct_lora/tuning_03/lora_fused_model"
model, tokenizer = load(model_path)

prompt =  """
Say hello!
"""
messages = [{"role": "user", "content": prompt}]
prompt = tokenizer.apply_chat_template(
    messages, tokenize=False, add_generation_prompt=True
)

response = generate(model, tokenizer, prompt=prompt, verbose=False, max_tokens=128000)
print(response.hyperparams)

{‘temp’: 0.6, ‘top_p’: 1.0, ‘max_kv_size’: None, ‘cache_history’: None}

[nsswifter]

[awni]

This is not correct. There is no field hyperparams on the response.

Was this answer generated by an LLM?

[hschaeufler]

This is not correct. There is no field hyperparams on the response.
Was this answer generated by an LLM?

Since I aimed to provide a helpful answer, I used an LLM as a tool to help articulate my response and gather insights. I recognize the importance of transparency, and I truly apologize this came across as misleading; That was not my intention.

Is this also generated by an LLM?

How @awni said: I don't think this will work like intended. I think the response of generate is a string. I don't quite understand the second example either. I want to read the default/hyper parameters from mlx-lm without having to open or adapt the code?

[hschaeufler]

This is not correct. There is no field hyperparams on the response.
Was this answer generated by an LLM?

Since I aimed to provide a helpful answer, I used an LLM as a tool to help articulate my response and gather insights. I recognize the importance of transparency, and I truly apologize this came across as misleading; That was not my intention.

Is this also generated by an LLM?
How @awni said: I don't think this will work like intended. I think the response of generate is a string. I don't quite understand the second example either. I want to read the default/hyper parameters from mlx-lm without having to open or adapt the code?

Is this also generated by an LLM?

No, it does NOT

Sorry, didn't mean to sound offensive. I'm always grateful for help. That's why I wanted to understand how you meant the second example?