Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat(lm): add support for o3-mini and openai reasoning models #7649

Merged
merged 1 commit into from
Feb 3, 2025

Conversation

kalanyuz
Copy link
Contributor

@kalanyuz kalanyuz commented Feb 3, 2025

Use max_completion_tokens for OpenAI reasoning models (o1/o3)

Problem

The OpenAI o3 model family deprecated max_tokens in favor of max_completion_tokens. Additionally, both o1 and o3 models (including their mini variants) are part of the same "reasoning model" family that require specific configuration (temperature=1.0, tokens >= 5000). This change is does this behind the scenes so that it wouldn't introduce any backward compatibility issues.

Changes

  1. Updated model family detection to use regex pattern matching for all variations:

    • Base models: o1, o3
    • Mini variants: o1-mini, o3-mini
    • Any other variants starting with o1/o3
  2. Unified parameter handling for reasoning models:

    • Use max_completion_tokens instead of max_tokens for all reasoning models
    • Simplified error message to refer to "reasoning models" as a family
  3. Added comprehensive tests:

    • Verify parameter requirements are enforced
    • Check correct parameter naming (max_completion_tokens vs max_tokens)
    • Test various model name patterns

Testing

The changes are verified by new test cases in test_clients/test_lm.py:

  • test_reasoning_model_requirements: Verifies temperature and token requirements
  • test_reasoning_model_token_parameter: Tests parameter naming across different model variants

Example Usage

# Old way (would fail for o3 models)
lm = dspy.LM("openai/o3", max_tokens=5000, temperature=1.0)

# New way (works for all reasoning models)
lm = dspy.LM("openai/o3-mini", max_tokens=5000, temperature=1.0)
# Internally uses max_completion_tokens for all o1/o3 family models

@kalanyuz
Copy link
Contributor Author

kalanyuz commented Feb 3, 2025

@okhat Please have a look at this one

@kalanyuz kalanyuz mentioned this pull request Feb 3, 2025
@okhat okhat merged commit 80cdcbe into stanfordnlp:main Feb 3, 2025
4 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants