Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

metric.evaluate() raise TypeError: unsupported operand type(s) for +=: 'NoneType' and 'float' #1262

Open
rawheel opened this issue Jan 10, 2025 · 0 comments

Comments

@rawheel
Copy link

rawheel commented Jan 10, 2025

Describe the bug
TypeError in metric.evaluate() when called directly.

When the evaluate() method of a metric (derived from BaseMetric) is called directly (without first calling measure()), it throws a TypeError: unsupported operand type(s) for +=: 'NoneType' and 'int/float'. This is because the evaluation_cost attribute in the BaseMetric class is initialized to None.

During the evaluation process, the code attempts to increment evaluation_cost (e.g., evaluation_cost += cost). However, since evaluation_cost is None by default, this operation results in the TypeError.

Expected Behavior: evaluation_cost should be initialized to 0 to allow numerical additions.

To Reproduce
Steps to reproduce the behavior:

  1. Instantiate a metric (e.g., metric = GEval()).
  2. Call metric.evaluate(test_case) directly.

Expected behavior
A clear and concise description of what you expected to happen.

Screenshots
If applicable, add screenshots to help explain your problem.

Desktop (please complete the following information):

  • OS: iOS
  • Browser: Chrome
  • Version [e.g. 22]

Additional context
Add any other context about the problem here.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant