How to limit the generation of one_or_more?

[aalyousfi]

Hello,

How can I limit the generation of one_or_more? Sometimes it tends to continue generation indefinitely.

Is there a way to say one_or_more but with a maximum of 5 for example? It would be very useful to be able to do that.

Or is there a way to put a limit on the total generated tokens by one_or_more?

Or do you suggest alternative ways, other than using one_or_more, to solve this issue?

[slundberg]

Great question! You can handle this by defining your own custom guidance function. Below is an (untested) example that handles what you are asking (note that right now guidance grammars can't count so we have to repeat the content that many times):

from guidance import optional
import numpy as np

@guidance(stateless=True)
def repeat_range(lm, content, min_count=0, max_count=np.inf):
    for _ in range(min_count):
        lm += content
    if max_count == np.inf:
        lm += zero_or_more(content)
    else:
        for _ in range(max_count - min_count):
            lm += optional(content)
    return lm