Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Easier api for transforming modules with assertions #354

Merged
merged 3 commits into from
Feb 11, 2024

Conversation

Shangyint
Copy link
Collaborator

Now, it is possible to write assert_transform_module(module) to transform a module to handle suggestions and assertions. The default behavior is implemented by the backtrack_handler, which automatically backtracks for failing assertions and suggestions.

@okhat
Copy link
Collaborator

okhat commented Feb 8, 2024

That’s so cool! Can we also shorten that function’s name?

Or even make it a module method?

program.activate_assertions()

@Shangyint
Copy link
Collaborator Author

Great advice! Now we can do:

a = Program()
a.activate_assertions()
# now a can backtrack and self-refine

or

teleprompter.compile(student=Program().activate_assertions())

@okhat
Copy link
Collaborator

okhat commented Feb 8, 2024

Amazing! Before I merge,

will this break existing code?

is it an in-place or new-value method? either one is fine just checking

Copy link
Collaborator

@arnavsinghvi11 arnavsinghvi11 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

activate_assertions is nice!

to clarify, longformqa_with_assertions = assert_transform_module(LongFormQAWithAssertions().map_named_predictors(Retry), backtrack_handler)
would look something like this now?

longformqa_with_assertions = LongFormQAWithAssertions().activate_assertions()

@Shangyint
Copy link
Collaborator Author

Yes. And it shouldn't break any existing code! Could you double check both the old and new api works @arnavsinghvi11?

@okhat
Copy link
Collaborator

okhat commented Feb 8, 2024

Arnav’s answer implies it is NOT in place, just to be sure right?

@Shangyint
Copy link
Collaborator Author

Thanks! Pushed a fix. Both should work now

@arnavsinghvi11
Copy link
Collaborator

Looks good @Shangyint ! Just a small issue which relies on this change to be merged for activate_assertions() to work.

@Shangyint Shangyint merged commit cdc9c88 into main Feb 11, 2024
1 check passed
@Shangyint Shangyint deleted the misc-assertion-api branch February 11, 2024 03:14
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants