add use_all_gather for option

[SangbumChoi]

What does this PR do?

Fixes #3147
@muellerzr

I think testing might need but not sure where to add this corresponding test

Before submitting

• This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).
• Did you read the contributor guideline,
  Pull Request section?
• Was this discussed/approved via a Github issue or the forum? Please add a link
  to it if that's the case.
• Did you make sure to update the documentation with your changes? Here are the
  documentation guidelines, and
  here are tips on formatting docstrings.
• Did you write any new necessary tests?

Who can review?

Anyone in the community is free to review the PR once the tests have passed. Feel free to tag
members/contributors who may be interested in your PR.

[SangbumChoi]

Also have additional question for LLM inference evaluation process. Does LLM evaluation process requires to have all_gather function to make all the prediction in all processes?

[muellerzr]

Thanks! Tests should be in src/accelerate/test_utils/scripts/test_ops.py

The all_gather is so that we can get the results across all processes. It's there as an example and for simplicity for the user. (But it is not generally required, no)

[SangbumChoi]

@muellerzr Hi I haved added test. However, to be more strict test I think we can add additional checking when the process is not in the state.is_main_process and check the return is empty list. Let me know how do you think about this.

[SangbumChoi]

@muellerzr Gentle Ping!