Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

fixes in _partition_param_sec function #5613

Merged
merged 1 commit into from
Jun 11, 2024
Merged

fixes in _partition_param_sec function #5613

merged 1 commit into from
Jun 11, 2024

Conversation

mmhab
Copy link
Contributor

@mmhab mmhab commented Jun 4, 2024

There are few fixes:

  • When param.ds_secondary_tensor is not None and the param has not been updated we don't need to update the param.ds_secondary_tensor.
  • In HPU the 2nd tensor partition will always be completed before the all-gather, so we don't need to add synchronize().

@mmhab
fixes in _partition_param_sec function
d04c396 
There are few fixes:
- When param.ds_secondary_tensor is not None and the param has not been
updated we don't need to update the param.ds_secondary_tensor.
- In HPU the 2nd tensor partition will allways be completed before the
  all-gather, so we don't need to add synchronize().
@mmhab mmhab requested review from tjruwase and mrwyattii as code owners June 4, 2024 13:31
@tjruwase tjruwase requested review from samadejacobs and removed request for mrwyattii June 9, 2024 22:38
@samadejacobs samadejacobs added this pull request to the merge queue Jun 11, 2024
Merged via the queue into deepspeedai:master with commit b6e24ad Jun 11, 2024
14 checks passed
@rosario-purple rosario-purple mentioned this pull request Jun 21, 2024
Open
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@samadejacobs samadejacobs samadejacobs approved these changes

@tjruwase tjruwase Awaiting requested review from tjruwase tjruwase is a code owner

Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
2 participants
@mmhab @samadejacobs