Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Feature][PyTorch migration rule] Added PyTorch build script migration rule file #2564

Open
wants to merge 5 commits into
base: SYCLomatic
Choose a base branch
from

Conversation

TejaX-Alaghari
Copy link
Contributor

This PR introduces python_build_script_migration_rule_pytorch.yaml rule file for Python script migration using PyTorch's XPU class instead of IPEX's XPU

@TejaX-Alaghari TejaX-Alaghari requested a review from a team as a code owner December 11, 2024 16:32
clang/lib/DPCT/DPCT.cpp Outdated Show resolved Hide resolved
Copy link
Contributor

@zhiweij1 zhiweij1 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Please update

@TejaX-Alaghari TejaX-Alaghari force-pushed the py_mig branch 2 times, most recently from 30509a6 to 628cd85 Compare January 6, 2025 18:25
Copy link
Contributor

@tomflinda tomflinda left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@TejaX-Alaghari pls enabled the E2E test in the test repo.

case MigratePythonBuildScriptSpecifiedButPythonRuleFileNotSpecified:
StatusString =
"Error: The option -migrate-build-script=Python requires python rule "
"file option --rule-file=python_migration_rule_[ipex|pytorch].yaml be "
Copy link
Contributor

@zhimingwang36 zhimingwang36 Jan 8, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
"file option --rule-file=python_migration_rule_[ipex|pytorch].yaml be "
"rules to be explicitly loaded with option --rule-file, for example --rule-file={dpct_install_folder}/extensions/python_rules/python_build_script_migration_rule_[ipex|pytorch].yaml to load the predefined rules.

Copy link
Contributor Author

@TejaX-Alaghari TejaX-Alaghari Jan 8, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done
Please verify

std::string getBuildScriptNotSpecifiedWarning() {
return "Warning: Only CMake scripts will be migrated as no "
"--migrate-build-script option is provided. "
"See https://www.intel.com/content/www/us/en/docs/"
Copy link
Contributor

@zhimingwang36 zhimingwang36 Jan 8, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

suggest to remove the URL link. It's not necessary. And for SYCLomatic, it's better to provide URL to SYCLomatic doc

Copy link
Contributor Author

@TejaX-Alaghari TejaX-Alaghari Jan 8, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I've removed the link now.
I will raise another PR for adding SYCLomatic doc link.
And in syclct, I'll later cherry-pick to remove it.

Is that fine?

@@ -2389,7 +2389,7 @@ std::string DpctGlobalInfo::SYCLHeaderExtension = std::string();
clang::tooling::UnifiedPath DpctGlobalInfo::CudaPath;
std::string DpctGlobalInfo::RuleFile = std::string();
UsmLevel DpctGlobalInfo::UsmLvl = UsmLevel::UL_None;
BuildScriptKind DpctGlobalInfo::BuildScriptVal = BuildScriptKind::BS_None;
unsigned DpctGlobalInfo::BuildScriptType = 0;
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

if BuildScriptType default to zero, then it default to (BS_CMake), then migrateCMakeScripts() will be defatult return true. which conflict with option desc: "By default, no build script is migrated.\n" in DPCTOptions.inc.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Here, BuildScriptType holds the value of the selected option by user (by cl::bits option), not the BS_CMake enum value.
So,
0 => No build script migration
1 => CMake script migration
2 => Python script migration
3 => Both CMake and Python script migration

if (MigrateBuildScriptOnly) {
if (!BuildScriptsSpecified) {
llvm::errs() << getBuildScriptNotSpecifiedWarning();
DpctGlobalInfo::setBuildScript(1); // enable cmake script migration
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
DpctGlobalInfo::setBuildScript(1); // enable cmake script migration
DpctGlobalInfo::setBuildScript(1); // Please update magic number to meaning macro.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Added the logic to calculate bits value for CMake script selection (1 << BS_CMake)

@TejaX-Alaghari
Copy link
Contributor Author

The tests failure is due to expected ngt-check-options test resulting conflict between migrate-build-script-only and migrate-build-script

This is because DAK_BuildScript being added to DPCT_OPTION_ACTIONS for --migrate-build-script (here)
If this line is not necessary, please let me know. I'll revert it

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants