[TorchFX] Export to `torch.export.export_for_training` #3075

daniil-lyakhov · 2024-11-11T17:15:18Z

Changes

TorchFX Unit tests are moved from torch._export.capture_pre_autograd_graph to torch.export.export_for_training
ALL REFERENCE GRAPHS WERE VALIDATED MANUALLY
BC types for fuse_bn_node are updated
NNCFGraphBuilder is updated to support a batch-norm type with only one output node (instead of three)
Model extractor does not traverse down from constans to prevent redundant nodes in the extracted model when the constant is shared
shared_constants_unification_transformation is removed
Tests which require capture_pre_autograd_graph are removed

Reason for changes

To migrate to the lates and recommended export method for TorchFX backend

Related tickets

#2766
#2987

Tests

test_shared_constants_unification_not_connected_const
post_training_quantization/540/ is finished successfully

daniil-lyakhov · 2024-11-13T13:47:13Z

tests/torch/conftest.py

@@ -120,6 +128,10 @@ def pytest_configure(config: Config):
    if regen_dot:


What if I would like to update json files only?

The regen_dot was added there by mistake, I have updated the suggestion. https://github.com/openvinotoolkit/nncf/pull/3075/files/68bb7b6f09bba89935ae44a7aff6e8f3bee237ea#r1841827740

anzr299 · 2024-11-13T09:11:27Z

tests/torch/fx/test_model_transformer.py

@@ -384,17 +380,53 @@ def count_constants(model: torch.fx.GraphModule) -> int:
    return num_constant_nodes


+@pytest.mark.skipif(sys.platform.startswith("win"), reason="capture_pre_autograd_graph is not supported on Windows")
 def test_create_shared_constant_transformation():
    model = MultiBranchesConnectedModel()
    ex_inputs = torch.ones((1, 3, 3, 3))


would you suggest we remove this transformation completely? Since, the shared nodes are taken care of now that we are using torch.export.export_for_training and we do not necessarily need to support torch._export.capture_pre_autograd

tests/torch/ptq/test_weights_compression.py

alexsu52

Please remove support for capture_pre_autograd form the tests.

daniil-lyakhov · 2024-11-13T13:45:21Z

tests/torch/fx/test_model_transformer.py

@@ -384,17 +380,53 @@ def count_constants(model: torch.fx.GraphModule) -> int:
    return num_constant_nodes


+@pytest.mark.skipif(sys.platform.startswith("win"), reason="capture_pre_autograd_graph is not supported on Windows")
 def test_create_shared_constant_transformation():
    model = MultiBranchesConnectedModel()
    ex_inputs = torch.ones((1, 3, 3, 3))


daniil-lyakhov · 2024-11-13T13:47:13Z

tests/torch/conftest.py

@@ -120,6 +128,10 @@ def pytest_configure(config: Config):
    if regen_dot:


What if I would like to update json files only?

daniil-lyakhov · 2024-11-13T13:48:43Z

Please remove support for capture_pre_autograd form the tests.

capture_pre_autograd is completely removed from unit tests

anzr299 · 2024-11-14T09:10:54Z

tests/torch/conftest.py

    if regen_dot:
        os.environ["NNCF_TEST_REGEN_DOT"] = "1"

+    regen_json = config.getoption("--regen-json", False)
+    if regen_json:
+        os.environ["NNCF_TEST_REGEN_JSON"] = "1"
+


Suggested change

if regen_dot:

os.environ["NNCF_TEST_REGEN_DOT"] = "1"

regen_json = config.getoption("--regen-json", False)

if regen_json:

os.environ["NNCF_TEST_REGEN_JSON"] = "1"

for option, var in [("--regen-dot", "NNCF_TEST_REGEN_DOT"), ("--regen-json", "NNCF_TEST_REGEN_JSON")]:

regen = config.getoption(option, False)

if regen:

os.environ[env_var] = "1"

Done, please check

…it#3075) ### Changes * TorchFX Unit tests are moved from `torch._export.capture_pre_autograd_graph` to `torch.export.export_for_training` ALL REFERENCE GRAPHS WERE VALIDATED MANUALLY * BC types for `fuse_bn_node` are updated * NNCFGraphBuilder is updated to support a batch-norm type with only one output node (instead of three) * Model extractor does not traverse down from constans to prevent redundant nodes in the extracted model when the constant is shared * `shared_constants_unification_transformation` is removed * Tests which require `capture_pre_autograd_graph` are removed ### Reason for changes * To migrate to the lates and recommended export method for TorchFX backend ### Related tickets openvinotoolkit#2766 ### Tests test_shared_constants_unification_not_connected_const post_training_quantization/540/ is finished successfully

PR #3075 to the release branch: ### Changes * TorchFX Unit tests are moved from `torch._export.capture_pre_autograd_graph` to `torch.export.export_for_training` ALL REFERENCE GRAPHS WERE VALIDATED MANUALLY * BC types for `fuse_bn_node` are updated * NNCFGraphBuilder is updated to support a batch-norm type with only one output node (instead of three) * Model extractor does not traverse down from constans to prevent redundant nodes in the extracted model when the constant is shared * `shared_constants_unification_transformation` is removed * Tests which require `capture_pre_autograd_graph` are removed ### Reason for changes * To migrate to the lates and recommended export method for TorchFX backend ### Related tickets #2766 ### Tests test_shared_constants_unification_not_connected_const post_training_quantization/540/ is finished successfully

…it#3075) ### Changes * TorchFX Unit tests are moved from `torch._export.capture_pre_autograd_graph` to `torch.export.export_for_training` ALL REFERENCE GRAPHS WERE VALIDATED MANUALLY * BC types for `fuse_bn_node` are updated * NNCFGraphBuilder is updated to support a batch-norm type with only one output node (instead of three) * Model extractor does not traverse down from constans to prevent redundant nodes in the extracted model when the constant is shared * `shared_constants_unification_transformation` is removed * Tests which require `capture_pre_autograd_graph` are removed ### Reason for changes * To migrate to the lates and recommended export method for TorchFX backend ### Related tickets openvinotoolkit#2766 ### Tests test_shared_constants_unification_not_connected_const post_training_quantization/540/ is finished successfully

github-actions bot added NNCF PT Pull requests that updates NNCF PyTorch experimental labels Nov 11, 2024

daniil-lyakhov force-pushed the dl/fx/export_for_training branch 2 times, most recently from a33ce2e to 78f8588 Compare November 12, 2024 10:27

WIP

78f8588

daniil-lyakhov force-pushed the dl/fx/export_for_training branch from 2d8f400 to 707d4cd Compare November 12, 2024 10:51

test_shared_constants_unification_not_connected_const

707d4cd

daniil-lyakhov requested review from anzr299 and alexsu52 November 12, 2024 12:19

daniil-lyakhov marked this pull request as ready for review November 12, 2024 12:19

daniil-lyakhov requested a review from a team as a code owner November 12, 2024 12:19

daniil-lyakhov force-pushed the dl/fx/export_for_training branch from 84d4039 to 68bb7b6 Compare November 12, 2024 13:12

Skip capture_pre_autograd tests on Windows

68bb7b6

anzr299 approved these changes Nov 13, 2024

View reviewed changes

alexsu52 reviewed Nov 13, 2024

View reviewed changes

daniil-lyakhov requested a review from alexsu52 November 13, 2024 13:47

daniil-lyakhov commented Nov 13, 2024

View reviewed changes

daniil-lyakhov requested a review from anzr299 November 13, 2024 13:48

shared_constants_unification_transformation is removed

04290c2

alexsu52 approved these changes Nov 13, 2024

View reviewed changes

anzr299 reviewed Nov 14, 2024

View reviewed changes

daniil-lyakhov force-pushed the dl/fx/export_for_training branch from 9e946ee to 29f15e0 Compare November 14, 2024 09:30

anzr299 approved these changes Nov 14, 2024

View reviewed changes

Comment

29f15e0

alexsu52 merged commit 90d15a6 into openvinotoolkit:develop Nov 14, 2024
14 checks passed

daniil-lyakhov mentioned this pull request Nov 14, 2024

[Release][TorchFX] Export to torch.export.export_for_training #3087

Merged

daniil-lyakhov mentioned this pull request Nov 25, 2024

[Torch FX] Support for ExportedProgram from torch.export.export in Weights Compression #2987

Closed

1 task

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[TorchFX] Export to `torch.export.export_for_training` #3075

[TorchFX] Export to `torch.export.export_for_training` #3075

daniil-lyakhov commented Nov 11, 2024 •

edited

Loading

daniil-lyakhov Nov 13, 2024

anzr299 Nov 14, 2024 •

edited

Loading

anzr299 Nov 13, 2024

daniil-lyakhov Nov 13, 2024

alexsu52 left a comment •

edited

Loading

daniil-lyakhov Nov 13, 2024

daniil-lyakhov Nov 13, 2024

daniil-lyakhov commented Nov 13, 2024

anzr299 Nov 14, 2024

daniil-lyakhov Nov 14, 2024

		@@ -120,6 +128,10 @@ def pytest_configure(config: Config):
		if regen_dot:

[TorchFX] Export to torch.export.export_for_training #3075

[TorchFX] Export to torch.export.export_for_training #3075

Conversation

daniil-lyakhov commented Nov 11, 2024 • edited Loading

Changes

Reason for changes

Related tickets

Tests

daniil-lyakhov Nov 13, 2024

Choose a reason for hiding this comment

anzr299 Nov 14, 2024 • edited Loading

Choose a reason for hiding this comment

anzr299 Nov 13, 2024

Choose a reason for hiding this comment

daniil-lyakhov Nov 13, 2024

Choose a reason for hiding this comment

alexsu52 left a comment • edited Loading

Choose a reason for hiding this comment

daniil-lyakhov Nov 13, 2024

Choose a reason for hiding this comment

daniil-lyakhov Nov 13, 2024

Choose a reason for hiding this comment

daniil-lyakhov commented Nov 13, 2024

anzr299 Nov 14, 2024

Choose a reason for hiding this comment

daniil-lyakhov Nov 14, 2024

Choose a reason for hiding this comment

[TorchFX] Export to `torch.export.export_for_training` #3075

[TorchFX] Export to `torch.export.export_for_training` #3075

daniil-lyakhov commented Nov 11, 2024 •

edited

Loading

anzr299 Nov 14, 2024 •

edited

Loading

alexsu52 left a comment •

edited

Loading