[Q&A] How do I get the decrypted model from federated learning with homomorphic encryption?

[thibaultferaux]

Python version (python3 -V)

3.10.12

NVFlare version (python3 -m pip list | grep "nvflare")

2.5.2

NVFlare branch (if running examples, please use the branch that corresponds to the NVFlare version, git branch)

2.5

Operating system

Ubuntu 22.04.3 LTS

Have you successfully run any of the following examples?

• hello-numpy-sag with simulator
• hello-pt with simulator
• hello-numpy-sag with POC
• hello-pt with POC

Please describe your question

I have trained a model with homomorphic encryption like the example here but when I download the job with the trained model in the server to do inference on I get a model where the weights are binary instead of torch tensors. I think this is because the downloaded model is still decrypted. How can I make sure the server saves the decrypted model so I can download it and perform inference on the trained model?

This is my config-fed-client.json:

{
  "format_version": 2,
  "DATA_PATH": "/tmp/data",
  "AGGREGATION_EPOCHS": 2,
  "executors": [
    {
      "tasks": [
        "train",
        "submit_model",
        "validate"
      ],
      "executor": {
        "id": "Executor",
        "path": "nvflare.app_common.executors.model_learner_executor.ModelLearnerExecutor",
        "args": {
          "learner_id": "mhist-learner"
        }
      }
    }
  ],
  "components": [
    {
      "id": "mhist-learner",
      "path": "src.mhist_model_learner.MHISTModelLearner",
      "args": {
        "train_idx_root": "{DATA_PATH}",
        "aggregation_epochs": "{AGGREGATION_EPOCHS}",
        "lr": 0.001
      }
    },
    {
      "id": "log_writer",
      "path": "nvflare.app_opt.tracking.tb.tb_writer.TBWriter",
      "args": {
        "event_type": "analytix_log_stats"
      }
    },
    {
      "id": "event_to_fed",
      "path": "nvflare.app_common.widgets.convert_to_fed_event.ConvertToFedEvent",
      "args": {
        "events_to_convert": [
          "analytix_log_stats"
        ],
        "fed_event_prefix": "fed."
      }
    }
  ],
  "task_data_filters": [
    {
      "tasks": ["train", "validate"],
      "filters":[
          {
          "path": "nvflare.app_opt.he.model_decryptor.HEModelDecryptor",
          "args": {
          }
        }
      ]
    }
  ],
  "task_result_filters": [
    {
      "tasks": ["train"],
      "filters":[
          {
          "path": "nvflare.app_opt.he.model_encryptor.HEModelEncryptor",
          "args": {
            "weigh_by_local_iter": true
          }
        }
      ]
    },
    {
      "tasks": ["submit_model"],
      "filters":[
          {
          "path": "nvflare.app_opt.he.model_encryptor.HEModelEncryptor",
          "args": {
            "weigh_by_local_iter": false
          }
        }
      ]
    }
  ]
}

This is my config_fed_server.json:

{
    "format_version": 2,
    "min_clients": 3,
    "num_rounds": 2,
    "workflows": [
        {
            "id": "scatter_and_gather",
            "path": "nvflare.app_common.workflows.scatter_and_gather.ScatterAndGather",
            "args": {
                "min_clients": "{min_clients}",
                "num_rounds": "{num_rounds}",
                "start_round": 0,
                "wait_time_after_min_received": 10,
                "aggregator_id": "aggregator",
                "persistor_id": "persistor",
                "shareable_generator_id": "shareable_generator",
                "train_task_name": "train",
                "train_timeout": 0
            }
        },
        {
            "id": "cross_site_model_eval",
            "path": "nvflare.app_opt.he.cross_site_model_eval.HECrossSiteModelEval",
            "args": {
                "model_locator_id": "model_locator",
                "submit_model_timeout": 600,
                "validation_timeout": 6000,
                "cleanup_models": true
            }
        }
    ],
    "components": [
        {
            "id": "serialize_filter",
            "path": "nvflare.app_opt.he.model_serialize_filter.HEModelSerializeFilter",
            "args": {
            }
        },
        {
            "id": "persistor",
            "path": "nvflare.app_opt.pt.file_model_persistor.PTFileModelPersistor",
            "args": {
                "model": {
                    "path": "src.efficientnet_model.CustomEfficientNet",
                    "args": {}
                },
                "filter_id": "serialize_filter"
            }
        },
        {
            "id": "shareable_generator",
            "path": "nvflare.app_opt.he.model_shareable_generator.HEModelShareableGenerator",
            "args": {}
        },
        {
            "id": "aggregator",
            "path": "nvflare.app_opt.he.intime_accumulate_model_aggregator.HEInTimeAccumulateWeightedAggregator",
            "args": {
                "weigh_by_local_iter": false
            }
        },
        {
            "id": "model_selector",
            "path": "nvflare.app_common.widgets.intime_model_selector.IntimeModelSelector",
            "args": {
            }
        },
        {
            "id": "model_locator",
            "path": "nvflare.app_opt.pt.file_model_locator.PTFileModelLocator",
            "args": {
                "pt_persistor_id": "persistor"
            }
        },
        {
            "id": "json_generator",
            "path": "nvflare.app_common.widgets.validation_json_generator.ValidationJsonGenerator",
            "args": {}
        },
        {
            "id": "receiver",
            "path": "nvflare.app_opt.tracking.tb.tb_receiver.TBAnalyticsReceiver",
            "args": {
                "events": [
                    "fed.analytix_log_stats"
                ]
            }
        }
    ],
    "task_data_filters": [],
    "task_result_filters": []
}

[YuanTingHsieh]

@thibaultferaux
I think for the HE example, we try to protect the weights against the server side attack.
So aggregation only happens with encrypted weights.

Each client need to store their corresponding "clear" weights locally after each round of training.

@holgerroth @ZiyueXu77 please correct me if I am wrong, thanks!

[ZiyueXu77]

Yes, the HE in this example is to protect against server, so on server side it is alway ciphertext.
What you can do to retrieve the global model is to save the global weights on the client side once they are received.

[thibaultferaux]

Thank you I saved the global model on the client side to use the model for inference.