The purpose of this example is to demonstrate following features of NVFlare,
- Run NVFlare in secure mode
- Show secure admin client and authentication
- Demonstrate local authorization policy
- Demonstrate local privacy policy
1. Install Python and Virtual Environment,
python3 -m venv nvflare-env source nvflare-env/bin/activate
2. Install NVFlare
pip install nvflare
3. The example is part of the NVFlare source code. The source code can be obtained like this,
git clone https://github.com/NVIDIA/NVFlare.git
4. TLS requires domain names. Please add following line in /etc/hosts
file,
127.0.0.1 server1
The project.yml
file defines all the sites and users (called admin in NVFlare)
used in the examples. The startup kits will be created by setup.sh
cd NVFlare/examples/advanced/federated-policies ./setup.sh
All the startup kits will be generated in this folder,
workspace/fed_policy/prod_00
Note
workspace
folder is removed everytime setup.sh
is run. Please do not save customized
files in this folder.
This script will start up the server and 2 clients,
./start.sh
In secure mode, NVFlare creates one startup kit for each user. There are 5 users in
this example so there are 5 folders for admin login under workspace/fed_policy/prod_00
folder.
To login as an user, the appropriate folder must be selected.
For example, this is how to login as [email protected]
user,
cd workspace/fed_policy/prod_00/[email protected] ./startup/fl_admin.sh
At the prompt, enter the user email [email protected]
Multiple users can login at the same time by using multiple terminals.
The setup.sh has copied the jobs folder to the workspace folder. So jobs can be submitted like this, type the following command in the admin console:
- ::
- submit_job ../../job1
server1
: NVFlare serversite_a
: Client owned by a.org with a customized authorization policy, which only allows
users from the same org to submit job.
* site_b
: Client owned by b.org with a customized privacy policy. The policy defines
two scopes public
and private
. A custom filter is applied to private
.
[email protected]
: Super user with roleproject_admin
who can do everything[email protected]
: Admin for a.org with roleorg_admin
[email protected]
: Lead trainer for a.org with rolelead
[email protected]
: Lead trainer for b.org with rolelead
[email protected]
: Regular user for b.org with rolemember
All the jobs run the same app (numpy-sag) but have different scopes defined in meta.json
.
- job1: Scope is
public
. No filters. - job2: Scope is
test
. Test filters are applied to data and result. - job3: Scope is
private
. PercentilePrivacy filter is applied to result. - job4: It has no scope defined.
- job5: It defines an non-existent scope
foo
We will demo some authorization behaviors.
Since authorization decision is determined using each site's authorization.json and each admin user's role,
we just use job1
in all the following tests.
User | Command | Expected behavior |
---|---|---|
[email protected] | submit_job ../../job1 | Job deployed and started on all sites |
[email protected] | clone_job [the job ID that we previous submitted] | Job deployed and started on all sites |
[email protected] | clone_job [the job ID that we previous submitted] | Rejected because submitter is in a different org |
[email protected] | submit_job ../../job1 | Rejected because role "org_admin" is not allowed to submit jobs |
[email protected] | submit_job ../../job1 | site_a rejected the job because the submitter is in a different org, while site_b accepted the job so the job will still run since in meta.json we specify min_clients as 1 |
site_a has no privacy policy defined. So we will test the following cases on site_b.
In each job's meta.json we specified their "scope" and in site's privacy.json file each site will define its own privacy filters to apply for that scope.
Note that default jobs are treated in "public" scope.
Let's just use user [email protected] for the following tests.
Job | Expected behavior |
---|---|
job1 | Job deployed with no filters |
job2 | Job deployed with TestFilter applied |
job3 | Job deployed with PercentilePrivacy filter applied to the result |
job4 | Job deployed using default scope public |
job5 | Job rejected by site_b because foo doesn't exist |
All NVFlare server and clients can be stopped by using this script,
./stop.sh