Bulk upserts / sync of tuples with the source data

[velmohan]

I have a scenario where I need to synchronize OpenFGA tuples with some Active Directory (AD) data. Say, for example, the tuple I am interested in adding in OpenFGA is:

user: user:xyz, relation: member object: group:xyz

In order to sync with the AD data, I have to do 3 things in OpenFGA.

1. I have to find all groups used in the tuples which will be a strict subset of what the AD server might have.
2. Find which tuples need to be added and which needs to be deleted for each AD group.
3. Use the Write API to add or delete users as found from 2

As far as I am aware, there is no support for queries like in 1 in OpenFGA yet. So, as tactical solution, I am reading all tuples from OpenFGA to achieve this. Since I have all tuples, then 2 and 3 are easily done. But this solution is obviously not scalable.

Are there any suggestions on how to do this right. Other options I am considering is

1. Read data directly from the DB to perform 1 and 2. This is something I have been trying to avoid.
2. Contributing code to

• do a type query to find all objects of a certain type and
• to perform a bulk upsert, unless there are any objections for adding them.

[aaguiarz]

Hi @velmohan

When you need to synchronize a data source with OpenFGA, you would need to somehow subscribe to changes in that data source, so you can properly write / delete tuples when needed.

Bulk inserts are useful when you want to bootstrap the system with the initial data, but you would not need upsert for that scenario.

If you are using an identity provider that uses Active Directory, and you can get claims in the id token for the specific data you want to use in OpenFGA (e.g. group membership), then you can pass those tuples as contextual tuples.

Let me know if this helps, it's a pretty common scenario and we'd like to find a way to make it work.

[velmohan]

Thanks @aaguiarz. I agree. Active Directory (AD) changes can be synced using SCIM protocol or something similar. Even if we do this for AD data, we have some other data sources that do not provide a way to subscribe to the changes as of now. This is a big challenge for us now.

Auditability is a critical requirement of our system. I have discussed this topic before with @rhamzeh. Using contextual tuples would probably make this auditability requirement near impossible to implement and hence I am trying to avoid using contextual tuples for this core data.

I will have a detailed review about this next week and will keep you posted.

[aaguiarz]

Hi @velmohan

I don't see why contextual tuples would impact auditability. You'd have a log with the contextual tuple at the moment of check. Can you elaborate on what's the problem that you see?

[velmohan]

Hi @aaguiarz

My concern was that we would not be able to find out if a user had a permission to do a certain action at a given time unless we ran a check at that time with the context tuples which then would be in the logs. In particular, am I correct to say we would not be able to use Expand endpoint to explain how a user was allowed to perform an action.

[aaguiarz]

@velmohan there's no way in OpenFGA to know if a user had a specific permission at a specific time (besides looking at the logs and see if there's a check() request at that time).

The only way to achieve that would be to apply all tuple changes from the beginning of time until the specific moment. You can do that by consuming the /changes endpoint, but it's not very pratical.

[velmohan]

Thanks for the clarification @aaguiarz . I am experimenting some solutions for the sync issue. I will post what works for us.