-
Notifications
You must be signed in to change notification settings - Fork 54
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Refactor redaction mechanism to a single interface #929
Labels
Comments
yrobla
added a commit
that referenced
this issue
Feb 20, 2025
Instead use an uuid generator as we do for pii, and reuse same session store mechanism Closes: #929
yrobla
added a commit
that referenced
this issue
Feb 20, 2025
Instead use an uuid generator as we do for pii, and reuse same session store mechanism Closes: #929
yrobla
added a commit
that referenced
this issue
Feb 20, 2025
Instead use an uuid generator as we do for pii, and reuse same session store mechanism Closes: #929
yrobla
added a commit
that referenced
this issue
Feb 21, 2025
Instead use an uuid generator as we do for pii, and reuse same session store mechanism Closes: #929
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Description
Right now we have secrets scanning , which encrypts matched tokens, secrets etc using AES CGM. This in turn has a session manager to track the mapping of secrets to encrypted strings. In hindsight this was over engineered, we really don't need this level of encryption. It's fine to just mask / redact the string, as the LLM can never determine what the original content was. This then leaves the session manager, again encryption does not add much here, as access to the local machine would result in access to the keys within the session manager.
For PII protections I took a different approach, which is to redact using UUIDs. UUIDs work well, they make a nice unique mappable element, have much less processing overhead.
With this in mind, I am repurposing this issue to have a single interface which accepts a string (secret, PII or user nominated string) and returns a UUID. A second interface can then be called, which presents a UUID and is returned with the unredacted metadata (secrets, PII etc). We also then end up with a single session manager.
We can then add context aware sanitation and refactor PII and secrets scanning to using a single interface reaction factory
Additional Context
No response
The text was updated successfully, but these errors were encountered: