Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

MEMOIZATION cache does not work with other namespace except 'argo' #719

Open
2 tasks
heidongxianhua opened this issue Nov 28, 2023 · 5 comments
Open
2 tasks

Comments

@heidongxianhua
Copy link

Checklist

  • Checked the syntax is legal within dflow.
  • Tested using the latest version.

Summary

What happened/what you expected to happen?

What version are you running?

Diagnostics

Paste the smallest python script that reproduces the bug
when I run the examples/test_big_parameter.py, and change the namespace to test, the cache can not hit, but it work well when the namespace is 'argo'.

with the namespace 'test'
image

with the namespace 'argo'
image

@saltball
Copy link

Could you please provide details on how you change the namespace? Do you change the config['namespace'] from dflow.config, or do you use Workflow(...,namespace='test')? If it’s the latter, please try adding dflow.config['namespace']='test' in the script and then check if there are any remaining issues.

@heidongxianhua
Copy link
Author

Could you please provide details on how you change the namespace? Do you change the config['namespace'] from dflow.config, or do you use Workflow(...,namespace='test')? If it’s the latter, please try adding dflow.config['namespace']='test' in the script and then check if there are any remaining issues.

yes, I change the namespace in config, https://github.com/deepmodeling/dflow/blob/master/src/dflow/config.py#L33C40-L33C40

@zjgemi
Copy link
Collaborator

zjgemi commented Dec 20, 2023

Are your Argo components installed in the test namespace as well? Because Argo will attempt to look for memoized configmaps in the same namespace by default.

@heidongxianhua
Copy link
Author

Are your Argo components installed in the test namespace as well? Because Argo will attempt to look for memoized configmaps in the same namespace by default.

yeah, maybe it is. And, which components should I install in the test namespace too? @zjgemi

@zjgemi
Copy link
Collaborator

zjgemi commented Jan 5, 2024

https://argo-workflows.readthedocs.io/en/latest/managed-namespace/
If you only run workflows that are maintained by the same team that manages the Argo Workflows installation, you may want a namespace install (install all Argo components in the test namespace).
If you want some users to run workflows in the test namespace without granting them privileges in the namespace where Argo components is installed. Things are more complicated. You may try to submit memoized configmaps to the namespace where Argo components is installed and submit workflows to the test namespace to see if it works.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

3 participants