[Feature]: Support for Pydantic v2

[movchan74]

Problem

I'm encountering a dependency conflict due to incompatible versions of pydantic between dstack and another package my project relies on. Specifically:

• My project (aana version 0.2.2.2) requires pydantic >=2.0.
• dstack (version 0.18.17) depends on pydantic >=1.10.10, <2.0.0.

As a result, I'm unable to resolve dependencies when adding dstack alongside other tools in my environment that are already using Pydantic v2.

The broader AI and software community are shifting towards Pydantic v2 for its improved performance and features, and many AI projects have either fully migrated to v2 or at least added support for it. This version conflict is limiting adoption and making it challenging to integrate dstack into modern AI/ML projects that have already upgraded to Pydantic v2.

Solution

No response

Workaround

No response

Would you like to help us implement this feature by sending a PR?

No

[r4victor]

@movchan74, we have plans to migrate dstack to Pydantic v2. The migration is currently non-trivial since some of the dstack dependencies do not yet support Pydantic v2 (https://github.com/zmievsa/pydantic-duality for sure). We'll explore how it can be done.

You're using the dstack Python API, so you cannot install dstack into a separate venv? Is it the case?

[movchan74]

It's great to hear that migrating to Pydantic v2 is on the roadmap! I understand that it’s non-trivial.

For now, this isn't a major blocker, as I can indeed set up a separate venv for dstack as you suggested. I'm still exploring how best to integrate dstack into our product, so I’ll work with this workaround in the meantime.

Thanks again for the quick response and for considering the upgrade!

[zmievsa]

Will add support to pydantic duality for pydantic 2 this week.