ModuleNotFoundError: No module named 'qa_mdt'

[saba1999AI]

ERROR: pip's dependency resolver does not currently take into account all the packages that are installed. This behaviour is the source of the following dependency conflicts.
sentence-transformers 3.3.1 requires transformers<5.0.0,>=4.41.0, but you have transformers 4.30.2 which is incompatible.
Successfully installed torch-2.3.0+cu121 torchvision-0.18.0+cu121
Traceback (most recent call last):
File "/content/AudioLDM2/OpenMusic/gradio/gradio_app.py", line 19, in
from qa_mdt.pipeline import MOSDiffusionPipeline
ModuleNotFoundError: No module named 'qa_mdt'

[ivcylc]

cd ..
mv openmusic qa_mdt

[Uj947nXmRqV2nRaWshKtHzTvckUUpD]

cd .. mv openmusic qa_mdt

i get same error even after renaming main folder, running command from within qa_mdt folder where i cloned the repo:
python gradio/gradio_app.py

Traceback (most recent call last):
File "/home/user/qa_mdt/gradio/gradio_app.py", line 19, in
from qa_mdt.pipeline import MOSDiffusionPipeline
ModuleNotFoundError: No module named 'qa_mdt'

Later edit: i also renamed folder openmusic from withing the repo folder, but getting same error

Later edit 2:
The README should be modified to add the main requiremenets as well

pip install -r gradio/requirements.txt
python gradio/gradio_app.py 

to

pip install requirements.txt
pip install -r gradio/requirements.txt
python gradio/gradio_app.py 

[ivcylc]

For the first question, i recommond you to add:
`import sys

sys.path.append('path/to/father_folder_of_qa_mdt')`
at the top of your python file

For the second, you are right, you can pull a request~

[Uj947nXmRqV2nRaWshKtHzTvckUUpD]

i tried with sys.path.append('~/OpenMusic'), but got same error
then i tried sys.path.append('/home/user/OpenMusic') and now i get a different error

Traceback (most recent call last):
File "/home/user/OpenMusic/gradio/gradio_app.py", line 21, in
from qa_mdt.pipeline import MOSDifuserPipeline
File "/home/user/OpenMusic/qa_mdt/pipeline.py", line 1, in
from diffusers import DifuserPipeline
File "/home/user/miniconda3/envs/OpenMusic_python_3.10/lib/python3.10/site-packages/diffusers/init.py", line 5, in
from .utils import (
File "/home/user/miniconda3/envs/OpenMusic_python_3.10/lib/python3.10/site-packages/diffusers/utils/init.py", line 38, in
from .dynamic_modules_utils import get_class_from_dynamic_module
File "/home/user/miniconda3/envs/OpenMusic_python_3.10/lib/python3.10/site-packages/diffusers/utils/dynamic_modules_utils.py", line 28, in
from huggingface_hub import cached_download, hf_hub_download, model_info
ImportError: cannot import name 'cached_download' from 'huggingface_hub' (/home/user/miniconda3/envs/OpenMusic_python_3.10/lib/python3.10/site-packages/huggingface_hub/init.py)

Later edit: this new error seems related to huggingface/huggingface-inference-toolkit#92
huggingface_hub must be <0.26.0

what gradio version should i use?

[ivcylc]

trying sys.path.append('/home/user/OpenMusic') is correct

for the new error, so sorry i have forgot the gradio version, but you can also try to see this video for specific details Youtube video on how to run this project locally

Any problems feel free here

[Uj947nXmRqV2nRaWshKtHzTvckUUpD]

was able to pinpoint to this working gradio version
pip install gradio==5.14.0

also along the way i had to fix many many conflicts and dependency issues in requirements files
it's quite a mess

using python 3.10

in the end i was able to get the local server to run in http://127.0.0.1:7860/, by running python gradio/gradio_app.py

but now i get another error and generation doesn't work:

...
which: no node in (/home/user/miniconda3/envs/OpenMusic_python_3.10/bin:/home/linuxbrew/.linuxbrew/bin:/home/linuxbrew/.linuxbrew/sbin:/home/user/miniconda3/condabin:/home/user/.local/bin:/home/user/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games:/usr/local/games:/usr/lib/wsl/lib)
* Running on local URL:  http://127.0.0.1:7860
Caching examples at: '/home/user/OpenMusic/.gradio/cached_examples/14'
Seed set to 0
Add-ons: [<function waveform_rs_48k at 0x7f75f99cbf40>]
Dataset initialize finished
Reload ckpt specified in the config file ./qa_mdt/checkpoint_389999.ckpt
ERROR:    Exception in ASGI application
Traceback (most recent call last):
  File "/home/user/miniconda3/envs/OpenMusic_python_3.10/lib/python3.10/site-packages/uvicorn/protocols/http/h11_impl.py", line 403, in run_asgi
    result = await app(  # type: ignore[func-returns-value]
  File "/home/user/miniconda3/envs/OpenMusic_python_3.10/lib/python3.10/site-packages/uvicorn/middleware/proxy_headers.py", line 60, in __call__
    return await self.app(scope, receive, send)
  File "/home/user/miniconda3/envs/OpenMusic_python_3.10/lib/python3.10/site-packages/fastapi/applications.py", line 1054, in __call__
    await super().__call__(scope, receive, send)
  File "/home/user/miniconda3/envs/OpenMusic_python_3.10/lib/python3.10/site-packages/starlette/applications.py", line 112, in __call__
    await self.middleware_stack(scope, receive, send)
  File "/home/user/miniconda3/envs/OpenMusic_python_3.10/lib/python3.10/site-packages/starlette/middleware/errors.py", line 187, in __call__
    raise exc
  File "/home/user/miniconda3/envs/OpenMusic_python_3.10/lib/python3.10/site-packages/starlette/middleware/errors.py", line 165, in __call__
    await self.app(scope, receive, _send)
  File "/home/user/miniconda3/envs/OpenMusic_python_3.10/lib/python3.10/site-packages/gradio/route_utils.py", line 789, in __call__
    await self.app(scope, receive, send)
  File "/home/user/miniconda3/envs/OpenMusic_python_3.10/lib/python3.10/site-packages/starlette/middleware/exceptions.py", line 62, in __call__
    await wrap_app_handling_exceptions(self.app, conn)(scope, receive, send)
  File "/home/user/miniconda3/envs/OpenMusic_python_3.10/lib/python3.10/site-packages/starlette/_exception_handler.py", line 53, in wrapped_app
    raise exc
  File "/home/user/miniconda3/envs/OpenMusic_python_3.10/lib/python3.10/site-packages/starlette/_exception_handler.py", line 42, in wrapped_app
    await app(scope, receive, sender)
  File "/home/user/miniconda3/envs/OpenMusic_python_3.10/lib/python3.10/site-packages/starlette/routing.py", line 715, in __call__
    await self.middleware_stack(scope, receive, send)
  File "/home/user/miniconda3/envs/OpenMusic_python_3.10/lib/python3.10/site-packages/starlette/routing.py", line 735, in app
    await route.handle(scope, receive, send)
  File "/home/user/miniconda3/envs/OpenMusic_python_3.10/lib/python3.10/site-packages/starlette/routing.py", line 288, in handle
    await self.app(scope, receive, send)
  File "/home/user/miniconda3/envs/OpenMusic_python_3.10/lib/python3.10/site-packages/starlette/routing.py", line 76, in app
    await wrap_app_handling_exceptions(app, request)(scope, receive, send)
  File "/home/user/miniconda3/envs/OpenMusic_python_3.10/lib/python3.10/site-packages/starlette/_exception_handler.py", line 53, in wrapped_app
    raise exc
  File "/home/user/miniconda3/envs/OpenMusic_python_3.10/lib/python3.10/site-packages/starlette/_exception_handler.py", line 42, in wrapped_app
    await app(scope, receive, sender)
  File "/home/user/miniconda3/envs/OpenMusic_python_3.10/lib/python3.10/site-packages/starlette/routing.py", line 73, in app
    response = await f(request)
  File "/home/user/miniconda3/envs/OpenMusic_python_3.10/lib/python3.10/site-packages/fastapi/routing.py", line 301, in app
    raw_response = await run_endpoint_function(
  File "/home/user/miniconda3/envs/OpenMusic_python_3.10/lib/python3.10/site-packages/fastapi/routing.py", line 212, in run_endpoint_function
    return await dependant.call(**values)
  File "/home/user/miniconda3/envs/OpenMusic_python_3.10/lib/python3.10/site-packages/gradio/routes.py", line 1425, in startup_events
    await app.get_blocks().run_extra_startup_events()
  File "/home/user/miniconda3/envs/OpenMusic_python_3.10/lib/python3.10/site-packages/gradio/blocks.py", line 2894, in run_extra_startup_events
    await startup_event()
  File "/home/user/miniconda3/envs/OpenMusic_python_3.10/lib/python3.10/site-packages/gradio/helpers.py", line 460, in _start_caching
    await self.cache()
  File "/home/user/miniconda3/envs/OpenMusic_python_3.10/lib/python3.10/site-packages/gradio/helpers.py", line 526, in cache
    prediction = await self.root_block.process_api(
  File "/home/user/miniconda3/envs/OpenMusic_python_3.10/lib/python3.10/site-packages/gradio/blocks.py", line 2051, in process_api
    result = await self.call_function(
  File "/home/user/miniconda3/envs/OpenMusic_python_3.10/lib/python3.10/site-packages/gradio/blocks.py", line 1598, in call_function
    prediction = await anyio.to_thread.run_sync(  # type: ignore
  File "/home/user/miniconda3/envs/OpenMusic_python_3.10/lib/python3.10/site-packages/anyio/to_thread.py", line 56, in run_sync
    return await get_async_backend().run_sync_in_worker_thread(
  File "/home/user/miniconda3/envs/OpenMusic_python_3.10/lib/python3.10/site-packages/anyio/_backends/_asyncio.py", line 2461, in run_sync_in_worker_thread
    return await future
  File "/home/user/miniconda3/envs/OpenMusic_python_3.10/lib/python3.10/site-packages/anyio/_backends/_asyncio.py", line 962, in run
    result = context.run(func, *args)
  File "/home/user/miniconda3/envs/OpenMusic_python_3.10/lib/python3.10/site-packages/gradio/utils.py", line 883, in wrapper
    response = f(*args, **kwargs)
  File "/home/user/OpenMusic/gradio/gradio_app.py", line 30, in generate_waveform
    pipe(high_quality_description)
  File "/home/user/miniconda3/envs/OpenMusic_python_3.10/lib/python3.10/site-packages/torch/utils/_contextlib.py", line 115, in decorate_context
    return func(*args, **kwargs)
  File "/home/user/OpenMusic/qa_mdt/pipeline.py", line 74, in __call__
    infer(
  File "/home/user/OpenMusic/qa_mdt/infer/infer_mos5.py", line 66, in infer
    latent_difuser = instantiate_from_config(configs["model"])
  File "/home/user/OpenMusic/qa_mdt/audioldm_train/utilities/model_util.py", line 104, in instantiate_from_config
    return get_obj_from_str(config["target"])(**config.get("params", dict()))
  File "/home/user/OpenMusic/qa_mdt/audioldm_train/utilities/model_util.py", line 112, in get_obj_from_str
    return getattr(importlib.import_module(module, package=None), cls)
  File "/home/user/miniconda3/envs/OpenMusic_python_3.10/lib/python3.10/importlib/__init__.py", line 126, in import_module
    return _bootstrap._gcd_import(name[level:], package, level)
  File "<frozen importlib._bootstrap>", line 1050, in _gcd_import
  File "<frozen importlib._bootstrap>", line 1027, in _find_and_load
  File "<frozen importlib._bootstrap>", line 1006, in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 688, in _load_unlocked
  File "<frozen importlib._bootstrap_external>", line 883, in exec_module
  File "<frozen importlib._bootstrap>", line 241, in _call_with_frames_removed
  File "/home/user/OpenMusic/qa_mdt/audioldm_train/modules/latent_difuser/ddpm.py", line 20, in <module>
    from qa_mdt.audioldm_train.conditional_models import *
  File "/home/user/OpenMusic/qa_mdt/audioldm_train/conditional_models.py", line 7, in <module>
    from qa_mdt.audioldm_train.modules.clap.open_clip import create_model
  File "/home/user/OpenMusic/qa_mdt/audioldm_train/modules/clap/open_clip/__init__.py", line 24, in <module>
    from .tokenizer import SimpleTokenizer, tokenize
  File "/home/user/OpenMusic/qa_mdt/audioldm_train/modules/clap/open_clip/tokenizer.py", line 164, in <module>
    _tokenizer = SimpleTokenizer()
  File "/home/user/OpenMusic/qa_mdt/audioldm_train/modules/clap/open_clip/tokenizer.py", line 78, in __init__
    merges = gzip.open(bpe_path).read().decode("utf-8").split("\n")
  File "/home/user/miniconda3/envs/OpenMusic_python_3.10/lib/python3.10/gzip.py", line 301, in read
    return self._buffer.read(size)
  File "/home/user/miniconda3/envs/OpenMusic_python_3.10/lib/python3.10/_compression.py", line 118, in readall
    while data := self.read(sys.maxsize):
  File "/home/user/miniconda3/envs/OpenMusic_python_3.10/lib/python3.10/gzip.py", line 488, in read
    if not self._read_gzip_header():
  File "/home/user/miniconda3/envs/OpenMusic_python_3.10/lib/python3.10/gzip.py", line 436, in _read_gzip_header
    raise BadGzipFile('Not a gzipped file (%r)' % magic)
gzip.BadGzipFile: Not a gzipped file (b've')

Later edit: i think i have to download the models. sorry i'm new to this project. just setting it up

[Uj947nXmRqV2nRaWshKtHzTvckUUpD]

these are the steps I followed so far:

conda create -n baseline_python_3.10 python=3.10
conda activate baseline_python_3.10

git clone https://github.com/ivcylc/OpenMusic.git; cd OpenMusic

// solve metadata error
pip install pip==24.0
	# also downgrade pipdeptree: pip install pipdeptree==2.23.1

// solve conflicts, in this order:
pip install poetry==1.8 # instead of required 1.7.1
pip install platformdirs==4.0.0 # poetry depends on it
pip install cachecontrol==0.14.2 # poetry depends on it
pip install pydantic-core==2.27.2
pip install --upgrade typing-extensions
pip install pydantic==2.10.
pip install huggingface-hub==0.25.2 # latest version before this issue https://github.com/huggingface/huggingface-inference-toolkit/issues/92
	# spaces requires gradio which requires huggingface-hub
pip install gradio==5.14.0 # latest version that doesn't require upgrading huggingface-hub
vim requirements.txt
	# comment all 3 poetry lines: poetry, poetry-core, poetry-plugin-export (will install 1.8.0 instead of required 1.6.0)
	# comment torchvision 0.16.0 which depends on torch==2.1.0; required torch==2.3.0
	# comment cachecontrol==0.13.1 which conflicts with poetry 1.8
	# comment pydantic-core
	# comment typing-extensions
	# comment huggingface-hub
	# comment typer, tomlkit, packaging # related to gradio
pip install spaces

	

pip install -r requirements.txt

pip install -r gradio/requirements.txt

python gradio/gradio_app.py

mv openmusic qa_mdt

python gradio/gradio_app.py

[Uj947nXmRqV2nRaWshKtHzTvckUUpD]

in the readme is specified "Down the main checkpoint of our QA-MDT model from https://huggingface.co/lichang0928/QA-MDT"
where should i put this file?

also "Before training, you need to download extra ckpts needed in ./audioldm_train/config/mos_as_token/qa_mdt.yaml and offset_pretrained_checkpoints.json
Noted that: All above checkpoints can be downloaded from:
flan-t5-large
clap_music
roberta-base
others"

where should i put these checkpoint files? also do i need to edit those yaml and json, how?

thank you

later edit: i found that main checkpoint (13504339928 B checkpoint_389999.ckpt) is already in the openmusic folder and qa_mdt (renamed) folders, which are actually duplicate folders, occupying a lot of space >64GB. the python script gradio_app.py shouldn't download it again after renaming into qa_mdt

[Uj947nXmRqV2nRaWshKtHzTvckUUpD]

i modified in script gradio/gradio_app.py, so that it goes into folder qa_mdt directly (to prevent duplication)

os.system('git clone https://huggingface.co/jadechoghari/openmusic')
to
os.system('git clone https://huggingface.co/jadechoghari/openmusic qa_mdt')

[Uj947nXmRqV2nRaWshKtHzTvckUUpD]

REF #9