Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Not building CUDA 12.6 #3835

Open
johnnynunez opened this issue Sep 25, 2024 · 1 comment
Open

Not building CUDA 12.6 #3835

johnnynunez opened this issue Sep 25, 2024 · 1 comment

Comments

@johnnynunez
Copy link

johnnynunez commented Sep 25, 2024

🐛 Describe the bug

It's not building with last version of cuda 12.6.1 in jetson agx orin

#!/usr/bin/env bash
set -ex
echo "Building torchaudio ${TORCHAUDIO_VERSION}"
   
apt-get update
apt-get install -y --no-install-recommends \
		git \
		pkg-config \
		libffi-dev \
		libsndfile1

rm -rf /var/lib/apt/lists/*
apt-get clean

git clone --branch v${TORCHAUDIO_VERSION} --recursive --depth=1 https://github.com/pytorch/audio /opt/torchaudio
cd /opt/torchaudio
git checkout v${TORCHAUDIO_VERSION}

BUILD_SOX=1 python3 setup.py bdist_wheel --verbose --dist-dir /opt

cd ../
rm -rf /opt/torchaudio

pip3 install --no-cache-dir --verbose /opt/torchaudio*.whl
pip3 show torchaudio && python3 -c 'import torchaudio; print(torchaudio.__version__);'

twine upload --verbose /opt/torchaudio*.whl || echo "failed to upload wheel to ${TWINE_REPOSITORY_URL}"
src/include -isystem /usr/local/lib/python3.10/dist-packages/torch/include -isystem /usr/local/lib/python3.10/dist-packages/torch/include/torch/csrc/api/include -isystem /usr/local/cuda/include -Wall -D_GLIBCXX_USE_CXX11_ABI=1 -O3 -DNDEBUG -std=gnu++17 -fPIC -D_GLIBCXX_USE_CXX11_ABI=1 -MD -MT src/libtorio/ffmpeg/CMakeFiles/_torio_ffmpeg4.dir/pybind/pybind.cpp.o -MF src/libtorio/ffmpeg/CMakeFiles/_torio_ffmpeg4.dir/pybind/pybind.cpp.o.d -o src/libtorio/ffmpeg/CMakeFiles/_torio_ffmpeg4.dir/pybind/pybind.cpp.o -c /opt/torchaudio/src/libtorio/ffmpeg/pybind/pybind.cpp
In file included from /usr/local/lib/python3.10/dist-packages/torch/include/c10/util/Exception.h:5,
                 from /usr/local/lib/python3.10/dist-packages/torch/include/ATen/BlasBackend.h:3,
                 from /usr/local/lib/python3.10/dist-packages/torch/include/ATen/Context.h:3,
                 from /usr/local/lib/python3.10/dist-packages/torch/include/ATen/ATen.h:7,
                 from /usr/local/lib/python3.10/dist-packages/torch/include/torch/csrc/api/include/torch/types.h:3,
                 from /opt/torchaudio/src/libtorio/ffmpeg/ffmpeg.h:3,
                 from /opt/torchaudio/src/libtorio/ffmpeg/hw_context.h:3,
                 from /opt/torchaudio/src/libtorio/ffmpeg/pybind/pybind.cpp:1:
/opt/torchaudio/src/libtorio/ffmpeg/pybind/pybind.cpp: In function ‘int torio::io::{anonymous}::{anonymous}::read_func(void*, uint8_t*, int)’:
/opt/torchaudio/src/libtorio/ffmpeg/pybind/pybind.cpp:125:19: warning: comparison of integer expressions of different signedness: ‘long unsigned int’ and ‘int’ [-Wsign-compare]
  125 |         chunk_len <= request,
      |         ~~~~~~~~~~^~~~~~~~~~
In file included from /opt/torchaudio/build/temp.linux-aarch64-cpython-310/_deps/f4-src/include/libavutil/avutil.h:296,
                 from /opt/torchaudio/build/temp.linux-aarch64-cpython-310/_deps/f4-src/include/libavutil/samplefmt.h:24,
                 from /opt/torchaudio/build/temp.linux-aarch64-cpython-310/_deps/f4-src/include/libavcodec/avcodec.h:31,
                 from /opt/torchaudio/src/libtorio/ffmpeg/ffmpeg.h:10,
                 from /opt/torchaudio/src/libtorio/ffmpeg/hw_context.h:3,
                 from /opt/torchaudio/src/libtorio/ffmpeg/pybind/pybind.cpp:1:
/opt/torchaudio/src/libtorio/ffmpeg/pybind/pybind.cpp: In function ‘int torio::io::{anonymous}::read_bytes(void*, uint8_t*, int)’:
/opt/torchaudio/build/temp.linux-aarch64-cpython-310/_deps/f4-src/include/libavutil/common.h:105:25: warning: comparison of integer expressions of different signedness: ‘std::basic_string_view<char>::size_type’ {aka ‘long unsigned int’} and ‘int’ [-Wsign-compare]
  105 | #define FFMIN(a,b) ((a) > (b) ? (b) : (a))
      |                     ~~~~^~~~~
/opt/torchaudio/src/libtorio/ffmpeg/pybind/pybind.cpp:202:19: note: in expansion of macro ‘FFMIN’
  202 |   auto num_read = FFMIN(wrapper->src.size() - wrapper->index, buf_size);
      |                   ^~~~~
[82/92] /usr/bin/c++ -DTORIO_FFMPEG_EXT_NAME=_torio_ffmpeg6 -DUSE_C10D_GLOO -DUSE_C10D_MPI -DUSE_C10D_NCCL -DUSE_CUDA -DUSE_DISTRIBUTED -DUSE_RPC -DUSE_TENSORPIPE -D_torio_ffmpeg6_EXPORTS -I/opt/torchaudio/src -I/usr/include/python3.10 -I/opt/torchaudio/build/temp.linux-aarch64-cpython-310/_deps/f6-src/include -isystem /usr/local/lib/python3.10/dist-packages/torch/include -isystem /usr/local/lib/python3.10/dist-packages/torch/include/torch/csrc/api/include -isystem /usr/local/cuda/include -Wall -D_GLIBCXX_USE_CXX11_ABI=1 -O3 -DNDEBUG -std=gnu++17 -fPIC -D_GLIBCXX_USE_CXX11_ABI=1 -MD -MT src/libtorio/ffmpeg/CMakeFiles/_torio_ffmpeg6.dir/pybind/pybind.cpp.o -MF src/libtorio/ffmpeg/CMakeFiles/_torio_ffmpeg6.dir/pybind/pybind.cpp.o.d -o src/libtorio/ffmpeg/CMakeFiles/_torio_ffmpeg6.dir/pybind/pybind.cpp.o -c /opt/torchaudio/src/libtorio/ffmpeg/pybind/pybind.cpp
In file included from /usr/local/lib/python3.10/dist-packages/torch/include/c10/util/Exception.h:5,
                 from /usr/local/lib/python3.10/dist-packages/torch/include/ATen/BlasBackend.h:3,
                 from /usr/local/lib/python3.10/dist-packages/torch/include/ATen/Context.h:3,
                 from /usr/local/lib/python3.10/dist-packages/torch/include/ATen/ATen.h:7,
                 from /usr/local/lib/python3.10/dist-packages/torch/include/torch/csrc/api/include/torch/types.h:3,
                 from /opt/torchaudio/src/libtorio/ffmpeg/ffmpeg.h:3,
                 from /opt/torchaudio/src/libtorio/ffmpeg/hw_context.h:3,
                 from /opt/torchaudio/src/libtorio/ffmpeg/pybind/pybind.cpp:1:
/opt/torchaudio/src/libtorio/ffmpeg/pybind/pybind.cpp: In function ‘int torio::io::{anonymous}::{anonymous}::read_func(void*, uint8_t*, int)’:
/opt/torchaudio/src/libtorio/ffmpeg/pybind/pybind.cpp:125:19: warning: comparison of integer expressions of different signedness: ‘long unsigned int’ and ‘int’ [-Wsign-compare]
  125 |         chunk_len <= request,
      |         ~~~~~~~~~~^~~~~~~~~~
In file included from /opt/torchaudio/build/temp.linux-aarch64-cpython-310/_deps/f6-src/include/libavutil/common.h:43,
                 from /opt/torchaudio/build/temp.linux-aarch64-cpython-310/_deps/f6-src/include/libavutil/avutil.h:296,
                 from /opt/torchaudio/build/temp.linux-aarch64-cpython-310/_deps/f6-src/include/libavcodec/avcodec.h:32,
                 from /opt/torchaudio/src/libtorio/ffmpeg/ffmpeg.h:10,
                 from /opt/torchaudio/src/libtorio/ffmpeg/hw_context.h:3,
                 from /opt/torchaudio/src/libtorio/ffmpeg/pybind/pybind.cpp:1:
/opt/torchaudio/src/libtorio/ffmpeg/pybind/pybind.cpp: In function ‘int torio::io::{anonymous}::read_bytes(void*, uint8_t*, int)’:
/opt/torchaudio/build/temp.linux-aarch64-cpython-310/_deps/f6-src/include/libavutil/macros.h:49:25: warning: comparison of integer expressions of different signedness: ‘std::basic_string_view<char>::size_type’ {aka ‘long unsigned int’} and ‘int’ [-Wsign-compare]
   49 | #define FFMIN(a,b) ((a) > (b) ? (b) : (a))
      |                     ~~~~^~~~~
/opt/torchaudio/src/libtorio/ffmpeg/pybind/pybind.cpp:202:19: note: in expansion of macro ‘FFMIN’
  202 |   auto num_read = FFMIN(wrapper->src.size() - wrapper->index, buf_size);
      |                   ^~~~~
[83/92] /usr/bin/c++ -DTORIO_FFMPEG_EXT_NAME=_torio_ffmpeg5 -DUSE_C10D_GLOO -DUSE_C10D_MPI -DUSE_C10D_NCCL -DUSE_CUDA -DUSE_DISTRIBUTED -DUSE_RPC -DUSE_TENSORPIPE -D_torio_ffmpeg5_EXPORTS -I/opt/torchaudio/src -I/usr/include/python3.10 -I/opt/torchaudio/build/temp.linux-aarch64-cpython-310/_deps/f5-src/include -isystem /usr/local/lib/python3.10/dist-packages/torch/include -isystem /usr/local/lib/python3.10/dist-packages/torch/include/torch/csrc/api/include -isystem /usr/local/cuda/include -Wall -D_GLIBCXX_USE_CXX11_ABI=1 -O3 -DNDEBUG -std=gnu++17 -fPIC -D_GLIBCXX_USE_CXX11_ABI=1 -MD -MT src/libtorio/ffmpeg/CMakeFiles/_torio_ffmpeg5.dir/pybind/pybind.cpp.o -MF src/libtorio/ffmpeg/CMakeFiles/_torio_ffmpeg5.dir/pybind/pybind.cpp.o.d -o src/libtorio/ffmpeg/CMakeFiles/_torio_ffmpeg5.dir/pybind/pybind.cpp.o -c /opt/torchaudio/src/libtorio/ffmpeg/pybind/pybind.cpp
In file included from /usr/local/lib/python3.10/dist-packages/torch/include/c10/util/Exception.h:5,
                 from /usr/local/lib/python3.10/dist-packages/torch/include/ATen/BlasBackend.h:3,
                 from /usr/local/lib/python3.10/dist-packages/torch/include/ATen/Context.h:3,
                 from /usr/local/lib/python3.10/dist-packages/torch/include/ATen/ATen.h:7,
                 from /usr/local/lib/python3.10/dist-packages/torch/include/torch/csrc/api/include/torch/types.h:3,
                 from /opt/torchaudio/src/libtorio/ffmpeg/ffmpeg.h:3,
                 from /opt/torchaudio/src/libtorio/ffmpeg/hw_context.h:3,
                 from /opt/torchaudio/src/libtorio/ffmpeg/pybind/pybind.cpp:1:
/opt/torchaudio/src/libtorio/ffmpeg/pybind/pybind.cpp: In function ‘int torio::io::{anonymous}::{anonymous}::read_func(void*, uint8_t*, int)’:
/opt/torchaudio/src/libtorio/ffmpeg/pybind/pybind.cpp:125:19: warning: comparison of integer expressions of different signedness: ‘long unsigned int’ and ‘int’ [-Wsign-compare]
  125 |         chunk_len <= request,
      |         ~~~~~~~~~~^~~~~~~~~~
In file included from /opt/torchaudio/build/temp.linux-aarch64-cpython-310/_deps/f5-src/include/libavutil/common.h:43,
                 from /opt/torchaudio/build/temp.linux-aarch64-cpython-310/_deps/f5-src/include/libavutil/avutil.h:296,
                 from /opt/torchaudio/build/temp.linux-aarch64-cpython-310/_deps/f5-src/include/libavutil/samplefmt.h:24,
                 from /opt/torchaudio/build/temp.linux-aarch64-cpython-310/_deps/f5-src/include/libavcodec/avcodec.h:30,
                 from /opt/torchaudio/src/libtorio/ffmpeg/ffmpeg.h:10,
                 from /opt/torchaudio/src/libtorio/ffmpeg/hw_context.h:3,
                 from /opt/torchaudio/src/libtorio/ffmpeg/pybind/pybind.cpp:1:
/opt/torchaudio/src/libtorio/ffmpeg/pybind/pybind.cpp: In function ‘int torio::io::{anonymous}::read_bytes(void*, uint8_t*, int)’:
/opt/torchaudio/build/temp.linux-aarch64-cpython-310/_deps/f5-src/include/libavutil/macros.h:49:25: warning: comparison of integer expressions of different signedness: ‘std::basic_string_view<char>::size_type’ {aka ‘long unsigned int’} and ‘int’ [-Wsign-compare]
   49 | #define FFMIN(a,b) ((a) > (b) ? (b) : (a))
      |                     ~~~~^~~~~
/opt/torchaudio/src/libtorio/ffmpeg/pybind/pybind.cpp:202:19: note: in expansion of macro ‘FFMIN’
  202 |   auto num_read = FFMIN(wrapper->src.size() - wrapper->index, buf_size);
      |                   ^~~~~
ninja: build stopped: subcommand failed.

Traceback (most recent call last):
  File "/opt/torchaudio/setup.py", line 143, in <module>
    _main()
  File "/opt/torchaudio/setup.py", line 99, in _main
    setup(
  File "/usr/local/lib/python3.10/dist-packages/setuptools/__init__.py", line 117, in setup
    return distutils.core.setup(**attrs)
  File "/usr/local/lib/python3.10/dist-packages/setuptools/_distutils/core.py", line 183, in setup
    return run_commands(dist)
  File "/usr/local/lib/python3.10/dist-packages/setuptools/_distutils/core.py", line 199, in run_commands
    dist.run_commands()
  File "/usr/local/lib/python3.10/dist-packages/setuptools/_distutils/dist.py", line 954, in run_commands
    self.run_command(cmd)
  File "/usr/local/lib/python3.10/dist-packages/setuptools/dist.py", line 950, in run_command
    super().run_command(command)
  File "/usr/local/lib/python3.10/dist-packages/setuptools/_distutils/dist.py", line 973, in run_command
    cmd_obj.run()
  File "/usr/local/lib/python3.10/dist-packages/setuptools/command/bdist_wheel.py", line 398, in run
    self.run_command("build")
  File "/usr/local/lib/python3.10/dist-packages/setuptools/_distutils/cmd.py", line 316, in run_command
    self.distribution.run_command(command)
  File "/usr/local/lib/python3.10/dist-packages/setuptools/dist.py", line 950, in run_command
    super().run_command(command)
  File "/usr/local/lib/python3.10/dist-packages/setuptools/_distutils/dist.py", line 973, in run_command
    cmd_obj.run()
  File "/usr/local/lib/python3.10/dist-packages/setuptools/_distutils/command/build.py", line 135, in run
    self.run_command(cmd_name)
  File "/usr/local/lib/python3.10/dist-packages/setuptools/_distutils/cmd.py", line 316, in run_command
    self.distribution.run_command(command)
  File "/usr/local/lib/python3.10/dist-packages/setuptools/dist.py", line 950, in run_command
    super().run_command(command)
  File "/usr/local/lib/python3.10/dist-packages/setuptools/_distutils/dist.py", line 973, in run_command
    cmd_obj.run()
  File "/opt/torchaudio/tools/setup_helpers/extension.py", line 99, in run
    super().run()
  File "/usr/local/lib/python3.10/dist-packages/setuptools/command/build_ext.py", line 98, in run
    _build_ext.run(self)
  File "/usr/local/lib/python3.10/dist-packages/Cython/Distutils/old_build_ext.py", line 186, in run
    _build_ext.build_ext.run(self)
  File "/usr/local/lib/python3.10/dist-packages/setuptools/_distutils/command/build_ext.py", line 359, in run
    self.build_extensions()
  File "/usr/local/lib/python3.10/dist-packages/Cython/Distutils/old_build_ext.py", line 195, in build_extensions
    _build_ext.build_ext.build_extensions(self)
  File "/usr/local/lib/python3.10/dist-packages/setuptools/_distutils/command/build_ext.py", line 476, in build_extensions
    self._build_extensions_serial()
  File "/usr/local/lib/python3.10/dist-packages/setuptools/_distutils/command/build_ext.py", line 502, in _build_extensions_serial
    self.build_extension(ext)
  File "/opt/torchaudio/tools/setup_helpers/extension.py", line 183, in build_extension
    subprocess.check_call(["cmake", "--build", "."] + build_args, cwd=self.build_temp)
  File "/usr/lib/python3.10/subprocess.py", line 369, in check_call
    raise CalledProcessError(retcode, cmd)
subprocess.CalledProcessError: Command '['cmake', '--build', '.', '--target', 'install']' returned non-zero exit status 1.
The command '/bin/sh -c /tmp/torchaudio/install.sh || /tmp/torchaudio/build.sh' returned a non-zero code: 1
Traceback (most recent call last):
  File "<frozen runpy>", line 198, in _run_module_as_main
  File "<frozen runpy>", line 88, in _run_code
  File "/home/johnny/Projects/jetson-containers/jetson_containers/build.py", line 112, in <module>
    build_container(args.name, args.packages, args.base, args.build_flags, args.build_args, args.simulate, args.skip_tests, args.test_only, args.push, args.no_github_api)
  File "/home/johnny/Projects/jetson-containers/jetson_containers/container.py", line 147, in build_container
    status = subprocess.run(cmd.replace(_NEWLINE_, ' '), executable='/bin/bash', shell=True, check=True)  
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/johnny/miniconda3/lib/python3.12/subprocess.py", line 571, in run
    raise CalledProcessError(retcode, process.args,
subprocess.CalledProcessError: Command 'sudo DOCKER_BUILDKIT=0 docker build --network=host --tag torchaudio:r36.3.0-cu126-cp310-torchaudio --file /home/johnny/Projects/jetson-containers/packages/pytorch/torchaudio/Dockerfile --build-arg BASE_IMAGE=torchaudio:r36.3.0-cu126-cp310-pytorch_2.4 --build-arg TORCHAUDIO_VERSION="2.4.0" /home/johnny/Projects/jetson-containers/packages/pytorch/torchaudio 2>&1 | tee /home/johnny/Projects/jetson-containers/logs/20240925_120647/build/torchaudio_r36.3.0-cu126-cp310-torchaudio.txt; exit ${PIPESTATUS[0]}' returned non-zero exit status 1.

Versions

Pytorch 2.4.0

@williamstar
Copy link

Not building cuda 12.5 too

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants