bump tracy to wolfpld/tracy@5479a42 #19801

dezhiAmd · 2025-01-24T00:30:50Z

Affter bump tracy to wolfpld/tracy@5479a42, the trace captured must be opened by the profiler built from iree\third_party\tracy

Here is the instructions on how to build it on windows:

Copy python.exe to python3.exe (they should be in the same folder like c:\python312)

Go to iree\\third_party\\tracy, run below two commands

  cmake -B profiler/build -S profiler -DCMAKE_BUILD_TYPE=Release
  cmake --build profiler/build --parallel --config Release

After that your profiler version is 0.11.2

Tracy-csvexport can export gpu-zones to csv file

Go to iree\\third_party\\tracy, run below two commands

  cmake -B csvexport/build -S profiler -DCMAKE_BUILD_TYPE=Release
  cmake --build csvexport/build --parallel --config Release

After that run tracy-csvexport like:
tracy-csvexport -g test.tracy > test.csv

Signed-off-by: Liao <[email protected]>

ScottTodd · 2025-01-24T00:47:13Z

third_party/tracy

Clarifying the PR title:

v0.11.1 would be https://github.com/wolfpld/tracy/releases/tag/v0.11.1 (wolfpld/tracy@5d542dc)

This is changing the commit to wolfpld/tracy@5479a42, which is the latest on main and has not yet been bundled into a stable release. See the rolling patch notes: https://github.com/wolfpld/tracy/blob/5479a42ef9346b64e6d1b860ae58aa8abdb0c7f6/NEWS#L5-L56

Updating to a commit that is not yet part of a stable release is okay, but please note that in the PR text. It would also be good to call out if this unreleased version is binary-compatible with v0.11.1, or else our developers would need to build from source vs being able to download prebuilt binaries on Windows from https://github.com/wolfpld/tracy/releases/tag/v0.11.1.

Also note that our developers can still build the standalone Tracy tools (profiler GUI, csv export tool, etc.) from the original Tracy source, without us needing to update the submodule version here, unless there has been a breaking change to the client/server protocol. Staying current and giving easy access to the code via the submodule is nice though.

Indeed, there is a breaking change to the client/server protocol. That is what I see when using profiler (v0.11.1) to capture current IREE trace

ScottTodd · 2025-01-24T00:50:50Z

third_party/tracy

Looks like some changes will be needed on our end for this: https://github.com/iree-org/iree/actions/runs/12940755493/job/36095982640?pr=19801#step:9:1832

[396/571] Building CXX object runtime/src/iree/hal/drivers/vulkan/CMakeFiles/iree_hal_drivers_vulkan_dynamic_symbols.objects.dir/dynamic_symbols.cc.o [397/571] Building CXX object tracy/CMakeFiles/IREETracyServer.dir/__/__/__/third_party/tracy/server/TracyTextureCompression.cpp.o FAILED: tracy/CMakeFiles/IREETracyServer.dir/__/__/__/third_party/tracy/server/TracyTextureCompression.cpp.o ccache /usr/bin/clang++ -I/home/runner/_work/iree/iree/build_tools/third_party/tracy/../../../third_party/tracy/imgui -I/usr/include/capstone -I/usr/include/capstone/capstone -O3 -DNDEBUG -std=gnu++17 -fPIC -Wno-unused-result -MD -MT tracy/CMakeFiles/IREETracyServer.dir/__/__/__/third_party/tracy/server/TracyTextureCompression.cpp.o -MF tracy/CMakeFiles/IREETracyServer.dir/__/__/__/third_party/tracy/server/TracyTextureCompression.cpp.o.d -o tracy/CMakeFiles/IREETracyServer.dir/__/__/__/third_party/tracy/server/TracyTextureCompression.cpp.o -c /home/runner/_work/iree/iree/third_party/tracy/server/TracyTextureCompression.cpp In file included from /home/runner/_work/iree/iree/third_party/tracy/server/TracyTextureCompression.cpp:3: In file included from /home/runner/_work/iree/iree/third_party/tracy/server/TracyEvent.hpp:12: In file included from /home/runner/_work/iree/iree/third_party/tracy/server/TracySortedVector.hpp:4: /home/runner/_work/iree/iree/third_party/tracy/server/TracySort.hpp:7:12: fatal error: 'ppqsort.h' file not found 7 | # include <ppqsort.h> | ^~~~~~~~~~~

Maybe in https://github.com/iree-org/iree/blob/main/build_tools/third_party/tracy/CMakeLists.txt ?

See also the release note callout for

Parallel sorting is now performed with PPQSort (which removes potential
dependency on TBB).

Hopefully we don't need to install that on our build systems. If so, we could swap TBB for PPQ?

https://github.com/iree-org/base-docker-images/blob/2cf522e9cd4bf1e2157bd9583f9866bef52e4a60/dockerfiles/manylinux_x86_64.Dockerfile#L9

https://github.com/iree-org/base-docker-images/blob/2cf522e9cd4bf1e2157bd9583f9866bef52e4a60/dockerfiles/cpubuilder_ubuntu_jammy.Dockerfile#L19

I'd look around here:

https://github.com/wolfpld/tracy/blob/5479a42ef9346b64e6d1b860ae58aa8abdb0c7f6/cmake/server.cmake#L32

https://github.com/wolfpld/tracy/blob/5479a42ef9346b64e6d1b860ae58aa8abdb0c7f6/cmake/vendor.cmake#L248-L255

iree/build_tools/third_party/tracy/CMakeLists.txt

Lines 28 to 35 in dd31890

# Deps slightly differ by platform but some are common.

pkg_check_modules(TRACY_DEPS

tbb

libzstd

)

pkg_check_modules(TRACY_CAPSTONE_DEPS

capstone

)

Looks like a source dependency on https://github.com/GabTux/PPQSort. Maybe we'd want to FetchContent that (or use a submodule like we already do for Tracy).

I am looking for how CMake files in tracy automatically downloaded all dependencies into folders like this:
C:\develop\tracy_src\profiler\build\.cpm-cache\capstone\c68db851083fcffa27a786090cedcce4c4f28330\include\capstone;
C:\develop\tracy_src\profiler\build\.cpm-cache\capstone\c68db851083fcffa27a786090cedcce4c4f28330\include;
C:\develop\tracy_src\profiler\build\.cpm-cache\ppqsort\b30541008a1455585354665a90f3f90ebc8d637e\include;
C:\develop\tracy_src\profiler\build\_deps\ppqsort-build\PackageProjectInclude;
C:\develop\tracy_src\profiler\build\.cpm-cache\imgui\64e6627cd30cedd5fb11399ce199c6bc4c3cecaf;
C:\develop\tracy_src\profiler\build\_deps\freetype-build\include;
C:\develop\tracy_src\profiler\build\.cpm-cache\freetype\e30ced2c97047bd2dfa8aa235a47683755a48043\include;
C:\develop\tracy_src\profiler\build\.cpm-cache\glfw\9fc60ad2c810b92f74235a589a9a0edee75bc2d3\include;

See the files I linked. We maintain our own CMake file for the Tracy server since depending on Tracy's CMake project would bring extra baggage along (more history there on #17488). I suspect that using FetchContent for PPQSort would make sense. We have a few examples in the repo of how that can be done.

It sounds like we can also drop TBB from our dockerfiles and docs (https://iree.dev/developers/performance/profiling-with-tracy/#linux) thanks to wolfpld/tracy@1c1faef, once this lands.

A change is applied to build_tools/third_party/tracy/CMakeLists.txt.
Much appreciated the help. And you are correct, we do not need TBB anymore.
A follow up PR might be needed to update the document and the submodules.

Signed-off-by: Liao <[email protected]>

ScottTodd · 2025-01-27T19:28:51Z

build_tools/third_party/tracy/CMakeLists.txt

+include(${TRACY_SOURCE_DIR}/cmake/config.cmake)
+include(${TRACY_SOURCE_DIR}/cmake/vendor.cmake)
+include(${TRACY_SOURCE_DIR}/cmake/server.cmake)


It looks like you entirely rewrote this file to be a fork of the upstream CMake file at https://github.com/wolfpld/tracy/blob/master/capture/CMakeLists.txt. When I looked at these files while working on #17488, I decided against such an inclusion because of files like https://github.com/wolfpld/tracy/blob/master/cmake/config.cmake and https://github.com/wolfpld/tracy/blob/master/cmake/vendor.cmake setting CMake settings and downloading packages that could affect the rest of the project. These might be scoped to the current directory and below, but introducing that complexity/risk didn't seem worth it to me.

There are now at least two things this PR is doing:

Reworking the CMake file to copy upstream

Updating the commit/version used

I'd focus on just one of those at a time.

For the build reworking, we could make another attempt to directly use the upstream CMake files without forking then further change how IREE_BUILD_TRACY is used and how we bundle the capture server into releases with code here:

iree/CMakeLists.txt

Line 81 in 9a34131

option(IREE_BUILD_TRACY "Enables building the 'iree-tracy-capture' CLI tool and includes it in runtime Python bindings." OFF)

iree/CMakeLists.txt

Lines 973 to 981 in 9a34131

if(IREE_BUILD_TRACY)

if(NOT CMAKE_SYSTEM_NAME MATCHES "Darwin|Linux")

message(WARNING "Building Tracy (IREE_BUILD_TRACY) on non-Darwin/Linux is unsupported and may fail below.")

endif()

add_subdirectory(build_tools/third_party/tracy ${CMAKE_CURRENT_BINARY_DIR}/tracy)

if(NOT TARGET IREETracyCaptureServer)

message(SEND_ERROR "Could not build Tracy. Either unset IREE_BUILD_TRACY or look for missing dependencies above and install them.")

endif()

endif()

iree/runtime/bindings/python/CMakeLists.txt

Lines 10 to 14 in 9a34131

if(IREE_BUILD_TRACY)

message(STATUS "Bundling Tracy CLI tools with Python API")

set(_TRACY_ENABLED ON)

list(APPEND _EXTRA_INSTALL_TOOL_TARGETS "IREETracyCaptureServer")

endif()

iree/runtime/setup.py

Lines 436 to 437 in 9a34131

if ENABLE_TRACY_TOOLS:

cmake_args.append("-DIREE_BUILD_TRACY=ON")

iree/runtime/setup.py

Lines 84 to 86 in 9a34131

ENABLE_TRACY_TOOLS = getenv_bool(

"IREE_RUNTIME_BUILD_TRACY_TOOLS", "@IREE_RUNTIME_BUILD_TRACY_TOOLS@"

)

iree/build_tools/python_deploy/build_linux_packages.sh

Line 161 in 9a34131

export IREE_RUNTIME_BUILD_TRACY_TOOLS=ON

We could also stop distributing iree-tracy-capture and always point users to the upstream instructions. I'm not sure how much use we're getting out of our own packaging of the tool. Or, if we want to keep distributing it but not carry all that build system plumbing code, we could have that setup.py file run a standalone cmake build for Tracy and then copy the output file over.

There are a few directions we could go, and I don't have a clear preference right now. If you can keep this update PR incremental (without changing the entire CMake setup), that would be much easier for now.

ScottTodd · 2025-01-27T19:36:47Z

build_tools/third_party/tracy/CMakeLists.txt

-# Server library
-#-------------------------------------------------------------------------------
+include(${TRACY_SOURCE_DIR}/cmake/config.cmake)
+include(${TRACY_SOURCE_DIR}/cmake/vendor.cmake)


To troubleshoot errors like https://github.com/iree-org/iree/actions/runs/12968320277/job/36176956225?pr=19801#step:9:1526,

See the Tracy manual (.pdf file) for what requirements Tracy has

Build using the relevant dockerfile, ghcr.io/iree-org/manylinux_x86_64@sha256:2e0246137819cf10ed84240a971f9dd75cc3eb62dc6907dfd2080ee966b3c9f4 with source https://github.com/iree-org/base-docker-images/blob/main/dockerfiles/manylinux_x86_64.Dockerfile in this case. The https://github.com/iree-org/base-docker-images/ README has instructions on how to work with dockerfiles and this repository pins specific versions of the images like here:

iree/.github/workflows/pkgci_build_packages.yml

Line 19 in 9a34131

MANYLINUX_DOCKER_IMAGE: ghcr.io/iree-org/manylinux_x86_64@sha256:2e0246137819cf10ed84240a971f9dd75cc3eb62dc6907dfd2080ee966b3c9f4

benvanik

We should drop the cmake changes here - bumping tracy should be independent from rewriting our build. We're losing functionality with these changes and not gaining much: if a user wants to do an upstream build then they shouldn't be using our cmake so matching the upstream cmake does not provide value in and of itself.

bump tracy to 0.11.1

c3b469a

Signed-off-by: Liao <[email protected]>

dezhiAmd marked this pull request as ready for review January 24, 2025 00:31

dezhiAmd requested review from ScottTodd and stellaraccident as code owners January 24, 2025 00:31

ScottTodd reviewed Jan 24, 2025

View reviewed changes

ScottTodd requested changes Jan 24, 2025

View reviewed changes

dezhiAmd changed the title ~~bump tracy to 0.11.1~~ bump tracy to wolfpld/tracy@5479a42 Jan 24, 2025

Rework CMakeLists.txt for tracy to fix compile error

095e5f8

Signed-off-by: Liao <[email protected]>

dezhiAmd requested a review from ScottTodd January 26, 2025 19:41

ScottTodd reviewed Jan 27, 2025

View reviewed changes

benvanik requested changes Jan 27, 2025

View reviewed changes

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

bump tracy to wolfpld/tracy@5479a42 #19801

bump tracy to wolfpld/tracy@5479a42 #19801

dezhiAmd commented Jan 24, 2025 •

edited

Loading

ScottTodd Jan 24, 2025

ScottTodd Jan 24, 2025

dezhiAmd Jan 24, 2025

ScottTodd Jan 24, 2025 •

edited

Loading

ScottTodd Jan 24, 2025

dezhiAmd Jan 24, 2025 •

edited

Loading

ScottTodd Jan 24, 2025

dezhiAmd Jan 25, 2025

ScottTodd Jan 27, 2025

ScottTodd Jan 27, 2025

benvanik left a comment

	# Deps slightly differ by platform but some are common.
	pkg_check_modules(TRACY_DEPS
	tbb
	libzstd
	)
	pkg_check_modules(TRACY_CAPSTONE_DEPS
	capstone
	)

	if(IREE_BUILD_TRACY)
	if(NOT CMAKE_SYSTEM_NAME MATCHES "Darwin\|Linux")
	message(WARNING "Building Tracy (IREE_BUILD_TRACY) on non-Darwin/Linux is unsupported and may fail below.")
	endif()
	add_subdirectory(build_tools/third_party/tracy ${CMAKE_CURRENT_BINARY_DIR}/tracy)
	if(NOT TARGET IREETracyCaptureServer)
	message(SEND_ERROR "Could not build Tracy. Either unset IREE_BUILD_TRACY or look for missing dependencies above and install them.")
	endif()
	endif()

	if(IREE_BUILD_TRACY)
	message(STATUS "Bundling Tracy CLI tools with Python API")
	set(_TRACY_ENABLED ON)
	list(APPEND _EXTRA_INSTALL_TOOL_TARGETS "IREETracyCaptureServer")
	endif()

	if ENABLE_TRACY_TOOLS:
	cmake_args.append("-DIREE_BUILD_TRACY=ON")

	ENABLE_TRACY_TOOLS = getenv_bool(
	"IREE_RUNTIME_BUILD_TRACY_TOOLS", "@IREE_RUNTIME_BUILD_TRACY_TOOLS@"
	)

bump tracy to wolfpld/tracy@5479a42 #19801

Are you sure you want to change the base?

bump tracy to wolfpld/tracy@5479a42 #19801

Conversation

dezhiAmd commented Jan 24, 2025 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

ScottTodd Jan 24, 2025 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

dezhiAmd Jan 24, 2025 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

benvanik left a comment

Choose a reason for hiding this comment

dezhiAmd commented Jan 24, 2025 •

edited

Loading

ScottTodd Jan 24, 2025 •

edited

Loading

dezhiAmd Jan 24, 2025 •

edited

Loading