Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat(profiling): add experimental outside-in stack sampling method (stack v2) #8471

Merged
merged 115 commits into from
Mar 5, 2024
Merged
Show file tree
Hide file tree
Changes from 5 commits
Commits
Show all changes
115 commits
Select commit Hold shift + click to select a range
31d079b
Refactor ddup into a peer directory
sanchda Feb 21, 2024
6d91204
Factor module more cleanly
sanchda Feb 21, 2024
c7f1cc4
Add V2 sampling code, some other minor refactors
sanchda Feb 21, 2024
1110483
Add v2 python code
sanchda Feb 21, 2024
4b363d7
Merge branch 'main' into sanchda/profiling_stack_v2
sanchda Feb 21, 2024
43ec683
Formatting
sanchda Feb 21, 2024
f667466
Fixup gitignore
sanchda Feb 21, 2024
c6317e8
Fixups
sanchda Feb 21, 2024
c207279
Stricter type conformance in ddup
sanchda Feb 23, 2024
3ca20fb
Fix incorrect counting of dropped frames
sanchda Feb 23, 2024
06d05b5
Make extensions optional
sanchda Feb 23, 2024
01a026b
Various cleanups to builds
sanchda Feb 23, 2024
8a708d5
First cut at string_viewifying
sanchda Feb 26, 2024
e76e8d0
Add more string_view stuff
sanchda Feb 26, 2024
6539f49
Update stackv2 to use new string_view interface
sanchda Feb 26, 2024
1dd33e0
stackv2 required, fix comment
sanchda Feb 26, 2024
b5c4976
Clarifying comment
sanchda Feb 26, 2024
9e7f184
Comment clarifying range
sanchda Feb 26, 2024
73045bb
More comments in setup_custom
sanchda Feb 26, 2024
f5eaa25
Remove stale comment
sanchda Feb 26, 2024
c3ff98f
Minor changes to build system
sanchda Feb 26, 2024
bc16174
Try to fix the broken resource paths
sanchda Feb 27, 2024
cf56f23
Fix build, respond to comments
sanchda Feb 27, 2024
904bbff
Streamline init
sanchda Feb 27, 2024
a26f037
Merge branch 'main' into sanchda/profiling_stack_v2
sanchda Feb 27, 2024
749d94c
Style checks, releasenotes
sanchda Feb 27, 2024
eeda82c
Merge branch 'sanchda/profiling_stack_v2' of github.com:DataDog/dd-tr…
sanchda Feb 27, 2024
e5043ed
Merge branch 'main' into sanchda/profiling_stack_v2
sanchda Feb 27, 2024
ca27591
Make libdd and py mutually exclusive
sanchda Feb 27, 2024
c1c2d7b
Merge branch 'sanchda/profiling_stack_v2' of github.com:DataDog/dd-tr…
sanchda Feb 27, 2024
26225c7
Merge branch 'main' into sanchda/profiling_stack_v2
sanchda Feb 27, 2024
5198cc1
Style and fixups
sanchda Feb 27, 2024
2d6dd5a
Kludge to help expose some info in CI
sanchda Feb 27, 2024
4cfe139
Merge branch 'main' into sanchda/profiling_stack_v2
sanchda Feb 27, 2024
7cb4e31
Fix typo
sanchda Feb 27, 2024
e2912dc
Fix warnings
sanchda Feb 27, 2024
4948cae
Get better visibility into smoketest breaks
sanchda Feb 27, 2024
78c1ac2
Merge branch 'main' into sanchda/profiling_stack_v2
sanchda Feb 27, 2024
98c7353
Roll back temp measures, update changelog
sanchda Feb 27, 2024
12960ba
Merge branch 'sanchda/profiling_stack_v2' of github.com:DataDog/dd-tr…
sanchda Feb 27, 2024
d606fbb
Refactor interval stuff
sanchda Feb 27, 2024
e819ac4
Merge branch 'main' into sanchda/profiling_stack_v2
sanchda Feb 27, 2024
8aff7b2
Move python declarations back into cmake harness
sanchda Feb 28, 2024
6fec0fc
Merge branch 'main' into sanchda/profiling_stack_v2
sanchda Feb 28, 2024
d4d86df
Const correctness, formatting
sanchda Feb 28, 2024
b1704e0
Merge branch 'main' into sanchda/profiling_stack_v2
sanchda Feb 28, 2024
50c06d8
style
sanchda Feb 28, 2024
0c77568
Merge branch 'sanchda/profiling_stack_v2' of github.com:DataDog/dd-tr…
sanchda Feb 28, 2024
4ca6a22
cppcheck cleanup
sanchda Feb 28, 2024
8d32bf8
Merge branch 'main' into sanchda/profiling_stack_v2
sanchda Feb 28, 2024
acf0459
Profiling interface fixup
sanchda Feb 28, 2024
bc7c466
Merge branch 'sanchda/profiling_stack_v2' of github.com:DataDog/dd-tr…
sanchda Feb 28, 2024
d5a75f8
Merge branch 'main' into sanchda/profiling_stack_v2
sanchda Feb 28, 2024
50f4d48
Merge branch 'sanchda/profiling_stack_v2' of github.com:DataDog/dd-tr…
sanchda Feb 28, 2024
e2e5237
Make engaging stack_v2 safer
sanchda Feb 28, 2024
f3e186f
Merge branch 'main' into sanchda/profiling_stack_v2
sanchda Feb 28, 2024
bd2bb65
Fix test, oops
sanchda Feb 28, 2024
d07da07
Merge branch 'sanchda/profiling_stack_v2' of github.com:DataDog/dd-tr…
sanchda Feb 28, 2024
2702915
Merge branch 'main' into sanchda/profiling_stack_v2
sanchda Feb 28, 2024
ed57403
Style
sanchda Feb 28, 2024
ef15efa
Merge branch 'sanchda/profiling_stack_v2' of github.com:DataDog/dd-tr…
sanchda Feb 28, 2024
2cecac5
Fix telemetry test
sanchda Feb 28, 2024
04f914e
Update ddtrace/profiling/collector/memalloc.py
sanchda Feb 29, 2024
f1114d9
Update ddtrace/internal/datadog/profiling/stack_v2/include/util/cast_…
sanchda Feb 29, 2024
86d02cb
Update ddtrace/settings/profiling.py
sanchda Feb 29, 2024
eeedd86
Merge branch 'main' into sanchda/profiling_stack_v2
sanchda Feb 29, 2024
a9c2935
Add import
sanchda Feb 29, 2024
62b1efd
Merge branch 'main' into sanchda/profiling_stack_v2
sanchda Feb 29, 2024
86d46b8
Off-by-one lol
sanchda Feb 29, 2024
99d5c8e
Try to fix the cmake config
sanchda Feb 29, 2024
4f354e2
Merge branch 'sanchda/profiling_stack_v2' of github.com:DataDog/dd-tr…
sanchda Feb 29, 2024
1ded848
Merge branch 'main' into sanchda/profiling_stack_v2
sanchda Feb 29, 2024
31a28e9
Hack: add binary dir to add_sub
sanchda Feb 29, 2024
761c032
Merge branch 'sanchda/profiling_stack_v2' of github.com:DataDog/dd-tr…
sanchda Feb 29, 2024
1d16297
Fixups for cmake, tests
sanchda Feb 29, 2024
57b801f
Merge branch 'main' into sanchda/profiling_stack_v2
sanchda Feb 29, 2024
4450868
Some fixups
sanchda Mar 1, 2024
b12856c
Merge branch 'main' into sanchda/profiling_stack_v2
sanchda Mar 1, 2024
cef8843
Fix silly package_data bug
sanchda Mar 1, 2024
0fdcf2c
Merge branch 'sanchda/profiling_stack_v2' of github.com:DataDog/dd-tr…
sanchda Mar 1, 2024
e9bcce6
Add module, comment
sanchda Mar 1, 2024
cc71c35
init all the things
sanchda Mar 1, 2024
cd95027
Tighten things up
sanchda Mar 1, 2024
ee8e4d7
Predict whether initialization will be required in ddup
sanchda Mar 1, 2024
2bc2c61
Invert config forcing
sanchda Mar 1, 2024
d6a4f1a
Update release note
sanchda Mar 1, 2024
71180c4
Comment fixups, make sofile propagation tighter
sanchda Mar 1, 2024
ee925f2
Merge branch 'main' into sanchda/profiling_stack_v2
sanchda Mar 1, 2024
0c33656
Various
sanchda Mar 1, 2024
2da8782
Merge branch 'main' into sanchda/profiling_stack_v2
sanchda Mar 1, 2024
6c381f9
Slight refactor to help package_data work
sanchda Mar 1, 2024
50b24b0
Fix slotscheck issue
sanchda Mar 1, 2024
6b85fea
Path fix in slotscheck
sanchda Mar 1, 2024
c1b2b95
Downgrade module error to warning
sanchda Mar 1, 2024
1de4559
Further downgrade to debug
sanchda Mar 1, 2024
de03082
Clarifying comment
sanchda Mar 1, 2024
29029f6
Make native extensions all optional
sanchda Mar 1, 2024
bfa096f
Update releasenotes/notes/profiling-add-stack-v2-975ebe845cdadc8f.yaml
sanchda Mar 4, 2024
a07efd9
Update releasenotes/notes/profiling-add-stack-v2-975ebe845cdadc8f.yaml
sanchda Mar 4, 2024
af937a7
Update releasenotes/notes/profiling-add-stack-v2-975ebe845cdadc8f.yaml
sanchda Mar 4, 2024
6d56227
Update setup.py
sanchda Mar 4, 2024
4c256a6
Merge branch 'main' into sanchda/profiling_stack_v2
sanchda Mar 4, 2024
0faef83
Merge branch 'main' into sanchda/profiling_stack_v2
sanchda Mar 4, 2024
9e9d0dc
Merge branch 'main' into sanchda/profiling_stack_v2
sanchda Mar 5, 2024
ad1e0cc
Update ddtrace/internal/datadog/profiling/stack_v2/src/stack_v2.cpp
sanchda Mar 5, 2024
5da44b0
Update ddtrace/internal/datadog/profiling/stack_v2/src/stack_v2.cpp
sanchda Mar 5, 2024
5aee9c0
Update profiling.py
sanchda Mar 5, 2024
30ec823
Remove eager config checks
sanchda Mar 5, 2024
0b1e9ee
Merge branch 'main' into sanchda/profiling_stack_v2
sanchda Mar 5, 2024
1fa55c8
Make is_available a module attribute
sanchda Mar 5, 2024
432b5d5
Merge branch 'sanchda/profiling_stack_v2' of github.com:DataDog/dd-tr…
sanchda Mar 5, 2024
24489c6
Remove now-unneeded deps from profiler settings
sanchda Mar 5, 2024
ab53e5d
Merge branch 'main' into sanchda/profiling_stack_v2
sanchda Mar 5, 2024
cc8c1b8
Merge branch 'main' into sanchda/profiling_stack_v2
sanchda Mar 5, 2024
be9fbbe
Merge branch 'main' into sanchda/profiling_stack_v2
sanchda Mar 5, 2024
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Empty file.
30 changes: 30 additions & 0 deletions ddtrace/internal/datadog/profiling/cmake/AnalysisFunc.cmake
Original file line number Diff line number Diff line change
@@ -0,0 +1,30 @@
include(FindCppcheck)

function(add_ddup_config target)
target_compile_options(${target} PRIVATE
"$<$<CONFIG:Debug>:-Og -ggdb3>"
"$<$<CONFIG:Release>:-Os>"
-ffunction-sections -fno-semantic-interposition -Wall -Werror -Wextra -Wshadow -Wnon-virtual-dtor -Wold-style-cast
)
target_link_options(${target} PRIVATE
"$<$<CONFIG:Release>:-s>"
-Wl,--as-needed -Wl,-Bsymbolic-functions -Wl,--gc-sections
)
set_property(TARGET ${target} PROPERTY INTERPROCEDURAL_OPTIMIZATION TRUE)

# Propagate sanitizers
if (SANITIZE_OPTIONS)
# Some sanitizers (or the analysis--such as symbolization--tooling thereof) work better with frame
# pointers, so we include it here.
target_compile_options(${target} PRIVATE -fsanitize=${SANITIZE_OPTIONS} -fno-omit-frame-pointer)
target_link_options(${target} PRIVATE -fsanitize=${SANITIZE_OPTIONS})
endif()

# If DO_FANALYZER is specified and we're using gcc, then we can use -fanalyzer
if (DO_FANALYZER AND CMAKE_CXX_COMPILER_ID MATCHES "GNU")
target_compile_options(${target} PRIVATE -fanalyzer)
endif()

# If DO_CPPCHECK is specified, then we can use cppcheck
add_cppcheck_target(cppcheck_dd_${target} ${CMAKE_CURRENT_SOURCE_DIR})
endfunction()
Original file line number Diff line number Diff line change
@@ -1,7 +1,14 @@
# Only add this project if Datadog::Profiling is not already defined
if(TARGET Datadog::Profiling)
return()
endif()

include(ExternalProject)
set(TAG_LIBDATADOG
"v5.0.0"
CACHE STRING "libdatadog github tag")

set(Datadog_BUILD_DIR ${CMAKE_CURRENT_BINARY_DIR}/libdatadog)
set(Datadog_ROOT ${Datadog_BUILD_DIR}/libdatadog-${TAG_LIBDATADOG})

message(STATUS "${CMAKE_CURRENT_LIST_DIR}/tools/fetch_libdatadog.sh ${TAG_LIBDATADOG} ${Datadog_ROOT}")
Expand Down
64 changes: 12 additions & 52 deletions ddtrace/internal/datadog/profiling/dd_wrapper/CMakeLists.txt
Original file line number Diff line number Diff line change
Expand Up @@ -4,36 +4,17 @@ project(dd_wrapper
LANGUAGES CXX
)

# Infer some basic properties about the build. This is necessary because multiple
# extensions reuse this cmake file, so we need its artifacts to go in a consistent
# place
get_filename_component(dd_wrapper_BUILD_PARENT "${CMAKE_BINARY_DIR}/../.." ABSOLUTE)
set(dd_wrapper_BUILD_DIR "${dd_wrapper_BUILD_PARENT}/ddtrace.internal.datadog.profiling")
# Build in a predictable location. This is needed for setup.py
get_filename_component(dd_wrapper_BUILD_DIR "${CMAKE_CURRENT_SOURCE_DIR}/../ddtrace.internal.datadog.profiling" ABSOLUTE)

# Custom modules are in the parent directory
list(APPEND CMAKE_MODULE_PATH "${CMAKE_CURRENT_SOURCE_DIR}/../cmake")

# Setup state for FindLibdatadog
set(Datadog_BUILD_DIR "${dd_wrapper_BUILD_PARENT}/libdatadog")

# Includes
include(CMakePackageConfigHelpers) # For generating the config file
include(FetchContent)
include(ExternalProject)
include(FindLibdatadog)
include(FindCppcheck)

# Send some of these vars back up to the parent
if (DD_WRAPPER_IS_SUBPROJECT)
set(Datadog_INCLUDE_DIRS "${Datadog_INCLUDE_DIRS}" PARENT_SCOPE)
set(dd_wrapper_BUILD_DIR "${dd_wrapper_BUILD_PARENT}/ddtrace.internal.datadog.profiling" PARENT_SCOPE)
endif()

add_compile_options(
"$<$<CONFIG:Debug>:-Og;-ggdb3>"
"$<$<CONFIG:Release>:-Os -s>"
-fno-semantic-interposition -Wall -Werror -Wextra -Wshadow -Wnon-virtual-dtor -Wold-style-cast
)
include(AnalysisFunc)

# Library sources
add_library(dd_wrapper SHARED
Expand All @@ -45,19 +26,17 @@ add_library(dd_wrapper SHARED
src/interface.cpp
)

# Add common configuration flags
add_ddup_config(dd_wrapper)

# At present, C++17 is chosen as the minimum standard. This conflicts with the
# manylinux2014 standard upon which we must rely. We'll have to statically link
# libstdc++ to avoid this conflict, but need to remain mindful of symbol visibility
# and overall binary size.
target_compile_features(dd_wrapper PUBLIC cxx_std_17)

# Set some linker options
target_compile_options(dd_wrapper PRIVATE -fno-semantic-interposition)
target_compile_options(dd_wrapper PRIVATE -ffunction-sections -fdata-sections)
# Statically link libstdc++ to avoid manylinux2014 conflicts
target_link_options(dd_wrapper PRIVATE -static-libstdc++)
target_link_options(dd_wrapper PRIVATE -Wl,-Bsymbolic-functions)
target_link_options(dd_wrapper PRIVATE -Wl,--gc-sections)
set_property(TARGET dd_wrapper PROPERTY INTERPROCEDURAL_OPTIMIZATION TRUE)

target_include_directories(dd_wrapper PRIVATE
include
Expand All @@ -73,32 +52,13 @@ set_target_properties(dd_wrapper
LIBRARY_OUTPUT_DIRECTORY "${dd_wrapper_BUILD_DIR}"
)

# If DO_FANALYZER is specified and we're using gcc, then we can use -fanalyzer
if (DO_FANALYZER AND CMAKE_CXX_COMPILER_ID MATCHES "GNU")
target_compile_options(dd_wrapper PRIVATE -fanalyzer)
# If LIB_INSTALL_DIR is set, install the library
if (LIB_INSTALL_DIR)
install(TARGETS dd_wrapper
DESTINATION "${LIB_INSTALL_DIR}"
)
endif()

# If DO_CPPCHECK is specified, then we can use cppcheck
add_cppcheck_target(cppcheck_dd_wrapper ${CMAKE_CURRENT_SOURCE_DIR})

# Propagate sanitizers
if (SANITIZE_OPTIONS)
# Some sanitizers (or the analysis--such as symbolization--tooling thereof) work better with frame
# pointers, so we include it here.
target_compile_options(dd_wrapper PRIVATE -fsanitize=${SANITIZE_OPTIONS} -fno-omit-frame-pointer)
target_link_options(dd_wrapper PRIVATE -fsanitize=${SANITIZE_OPTIONS})
endif()

# If dd_wrapper_INSTALL_DIR is specified by the parent, use it
if (NOT dd_wrapper_INSTALL_DIR)
# If not, then just keep it in the build directory
set(dd_wrapper_INSTALL_DIR "${CMAKE_BINARY_DIR}/lib")
endif()
message(STATUS "dd_wrapper_INSTALL_DIR: ${dd_wrapper_INSTALL_DIR}")
install(TARGETS dd_wrapper
DESTINATION ${dd_wrapper_INSTALL_DIR}
)

# Add the tests
if (BUILD_TESTING)
enable_testing()
Expand Down
Loading
Loading