Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Nvbit support #2

Open
wants to merge 78 commits into
base: main
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
78 commits
Select commit Hold shift + click to select a range
91c1b31
White space and code reformatting
christopherscott0 Mar 1, 2024
0ec16a3
extracted normalize and create_spatter_file
christopherscott0 Mar 28, 2024
3c2b5e9
extracted get_top_target, update_source_lines & second_pass
christopherscott0 Mar 28, 2024
cffaf7b
Bug fixes and support trace file name handling improvements (doesnt r…
christopherscott0 Mar 30, 2024
156fe23
Fix sourceline lookup, json filename bug, added max line length define
christopherscott0 Mar 30, 2024
1f93c2f
Migrated to C++
christopherscott0 Mar 30, 2024
35c1c66
introduced Metrics class, partially used. tests pass.
christopherscott0 Mar 31, 2024
7ee76f7
Clean-ups and move some defines and Metrics to gs_patterns.h.
christopherscott0 Mar 31, 2024
36b962d
Refatored first pass and some required interfaces required carrying s…
christopherscott0 Apr 2, 2024
74570b6
Remove additional function statics for thread safety.
christopherscott0 Apr 2, 2024
aead2b4
Renamed some functions, removed unused vars, reordered update of stat…
christopherscott0 Apr 2, 2024
219fbe6
move file io to functions and fixup flush calls and error mesgs.
christopherscott0 Apr 2, 2024
d693277
Introduced GS exceptions
christopherscott0 Apr 2, 2024
37bca07
Split gs_patterns.cpp into a few headers and source files in prep for…
christopherscott0 Apr 2, 2024
9fa0dc6
Split static lib and binary (libgs_patterns_core.a and binary gs_patt…
christopherscott0 Apr 3, 2024
f90b736
Implemented MemPatterns abstract base class and MemPatternsForPin for…
christopherscott0 Apr 3, 2024
ded1409
Removed dependency on gzip files in gs_patterns_core.cpp, moved that …
christopherscott0 Apr 3, 2024
29d76ef
Removed use of .gz tracefile from gs_patterns_core.
christopherscott0 Apr 4, 2024
284cd99
Converted remaining functions used by MemPatternsForPin into member m…
christopherscott0 Apr 4, 2024
375e736
Implemented a proposed way to do second_pass.
christopherscott0 Apr 4, 2024
c5c16e0
Removed dependency on trace_entry_t in gs_patterns_core.cpp by provid…
christopherscott0 Apr 5, 2024
8ca5c83
Removed remaining references to trace_entry_t out of libgs_patterns.
christopherscott0 Apr 5, 2024
32c28d3
cleanup
christopherscott0 Apr 5, 2024
7fb401f
Renamed a few classes/files to improve readability.
christopherscott0 Apr 5, 2024
e5bd585
Removed remaining statics.
christopherscott0 Apr 5, 2024
88d7849
Simply some logic in libgs_patterns handle_trace_entry()
Apr 11, 2024
eb59f7a
mem_trace initial version
Apr 11, 2024
8983418
Link with libgs_patterns
Apr 11, 2024
8339707
Improved naming to beter descrive nvbit trace vs nvgs_patterns
christopherscott0 Apr 11, 2024
92af6f5
Add support for opcode_short_id, is_load, is_store and size from nvbi…
christopherscott0 Apr 11, 2024
40136b0
Use size, opcode_short_id, is_load() & is_store() rather than parsing…
Apr 11, 2024
abddae8
Ignore memory accesses w/ partial warps <non-functional>
christopherscott0 Apr 11, 2024
bdb3798
Exception handling around instrumental calls, log size <non-functional>
christopherscott0 Apr 12, 2024
6ce7e1f
Fix duplicate printf <non-functional>
Apr 12, 2024
00abeed
Support for reading and writing out trace files that includes headers…
Apr 13, 2024
de76408
fixes. <non-functional>
christopherscott0 Apr 13, 2024
d9a89e7
Config file can be used to set kernel, trace file and program binary,…
Apr 13, 2024
c410dbb
Merge remote-tracking branch 'origin/nvbit_support' into nvbit_support
Apr 13, 2024
0cfa257
Added a config setting for maximum trace count and setting a file pre…
Apr 15, 2024
2c1e43b
small fixes to handle max_trace_count
Apr 15, 2024
bba948a
Add support for source lines in nvbit traces(both reading and writing).
Apr 16, 2024
da046cd
Handle cases where --generate-line-info was not used to build kernel.
Apr 16, 2024
2d555f9
Fixed trace file read. Trace lines out now match trace lines read in …
christopherscott0 Apr 17, 2024
6966005
Formatting
christopherscott0 Apr 17, 2024
5860367
Improve accounting
Apr 17, 2024
46f9aae
Small improvements to redue vector copies and inline some methods.
Apr 18, 2024
56e1c31
Added ability to use tmpfile for intermediate traces used in second p…
christopherscott0 Apr 19, 2024
4f4ee11
Bug fixes. Error checking fixes. Bulk read of trace temp file. Progre…
christopherscott0 Apr 20, 2024
696fcb8
Expand PSize and fix breakout.
christopherscott0 Apr 20, 2024
1494f1d
Write trace file before 2nd pass rather than in destructor.
Apr 20, 2024
3eaa16f
Introduce instruction type, Improved numbers.
christopherscott0 Apr 20, 2024
a96b358
Renamed some address functions to better model. Fixes to Numbers, out…
christopherscott0 Apr 21, 2024
7a57d36
When not using vector for second pass, and not writing out trace file…
christopherscott0 Apr 23, 2024
99f594d
Combined runners into one for simplification, moved MemPatternsForPin…
Apr 24, 2024
a0b9993
Makefile moved.
christopherscott0 Apr 24, 2024
69db6da
If requested, write trace files out before generating patterns.
christopherscott0 Apr 24, 2024
7d848b2
cleanup
christopherscott0 Apr 24, 2024
9adbc8e
Add an option for verbose logging.
christopherscott0 Apr 24, 2024
07785ee
Fix shared lib linking, compiler warning. Remove extra calls to write…
christopherscott0 Apr 24, 2024
bf58ba3
Small cleanups.
Apr 26, 2024
6387353
Small Fixes for compiler warnings on macos.
Apr 26, 2024
8313506
gsnv_trace Readme
Apr 30, 2024
3fd1559
Description of config items.
Apr 30, 2024
cd7104d
Update readme for nvbit support.
christopherscott0 May 2, 2024
82c3581
Increasing PSIZE - previous size caused PSIZE exceeded when running U…
christopherscott0 May 2, 2024
5d8255d
Update log message.
christopherscott0 May 2, 2024
e259a0b
Formatting updates.
christopherscott0 May 15, 2024
2cb3778
Remove unneeded/unused allocation.
christopherscott0 Jun 21, 2024
7f112d7
Maximum memory access size is now templatized, can be different betwe…
christopherscott0 Jul 4, 2024
4046f35
Added determining of instruction address (iaddr) to traces generated …
Jul 6, 2024
1b2b150
Rename a method.
Jul 7, 2024
3ded852
When converting mem_access_t to trace_entry ensure addr_to_line uses …
Jul 11, 2024
011c45d
Bug fix: Use iaddr for mapping addr to line.
Jul 20, 2024
60a9672
Fix compiler warnings (unused variables, unreachable stmts etc)
christopherscott0 Jul 20, 2024
d95be9d
Support for building NVBit 1.7
Aug 3, 2024
79930c7
Merge remote-tracking branch 'origin/nvbit_support' into nvbit_support
Aug 3, 2024
8c9606e
Remove duplicate include settings
Aug 4, 2024
9316c5f
Fix another compiler warning
Aug 5, 2024
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
30 changes: 24 additions & 6 deletions CMakeLists.txt
Original file line number Diff line number Diff line change
@@ -1,9 +1,27 @@
cmake_minimum_required(VERSION 3.1...3.25)

project(
gs_patterns
VERSION 1.0
LANGUAGES C)
project( gs_patterns VERSION 1.0 LANGUAGES CXX)

add_executable(gs_patterns gs_patterns.c)
set (CMAKE_C_FLAGS "-lz -lm")
set(CMAKE_CXX_STANDARD 17)
set(CMAKE_CXX_STANDARD_REQUIRED On)
#set(CMAKE_CXX_EXTENSIONS Off)

add_library(gs_patterns_core SHARED
utils.h
utils.cpp
gs_patterns.h
gs_patterns_core.h
gs_patterns_core.cpp
gsnv_patterns.h
gsnv_patterns.cpp
gspin_patterns.h
gspin_patterns.cpp
)

add_executable( gs_patterns
gs_patterns_main.cpp
)

target_link_libraries(gs_patterns gs_patterns_core)

set(CMAKE_CXX_STANDARD_LIBRARIES "-lm -lz ${CMAKE_CXX_STANDARD_LIBRARIES}")
29 changes: 25 additions & 4 deletions README.md
Original file line number Diff line number Diff line change
@@ -1,5 +1,14 @@
# Description
Memory analysis tool for finding gather / scatter (gs) accesses from DynamoRio traces. gs_patterns discovers gather/scatters from analyzing access patterns in memory traces (doesn't just look for gs instructions). gs_patterns writes the "subtraces" to a binary trace and spatter yaml format. The source lines of the top aggressors are reported. Use the provided pin clients in the pin_tracing folder or use DynamoRio. Pin tends to be more reliable for larger applications.
Memory analysis tool for finding gather / scatter (gs) accesses from DynamoRio & NVBit traces.
gs_patterns discovers gather/scatters from analyzing access patterns in memory traces (doesn't just look for gs instructions). gs_patterns writes the "subtraces" to a binary trace and spatter yaml format.
The source lines of the top aggressors are reported.

For CPU applications use the provided pin client in pin_tracing folder (or DynamoRio). Pin tends to be more reliable for larger applications.

For CUDA kernels use the provided nvbit client in the nvbit_tracing folder.

See the README in the respective folders for more detailed information on these tools.


# Build
```
Expand All @@ -10,13 +19,25 @@ make
```

# Use

## For Pin/DynamoRio
```
gs_pattern <trace.gz> <binary>
gs_pattern <pin_trace.gz> <binary>
```
trace file should be gzipped. Binary file should be compiled with symbols turned on (-g)

## For NVBit (CUDA Kernels)

```
gs_pattern <nvbit_trace.gz> -nv
```

Trace file should be gzipped. For Pin or DynamoRio, Binary file should be compiled with symbols turned on (-g).

For NVBit tracing the kernel must be compiled with line numbers (--generate-line-info). Please see nvbit_tracing/README.md for detailed information on how to extract traces for CUDA kernels which are compatible with gs_patterns.

# How gs_patterns works
g/s accesses are found by looking at repeated instruction addresses (loops) that are memory instructions (scalar and vector). The first pass finds the top g/s's. The second pass focuses on the top g/s accesses and records the normalized address distances to a binary file and spatter yaml.
g/s accesses are found by looking at repeated instruction addresses (loops) that are memory instructions (scalar and vector).
The first pass finds the top g/s's. The second pass focuses on the top g/s accesses and records the normalized address distances to a binary file and spatter yaml.

# License
BSD-3 License. See [the LICENSE file](https://github.com/lanl/gs_patterns/blob/main/LICENSE).
Expand Down
Loading