Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[SYCL][DeviceSanitizer] Checking "sycl::free" related errors #12882

Merged
merged 109 commits into from
Apr 19, 2024
Merged
Show file tree
Hide file tree
Changes from 108 commits
Commits
Show all changes
109 commits
Select commit Hold shift + click to select a range
3d7882b
init
AllanZyne Dec 26, 2023
7bf11b1
fix code format
AllanZyne Dec 28, 2023
13304f9
clean code
AllanZyne Dec 28, 2023
eb3ae88
clean code
AllanZyne Dec 28, 2023
f8f76de
clean code
AllanZyne Dec 28, 2023
e58fa03
fix comment
AllanZyne Dec 28, 2023
ff8449b
fix PVC local memory
AllanZyne Jan 8, 2024
fc40624
clean code
AllanZyne Jan 8, 2024
f153c32
fix build
AllanZyne Jan 10, 2024
10b83d9
tmp: force SYCL_PI_UR_SOURCE_DIR
AllanZyne Jan 10, 2024
b9d0809
remove asan.module_ctor
AllanZyne Jan 10, 2024
727a642
remove itto from libsycl-sanitizer.o
AllanZyne Jan 10, 2024
fb52e33
fix post link and lit
AllanZyne Jan 23, 2024
1b9e96e
Merge branch 'sycl' into review/yang/sanitizer-cpu-local
AllanZyne Jan 23, 2024
006a1e5
clean code
AllanZyne Jan 23, 2024
f423014
lit add more options
AllanZyne Jan 23, 2024
f51e62d
clean code
AllanZyne Jan 24, 2024
874a38e
clean code
AllanZyne Jan 24, 2024
caecf6b
add tests
AllanZyne Jan 24, 2024
47bd3f1
Switch to use DeviceGlobal
zhaomaosu Jan 26, 2024
aaa1b80
Add asan spir test
AllanZyne Jan 26, 2024
5f9de16
change ur repo
AllanZyne Jan 26, 2024
66895de
add e2e test
AllanZyne Jan 26, 2024
1e32baa
fix format
AllanZyne Jan 26, 2024
ba03cd7
Merge branch 'sycl' into review/yang/sanitizer-cpu-local
AllanZyne Jan 26, 2024
3c921af
follow the comments
AllanZyne Jan 28, 2024
eb4a904
fix format
AllanZyne Jan 28, 2024
15afd9a
Merge branch 'sycl' into review/yang/sanitizer-cpu-local
AllanZyne Jan 29, 2024
27545b5
update test
AllanZyne Jan 29, 2024
ad5c2bc
install ur_adapter_opencl
AllanZyne Jan 29, 2024
8ca76b7
fix format
AllanZyne Jan 29, 2024
2c75324
fix lit test
AllanZyne Jan 29, 2024
58b8489
fix test
AllanZyne Jan 30, 2024
4d34fef
clean code
AllanZyne Jan 30, 2024
163ffda
Merge branch 'sycl' into review/yang/sanitizer-cpu-local
AllanZyne Jan 30, 2024
77e53c4
fix test
AllanZyne Jan 30, 2024
c30e14f
add tests
AllanZyne Feb 1, 2024
fc41411
Merge branch 'sycl' into review/yang/sanitizer-cpu-local
AllanZyne Feb 1, 2024
a3a7b36
add type
AllanZyne Feb 1, 2024
9707887
fix build
AllanZyne Feb 1, 2024
2776d4a
update ur repo and tag
AllanZyne Feb 1, 2024
f3eadd7
Merge branch 'sycl' into review/yang/sanitizer-cpu-local
AllanZyne Feb 2, 2024
b3d4fe7
Merge branch 'sycl' into review/yang/sanitizer-cpu-local
AllanZyne Feb 2, 2024
9ea798e
add aspect-fp64
AllanZyne Feb 3, 2024
32cf548
add tests
AllanZyne Feb 4, 2024
5e6d983
init support
AllanZyne Feb 7, 2024
703b940
Merge branch 'sycl' into review/yang/use-after-free
AllanZyne Feb 7, 2024
04cb89b
reorg tests
AllanZyne Feb 7, 2024
0fd1844
add tests
AllanZyne Feb 7, 2024
b56b333
update tests
AllanZyne Feb 7, 2024
b241cea
Merge branch 'sycl' into review/yang/use-after-free
AllanZyne Feb 20, 2024
84e8500
fix size of DeviceType
AllanZyne Feb 23, 2024
34ebffe
fix out-of-bounds test
AllanZyne Feb 29, 2024
6cc327a
update lit tests
AllanZyne Mar 1, 2024
ca6ed6b
wip
AllanZyne Mar 3, 2024
641bc4b
Merge branch 'sycl' into review/yang/use-after-free
AllanZyne Mar 3, 2024
edb9080
wip
AllanZyne Mar 4, 2024
4843366
update lit tests
AllanZyne Mar 4, 2024
09e175f
clean code
AllanZyne Mar 4, 2024
ab6975c
fix build
AllanZyne Mar 4, 2024
e3788ad
use __AsanDebug
AllanZyne Mar 6, 2024
005f463
lit tests prepare to support other devices
AllanZyne Mar 6, 2024
1a906d2
clean code
AllanZyne Mar 6, 2024
b2d1109
fix lit tests
AllanZyne Mar 7, 2024
646b684
fix lit tests
AllanZyne Mar 7, 2024
4e562eb
fix lit tests
AllanZyne Mar 8, 2024
8fdf0e3
Merge branch 'sycl' into review/yang/use-after-free
AllanZyne Mar 8, 2024
4f3a415
add symbolizer and typo fix
AllanZyne Mar 8, 2024
ee34c0b
Merge branch 'sycl' into review/yang/use-after-free
AllanZyne Mar 8, 2024
7e89aff
update lit tests name
AllanZyne Mar 12, 2024
090271e
test
AllanZyne Mar 12, 2024
ce226f7
fix lit tests
AllanZyne Mar 12, 2024
49a4f9e
Merge branch 'sycl' into review/yang/use-after-free
AllanZyne Mar 12, 2024
d81456a
lit tests
AllanZyne Mar 24, 2024
053f17a
Merge branch 'sycl' into review/yang/use-after-free
AllanZyne Mar 27, 2024
5a07c81
sync with latest code
AllanZyne Mar 28, 2024
e297e4b
fix format
AllanZyne Apr 1, 2024
5325526
Merge branch 'sycl' into review/yang/use-after-free
AllanZyne Apr 1, 2024
d739818
fix ci fail
AllanZyne Apr 2, 2024
7d93f23
Add metadata "device.sanitizer" for sycl-post-link
AllanZyne Apr 2, 2024
a35044a
fix llvm-spirv crash
AllanZyne Apr 2, 2024
d4ef3c2
use absolutely path for "__asan_file"
AllanZyne Apr 2, 2024
0178c1d
Merge branch 'sycl' into review/yang/use-after-free
AllanZyne Apr 2, 2024
7fda0b9
remove build llvm-symbolizer
AllanZyne Apr 2, 2024
d28267d
rename lit tests
AllanZyne Apr 2, 2024
e488394
add omp guard for DeviceGlobal
AllanZyne Apr 2, 2024
29c1d9e
Add lit tests
AllanZyne Apr 2, 2024
a5c740d
fix sycl-post-link lit test
AllanZyne Apr 2, 2024
aded3b9
Revert "Add lit tests"
AllanZyne Apr 2, 2024
8b55ebb
Merge branch 'sycl' into review/yang/use-after-free
AllanZyne Apr 2, 2024
a8a19eb
fix libdevice
AllanZyne Apr 3, 2024
5bf6ed8
remove bad-context test
AllanZyne Apr 3, 2024
a6e0330
Merge branch 'sycl' into review/yang/use-after-free
AllanZyne Apr 7, 2024
0768bd6
remove omp related code
AllanZyne Apr 10, 2024
61c6a0d
Add clangd flag
AllanZyne Apr 10, 2024
0fda5c2
clean code
AllanZyne Apr 10, 2024
a718182
Merge branch 'sycl' into review/yang/use-after-free
AllanZyne Apr 10, 2024
dd8e7c8
fix build
AllanZyne Apr 10, 2024
d9f4f57
untrack .clangd
AllanZyne Apr 11, 2024
97ec20c
revert .gitignore
AllanZyne Apr 12, 2024
dcd9c05
fix format
AllanZyne Apr 12, 2024
7947ad9
Add git ignore for libdevice
AllanZyne Apr 12, 2024
dd653ee
remove .gitignore from libdevice
AllanZyne Apr 13, 2024
e40a20a
Merge branch 'sycl' into review/yang/use-after-free
AllanZyne Apr 17, 2024
66c4ccf
Revert unexpect changes on ASan Pass
AllanZyne Apr 17, 2024
8367ad1
Merge branch 'sycl' into review/yang/use-after-free
aarongreig Apr 17, 2024
24535c1
Update to latest UR tag + sync adapters
aarongreig Apr 17, 2024
6fe6a8f
Revert UR repo override.
aarongreig Apr 17, 2024
b9c0aed
Use local lit config change "device_sanitizer_flags" to "device_asan_…
AllanZyne Apr 18, 2024
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
155 changes: 155 additions & 0 deletions libdevice/include/asan_libdevice.hpp
Original file line number Diff line number Diff line change
@@ -0,0 +1,155 @@
//===---- asan_libdevice.hpp - Structure and declaration for sanitizer ----===//
//
// Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions.
// See https://llvm.org/LICENSE.txt for license information.
// SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
//
//===----------------------------------------------------------------------===//
#pragma once

#include <cinttypes>

// NOTE This file should be sync with
// unified-runtime/source/loader/layers/sanitizer/device_sanitizer_report.hpp
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why can't we make it a public header in UR and just include it here?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hi, @aelovikov-intel
When building the whole project, clang is firstly built and libdevice is built using the fresh built clang, at that time, sycl/ur component hasn't been built and deployed, so we can only include standard c/c++ headers in libdevice.
Thanks very much.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why can't we change that?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hi, @aelovikov-intel
Since libdevice component is shared by sycl and omp and libdeivce is a parallel component with sycl and omp , so we intend to minimize its dependency on any sycl/omp specific header files.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

its dependency on any sycl/omp specific header files

How does it apply to UR?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hi, @aelovikov-intel
libdevice is a small component, the only dependency is clang itself, there is no technical block including UR header here but it seems to be too "heavy" to introduce UR dependency in order to include a single header, all other functionalities except sanitizer will never depend on UR.

Thanks very much.


enum class DeviceSanitizerErrorType : int32_t {
UNKNOWN,
OUT_OF_BOUNDS,
MISALIGNED,
USE_AFTER_FREE,
OUT_OF_SHADOW_BOUNDS,
UNKNOWN_DEVICE,
NULL_POINTER,
};

enum class DeviceSanitizerMemoryType : int32_t {
UNKNOWN,
USM_DEVICE,
USM_HOST,
USM_SHARED,
LOCAL,
PRIVATE,
MEM_BUFFER,
DEVICE_GLOBAL,
};

struct DeviceSanitizerReport {
int Flag = 0;

char File[256 + 1] = {};
char Func[256 + 1] = {};

int32_t Line = 0;

uint64_t GID0 = 0;
uint64_t GID1 = 0;
uint64_t GID2 = 0;

uint64_t LID0 = 0;
uint64_t LID1 = 0;
uint64_t LID2 = 0;

uintptr_t Address = 0;
bool IsWrite = false;
uint32_t AccessSize = 0;
DeviceSanitizerMemoryType MemoryType = DeviceSanitizerMemoryType::UNKNOWN;
DeviceSanitizerErrorType ErrorType = DeviceSanitizerErrorType::UNKNOWN;

bool IsRecover = false;
};

struct LocalArgsInfo {
uint32_t ArgIndex = 0;
uint64_t Size = 0;
uint64_t SizeWithRedZone = 0;
};

struct LaunchInfo {
uintptr_t PrivateShadowOffset =
0; // don't move this field, we use it in AddressSanitizerPass

uintptr_t LocalShadowOffset = 0;
uintptr_t LocalShadowOffsetEnd = 0;
DeviceSanitizerReport SanitizerReport;

uint32_t NumLocalArgs = 0;
LocalArgsInfo *LocalArgs = nullptr; // ordered by ArgIndex
};

constexpr unsigned ASAN_SHADOW_SCALE = 3;
constexpr unsigned ASAN_SHADOW_GRANULARITY = 1ULL << ASAN_SHADOW_SCALE;

// Based on the observation, only the last 24 bits of the address of the private
// variable have changed, we use 31 bits(2G) to be safe.
constexpr std::size_t ASAN_PRIVATE_SIZE = 0x7fffffffULL + 1;

// These magic values are written to shadow for better error
// reporting.
constexpr int kUsmDeviceRedzoneMagic = (char)0x81;
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
constexpr int kUsmDeviceRedzoneMagic = (char)0x81;
constexpr int kUsmDeviceRedzoneMagic = 0x81;

is the cast necessary?

Copy link
Contributor Author

@AllanZyne AllanZyne Apr 10, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

yes, it's necessary, if not, 0x81 will be treated as unsigned 0x81.

constexpr int kUsmHostRedzoneMagic = (char)0x82;
constexpr int kUsmSharedRedzoneMagic = (char)0x83;
constexpr int kMemBufferRedzoneMagic = (char)0x84;
constexpr int kDeviceGlobalRedzoneMagic = (char)0x85;
constexpr int kNullPointerRedzoneMagic = (char)0x86;

constexpr int kUsmDeviceDeallocatedMagic = (char)0x91;
constexpr int kUsmHostDeallocatedMagic = (char)0x92;
constexpr int kUsmSharedDeallocatedMagic = (char)0x93;
constexpr int kMemBufferDeallocatedMagic = (char)0x93;

constexpr int kSharedLocalRedzoneMagic = (char)0xa1;

// Same with host ASan stack
const int kPrivateLeftRedzoneMagic = (char)0xf1;
const int kPrivateMidRedzoneMagic = (char)0xf2;
const int kPrivateRightRedzoneMagic = (char)0xf3;

constexpr auto kSPIR_AsanShadowMemoryGlobalStart =
"__AsanShadowMemoryGlobalStart";
constexpr auto kSPIR_AsanShadowMemoryGlobalEnd = "__AsanShadowMemoryGlobalEnd";

constexpr auto kSPIR_DeviceType = "__DeviceType";
constexpr auto kSPIR_AsanDebug = "__AsanDebug";

constexpr auto kSPIR_AsanDeviceGlobalCount = "__AsanDeviceGlobalCount";
constexpr auto kSPIR_AsanDeviceGlobalMetadata = "__AsanDeviceGlobalMetadata";

inline const char *ToString(DeviceSanitizerMemoryType MemoryType) {
switch (MemoryType) {
case DeviceSanitizerMemoryType::USM_DEVICE:
return "Device USM";
case DeviceSanitizerMemoryType::USM_HOST:
return "Host USM";
case DeviceSanitizerMemoryType::USM_SHARED:
return "Shared USM";
case DeviceSanitizerMemoryType::LOCAL:
return "Local Memory";
case DeviceSanitizerMemoryType::PRIVATE:
return "Private Memory";
case DeviceSanitizerMemoryType::MEM_BUFFER:
return "Memory Buffer";
case DeviceSanitizerMemoryType::DEVICE_GLOBAL:
return "Device Global";
default:
return "Unknown Memory";
}
}

inline const char *ToString(DeviceSanitizerErrorType ErrorType) {
switch (ErrorType) {
case DeviceSanitizerErrorType::OUT_OF_BOUNDS:
return "out-of-bounds-access";
case DeviceSanitizerErrorType::MISALIGNED:
return "misaligned-access";
case DeviceSanitizerErrorType::USE_AFTER_FREE:
return "use-after-free";
case DeviceSanitizerErrorType::OUT_OF_SHADOW_BOUNDS:
return "out-of-shadow-bounds-access";
case DeviceSanitizerErrorType::UNKNOWN_DEVICE:
return "unknown-device";
case DeviceSanitizerErrorType::NULL_POINTER:
return "null-pointer-access";
default:
return "unknown-error";
}
}
56 changes: 0 additions & 56 deletions libdevice/include/device-sanitizer-report.hpp

This file was deleted.

2 changes: 1 addition & 1 deletion libdevice/include/sanitizer_device_utils.hpp
Original file line number Diff line number Diff line change
Expand Up @@ -40,4 +40,4 @@ class
T val;
};

enum DeviceType : uintptr_t { UNKNOWN, CPU, GPU_PVC, GPU_DG2 };
enum DeviceType : uint64_t { UNKNOWN, CPU, GPU_PVC, GPU_DG2 };
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

should uint32_t be enough here?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I tried uint32_t, but level zero failed to write global variable in uint32_t.

Loading
Loading