Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Store test content in a custom metadata section. #880

Open
wants to merge 1 commit into
base: main
Choose a base branch
from

Conversation

grynspan
Copy link
Contributor

@grynspan grynspan commented Jan 2, 2025

This PR uses the experimental symbol linkage margers feature in the Swift compiler to emit metadata about tests (and exit tests) into a dedicated section of the test executable being built. At runtime, we discover that section and read out the tests from it.

This has several benefits over our current model, which involves walking Swift's type metadata table looking for types that conform to a protocol:

  1. We don't need to define that protocol as public API in Swift Testing,
  2. We don't need to emit type metadata (much larger than what we really need) for every test function,
  3. We don't need to duplicate a large chunk of the Swift ABI sources in order to walk the type metadata table correctly, and
  4. Almost all the new code is written in Swift, whereas the code it is intended to replace could not be fully represented in Swift and needed to be written in C++.

This change will be necessary to support Embedded Swift because there is no type metadata section emitted for embedded targets.

The change also opens up the possibility of supporting generic types in the future because we can emit metadata without needing to emit a nested type (which is not always valid in a generic context.) That's a "future direction" and not covered by this PR specifically.

I've defined a layout for entries in the new swift5_tests section that should be flexible enough for us in the short-to-medium term and which lets us define additional arbitrary test content record types. The layout of this section is covered in depth in the new TestContent.md article.

This functionality is only available if a test target enables the experimental "SymbolLinkageMarkers" feature. We continue to emit protocol-conforming types for now—that code will be removed if and when the experimental feature is properly supported (modulo us adopting relevant changes to the feature's API.)

Resolves #764.

See Also

#735
swiftlang/swift#76698
swiftlang/swift#78411

Checklist:

  • Code and documentation should follow the style of the Style Guide.
  • If public symbols are renamed or modified, DocC references should be updated.

@grynspan grynspan added enhancement New feature or request tools integration Integration of swift-testing into tools/IDEs windows 🪟 Windows support linux 🐧 Linux support (all distros) darwin 🍎 macOS, iOS, watchOS, tvOS, and visionOS support performance Performance issues embedded-swift Embedded Swift issues wasi/wasm 🧭 WebAssembly support android 🤖 Android support freebsd 😈 FreeBSD support labels Jan 2, 2025
@grynspan grynspan added this to the Swift 6.x milestone Jan 2, 2025
@grynspan grynspan self-assigned this Jan 2, 2025
@grynspan grynspan force-pushed the jgrynspan/custom-metadata-section branch from e98f3bc to bb2526a Compare January 3, 2025 20:22
Documentation/ABI/TestContent.md Outdated Show resolved Hide resolved
Documentation/ABI/TestContent.md Outdated Show resolved Hide resolved
Documentation/ABI/TestContent.md Outdated Show resolved Hide resolved
Documentation/ABI/TestContent.md Outdated Show resolved Hide resolved
Documentation/ABI/TestContent.md Outdated Show resolved Hide resolved
Documentation/ABI/TestContent.md Outdated Show resolved Hide resolved
Documentation/ABI/TestContent.md Outdated Show resolved Hide resolved
Documentation/ABI/TestContent.md Outdated Show resolved Hide resolved
Documentation/ABI/TestContent.md Outdated Show resolved Hide resolved
Documentation/Porting.md Outdated Show resolved Hide resolved
@grynspan grynspan requested a review from compnerd January 5, 2025 17:54
@grynspan
Copy link
Contributor Author

grynspan commented Jan 5, 2025

@swift-ci test

Documentation/ABI/TestContent.md Outdated Show resolved Hide resolved
Documentation/ABI/TestContent.md Outdated Show resolved Hide resolved
@grynspan
Copy link
Contributor Author

grynspan commented Jan 6, 2025

@swift-ci test

@grynspan
Copy link
Contributor Author

grynspan commented Jan 6, 2025

@swift-ci test Linux

@grynspan
Copy link
Contributor Author

grynspan commented Jan 6, 2025

@swift-ci test Windows

1 similar comment
@grynspan
Copy link
Contributor Author

grynspan commented Jan 6, 2025

@swift-ci test Windows

@grynspan grynspan force-pushed the jgrynspan/custom-metadata-section branch 2 times, most recently from fe3dbab to 713b032 Compare January 6, 2025 20:17
@grynspan
Copy link
Contributor Author

grynspan commented Jan 6, 2025

@swift-ci test

@grynspan
Copy link
Contributor Author

grynspan commented Jan 6, 2025

@swift-ci test macOS

@grynspan grynspan force-pushed the jgrynspan/custom-metadata-section branch from 4e01f8b to d07e477 Compare January 10, 2025 23:11
@grynspan
Copy link
Contributor Author

@swift-ci test

@grynspan grynspan force-pushed the jgrynspan/custom-metadata-section branch 2 times, most recently from fc30eeb to 7cb5985 Compare January 13, 2025 19:54
@grynspan
Copy link
Contributor Author

@swift-ci test

@grynspan grynspan force-pushed the jgrynspan/custom-metadata-section branch 3 times, most recently from 1024c92 to f72f629 Compare January 15, 2025 20:35
@grynspan
Copy link
Contributor Author

@swift-ci test

@grynspan grynspan force-pushed the jgrynspan/custom-metadata-section branch from f72f629 to cfa505c Compare January 16, 2025 23:26
@grynspan
Copy link
Contributor Author

@swift-ci test

@grynspan grynspan force-pushed the jgrynspan/custom-metadata-section branch from cfa505c to db83b1d Compare January 17, 2025 19:25
@grynspan
Copy link
Contributor Author

@swift-ci test

@grynspan grynspan closed this Jan 17, 2025
@grynspan grynspan reopened this Jan 17, 2025
@grynspan
Copy link
Contributor Author

There needs to be a warning on that Close with comment button.

@grynspan grynspan added the less-c++ Work to reduce the size of our C++ codebase and/or dependencies label Jan 19, 2025
@grynspan grynspan force-pushed the jgrynspan/custom-metadata-section branch 4 times, most recently from d7ffa67 to a44121e Compare January 22, 2025 22:57
@grynspan
Copy link
Contributor Author

@swift-ci test

2 similar comments
@grynspan
Copy link
Contributor Author

@swift-ci test

@grynspan
Copy link
Contributor Author

@swift-ci test

@grynspan grynspan force-pushed the jgrynspan/custom-metadata-section branch from 8852595 to 55338a7 Compare January 30, 2025 14:39
This PR uses the experimental symbol linkage margers feature in the Swift
compiler to emit metadata about tests (and exit tests) into a dedicated section
of the test executable being built. At runtime, we discover that section and
read out the tests from it.

This has several benefits over our current model, which involves walking Swift's
type metadata table looking for types that conform to a protocol:

1. We don't need to define that protocol as public API in Swift Testing,
1. We don't need to emit type metadata (much larger than what we really need)
   for every test function,
1. We don't need to duplicate a large chunk of the Swift ABI sources in order to
   walk the type metadata table correctly, and
1. Almost all the new code is written in Swift, whereas the code it is intended
   to replace could not be fully represented in Swift and needed to be written
   in C++.

The change also opens up the possibility of supporting generic types in the
future because we can emit metadata without needing to emit a nested type (which
is not always valid in a generic context.) That's a "future direction" and not
covered by this PR specifically.

I've defined a layout for entries in the new `swift5_tests` section that should
be flexible enough for us in the short-to-medium term and which lets us define
additional arbitrary test content record types. The layout of this section is
covered in depth in the new [TestContent.md](Documentation/ABI/TestContent.md)
article.

This functionality is only available if a test target enables the experimental
`"SymbolLinkageMarkers"` feature. We continue to emit protocol-conforming types
for now—that code will be removed if and when the experimental feature is
properly supported (modulo us adopting relevant changes to the feature's API.)

#735
swiftlang/swift#76698
swiftlang/swift#78411
@grynspan grynspan force-pushed the jgrynspan/custom-metadata-section branch from 55338a7 to 725a40f Compare January 30, 2025 17:24
@grynspan
Copy link
Contributor Author

@swift-ci test

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
android 🤖 Android support darwin 🍎 macOS, iOS, watchOS, tvOS, and visionOS support embedded-swift Embedded Swift issues enhancement New feature or request freebsd 😈 FreeBSD support less-c++ Work to reduce the size of our C++ codebase and/or dependencies linux 🐧 Linux support (all distros) performance Performance issues tools integration Integration of swift-testing into tools/IDEs wasi/wasm 🧭 WebAssembly support windows 🪟 Windows support
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Translate Discovery.cpp to Swift
2 participants