Xcode 16 Support for Bluepill - Current Status and Call for Feedback

[nanwng]

We have been exploring ways to support Xcode 16 in Bluepill and have conducted several experiments to make it compatible. However, we’ve encountered persistent issues that prevent it from functioning as expected.

Recent Experiments

In our efforts to address Xcode 16 compatibility, we've experimented with various approaches. One recent attempt, outlined in this #571, seemed promising but ultimately didn’t resolve the core issues. The PR mainly touched dylib injection.

Investigations & Findings

After further investigation, we identified several key roadblocks that are preventing Xcode 16 support from working properly:

Missing Implementation

We are seeing errors like:

{86936} 20240812.233244 [ DEBUG ] (BP-1) BPTestBundleConnection debug message: 16:32:44.652 Talent[88061:3842881] Received test runner ready reply with error: (null)
{86936} 20240812.233244 [ DEBUG ] (BP-1) [OUTPUT] 2024-08-12 16:32:44.652093-0700 Talent[88061:3842854] [Default] Error while preparing for testing: (null)

These errors are due to an objc_msgSend failure for readyForTestingFuture coupled with readyForTestingPromise, which are coming from XCTest/XCTRunnerIDESession.h and we did not implement them in Bluepill.

Possibly, the connection to testmanagerd via BPTMDRunnerConnection.m is where we should implement the IDESession protocol?

XCTCapabilities

An even more severe linking error occurs when attempting to implement the basic XCTCapabilities

ld: cannot link directly with 'XCTestSupport' because product being built is not an allowed client of it
clang: error: linker command failed with exit code 1 (use -v to see invocation)

More digging, the error comes from https://github.com/apple-opensource/ld64/blob/e28c028b20af187a16a7161d89e91868a450cadc/src/ld/InputFiles.cpp#L839

And

❯ cat Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX.sdk/System/Library/PrivateFrameworks/XCTestSupport.framework/Versions/A/XCTestSupport.tbd
--- !tapi-tbd
tbd-version:     4
targets:         [ x86_64-macos, x86_64-maccatalyst, arm64-macos, arm64-maccatalyst,
                   arm64e-macos, arm64e-maccatalyst ]
install-name:    '/System/Library/PrivateFrameworks/XCTestSupport.framework/Versions/A/XCTestSupport'
current-version: 23182.1
compatibility-version: 0
allowable-clients:
  - targets:         [ x86_64-macos, x86_64-maccatalyst, arm64-macos, arm64-maccatalyst,
                       arm64e-macos, arm64e-maccatalyst ]
    clients:         [ AutomationInfrastructureIntegrationTests, XCTAutomationSupport,
                       XCTAutomationSupportTests, XCTDaemonControl, XCTDaemonControlTests,
                       XCTDaemonTests, XCTest-ToolHostedTests, XCTestCore, XCUIAutomation,
                       testmanagerd ]
exports:
  - targets:         [ x86_64-macos, x86_64-maccatalyst, arm64-macos, arm64-maccatalyst,
                       arm64e-macos, arm64e-maccatalyst ]
    symbols:         [ _XCTMakeSubcapability ]
    objc-classes:    [ XCTCapabilities, XCTCapabilitiesBuilder, XCTOSFaultRuntimeIssue ]

testmanagerd is listed as an allowable client. We might be able to hack it to add bp to the list. But it could be non-trivial and fragile.

Community Input

At this point, we're reaching out to the community to gather any insights, suggestions, or alternative approaches that could help make Bluepill compatible with Xcode 16. If anyone has had success with their own custom patches or workarounds, we would appreciate you sharing your experiences.

Next Steps

If there are no viable solutions within the community, we’re considering officially announcing the end of Xcode 16 support for Bluepill in a few weeks. We recommend transitioning to other test runners like xctestrunner, which has better compatibility with the latest versions of Xcode.

We'd love to hear your thoughts or ideas on this matter. Thank you for your contributions to the project!