Skip to content

CBL-8744 : Add binary test targets for the Swift test suites - #3564

Merged
pasin merged 2 commits into
masterfrom
CBL-8744
Aug 26, 2026
Merged

CBL-8744 : Add binary test targets for the Swift test suites#3564
pasin merged 2 commits into
masterfrom
CBL-8744

Conversation

@pasin

@pasin pasin commented Aug 26, 2026

Copy link
Copy Markdown
Contributor

Restructure Swift tests for running the tests against a pre-built CouchbaseLiteSwift binary instead of a build from the working tree, following the same design as the Objective-C binary test targets.

  • Add CBL_EE_Swift_Binary_Tests and CBL_Swift_Binary_Tests, which compile and link against frameworks staged in BinaryTests/Frameworks. Each runs on macOS and iOS. The community target leaves out the tests for Enterprise edition.

  • Remove @testable imports from the test files. Most were unused. The files that keep white box tests import the module conditionally, as a binary framework has no testability, and guard those tests with #if !CBL_BINARY_TEST.

  • Keep MultipeerReplicatorTest internal only, as its identity setup requires the internal issuer API.

  • Replace the internal keychain test API in the listener tests with public SecItem based cleanup helpers, and derive the anonymous identity common name at runtime in the same way as the Objective-C tests. The internal API is deleted in the EE repo.

  • Align the keychain item count helper between the Objective-C and Swift tests. Both take the item class and the discovered common name as arguments.

  • Delete testJSONRepresentation which access internal API and is not useful.

  • Add testCleanUpAnonymousIdentities to verify that the cleanup leaves no anonymous identities nor certificates in the keychain.

  • Convert the keychain guard early returns to XCTSkipUnless so that runs without keychain access report skips instead of passing vacuously.

  • Set CBL_BINARY_TEST in the targets' xcconfigs via SWIFT_ACTIVE_COMPILATION_CONDITIONS and GCC_PREPROCESSOR_DEFINITIONS.

  • Rename the shared iOS host application target to CBL_Binary_Tests_iOS_App, as it now hosts the binary tests of both languages.

  • Add a scheme per edition and platform, following the Objective-C binary test schemes.

  • Companion PR : https://github.com/couchbaselabs/couchbase-lite-ios-ee/pull/354

@pasin
pasin force-pushed the CBL-8744 branch 2 times, most recently from fd35258 to 511720b Compare August 26, 2026 02:31
Restructure Swift tests for running the tests against a pre-built
CouchbaseLiteSwift binary instead of a build from the working tree,
following the same design as the Objective-C binary test targets.

- Add CBL_EE_Swift_Binary_Tests and CBL_Swift_Binary_Tests, which
  compile and link against frameworks staged in BinaryTests/Frameworks.
  Each runs on macOS and iOS. The community target leaves out the tests
  for Enterprise edition.

- Remove @testable imports from the test files. Most were unused. The
  files that keep white box tests import the module conditionally, as a
  binary framework has no testability, and guard those tests with
  #if !CBL_BINARY_TEST.

- Keep MultipeerReplicatorTest internal only, as its identity setup
  requires the internal issuer API.

- Replace the internal keychain test API in the listener tests with
  public SecItem based cleanup helpers, and derive the anonymous
  identity common name at runtime in the same way as the Objective-C
  tests. The internal API is deleted in the EE repo.

- Align the keychain item count helper between the Objective-C and
  Swift tests. Both take the item class and the discovered common name
  as arguments.

- Delete testJSONRepresentation which uses an internal API and is not
  useful.

- Add testCleanUpAnonymousIdentities to verify that the cleanup leaves
  no anonymous identities nor certificates in the keychain.

- Convert the keychain guard early returns to XCTSkipUnless so that
  runs without keychain access report skips instead of passing
  vacuously.

- Set CBL_BINARY_TEST in the targets' xcconfigs via
  SWIFT_ACTIVE_COMPILATION_CONDITIONS and GCC_PREPROCESSOR_DEFINITIONS.

- Rename the shared iOS host application target to
  CBL_Binary_Tests_iOS_App, as it now hosts the binary tests of both
  languages.

- Add a scheme per edition and platform, following the Objective-C
  binary test schemes.

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR adds Swift “binary test” targets/schemes so the Swift test suites can run against pre-staged CouchbaseLiteSwift (and related) xcframeworks, mirroring the existing Objective-C binary test approach. As part of that, it hardens Swift tests to be black-box compatible (no @testable, no internal-only APIs) and reworks keychain cleanup in listener tests to avoid relying on internal APIs.

Changes:

  • Introduces CE/EE Swift binary test targets and schemes, wired to staged frameworks and CBL_BINARY_TEST compile conditions.
  • Updates Swift tests to remove @testable imports and conditionally compile white-box assertions out of binary test runs.
  • Replaces internal keychain cleanup APIs in Swift listener tests with SecItem-based helpers and adds coverage for anonymous identity cleanup.

Reviewed changes

Copilot reviewed 33 out of 34 changed files in this pull request and generated 3 comments.

Show a summary per file
File Description
xcconfigs/CBL_Swift_Binary_Tests.xcconfig Adds build settings for Swift binary test bundle (defines CBL_BINARY_TEST, disables headermap, runpath, bridging header).
xcconfigs/CBL_EE_Swift_Binary_Tests.xcconfig EE wrapper config including EE common + Swift binary test settings.
xcconfigs/CBL_Binary_Tests_iOS_App.xcconfig Renames/aligns the shared iOS host app xcconfig for binary tests.
Swift/Tests/VectorSearchTest+Lazy.swift Drops @testable import for binary compatibility.
Swift/Tests/VectorSearchTest.swift Drops @testable import for binary compatibility.
Swift/Tests/URLEndpointListenerTest+Collection.swift Switches keychain guard early-returns to XCTSkipUnless and removes @testable.
Swift/Tests/URLEndpointListenerTest.swift Reworks keychain cleanup to use SecItem* queries; adds cleanup test; removes @testable.
Swift/Tests/UnnestArrayTest.swift Guards internal-only assertions with #if !CBL_BINARY_TEST and conditionally uses @testable.
Swift/Tests/TLSIdentityTest.swift Guards internal-only section behind #if !CBL_BINARY_TEST and conditionally uses @testable.
Swift/Tests/ReplicatorTest+PendingDocIds.swift Drops @testable import for binary compatibility.
Swift/Tests/ReplicatorTest+CustomConflict.swift Drops @testable import for binary compatibility.
Swift/Tests/ReplicatorTest+Collection.swift Drops @testable import for binary compatibility.
Swift/Tests/QueryTest+Collection.swift Drops @testable import for binary compatibility.
Swift/Tests/QueryTest.swift Drops @testable import and removes the internal-API JSON-representation test.
Swift/Tests/PublisherTest.swift Drops @testable import for binary compatibility.
Swift/Tests/PartialIndexTest.swift Drops @testable import for binary compatibility.
Swift/Tests/MultipeerReplicatorTest.swift Explicitly errors under CBL_BINARY_TEST to keep internal-only test out of binary targets.
Swift/Tests/LogSinkTest.swift Drops @testable import for binary compatibility.
Swift/Tests/DatabaseTest.swift Drops @testable import for binary compatibility.
Swift/Tests/CollectionTest.swift Drops @testable import for binary compatibility.
Swift/Tests/CBLTestHelper.m Makes LiteCore exception-expectation helper a no-op for binary tests; refactors macros.
Swift/Tests/CBLTestHelper.h Removes direct exposure of LiteCore exception counter from header.
Swift/Tests/CBLTestCase.swift Drops @testable import for binary compatibility.
Swift/Query.swift Removes internal JSON encode/decode helpers from Query.
Swift/ModuleMap/Generated/CouchbaseLiteSwift-EE.private.modulemap Removes CBLURLEndpointListener+Swift.h from generated private module map.
Swift/ModuleMap/CouchbaseLiteSwift-EE.private.txt Removes CBLURLEndpointListener+Swift.h from the private header list template.
Objective-C/Tests/URLEndpointListenerTest+Main.m Aligns keychain item counting helper signature with Swift/updated logic.
CouchbaseLite.xcodeproj/xcshareddata/xcschemes/CBL_Swift_Binary_Tests.xcscheme Adds macOS Swift CE binary test scheme.
CouchbaseLite.xcodeproj/xcshareddata/xcschemes/CBL_Swift_Binary_Tests_iOS_App.xcscheme Adds iOS app-hosted Swift CE binary test scheme.
CouchbaseLite.xcodeproj/xcshareddata/xcschemes/CBL_ObjC_Binary_Tests_iOS_App.xcscheme Updates Obj-C iOS binary test scheme to use renamed host app target.
CouchbaseLite.xcodeproj/xcshareddata/xcschemes/CBL_EE_Swift_Binary_Tests.xcscheme Adds macOS Swift EE binary test scheme.
CouchbaseLite.xcodeproj/xcshareddata/xcschemes/CBL_EE_Swift_Binary_Tests_iOS_App.xcscheme Adds iOS app-hosted Swift EE binary test scheme.
CouchbaseLite.xcodeproj/xcshareddata/xcschemes/CBL_EE_ObjC_Binary_Tests_iOS_App.xcscheme Updates EE Obj-C iOS binary test scheme to use renamed host app target.
CouchbaseLite.xcodeproj/project.pbxproj Adds Swift binary test targets, embeds staged frameworks, renames host app target, and wires build phases/schemes.
Suppressed comments (2)

Swift/Tests/URLEndpointListenerTest.swift:146

  • After querying certificates, the code asserts success but then force-casts result to [SecCertificate] even when the query fails. This can crash instead of failing the test with the assertion message.
        XCTAssertEqual(status, errSecSuccess, "Cannot query certificates (OSStatus = \(status))")
        let certs = result as! [SecCertificate]
        for cert in certs {

Swift/Tests/URLEndpointListenerTest.swift:559

  • keychainItemCount asserts SecItemCopyMatching succeeded but still force-casts result and iterates it even when status != errSecSuccess. Since XCTest assertions don't abort, this can crash and obscure the real keychain error.
        let status = SecItemCopyMatching(query as CFDictionary, &result)
        if status == errSecItemNotFound { return 0 }
        XCTAssertEqual(status, errSecSuccess, "Cannot query keychain items (OSStatus = \(status))")

        var count = 0

💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.

Comment thread Swift/Tests/CBLTestHelper.m
Comment thread Swift/Tests/URLEndpointListenerTest.swift Outdated
Comment thread Swift/Tests/URLEndpointListenerTest.swift
@pasin
pasin requested a review from jianminzhao August 26, 2026 04:01
@pasin

pasin commented Aug 26, 2026

Copy link
Copy Markdown
Contributor Author

The PR validation from Jenkins is actually passed (See here). I do not know why it's waiting for the result and the link was pointing to localhost as well.

- Ensure CBLExpectExceptionsEnd is always called via try/finally blocks.

- Replace force unwraps and force casts with safe unwrapping and
  XCTFail or XCTUnwrap in URLEndpointListenerTest.
@pasin
pasin merged commit 2581952 into master Aug 26, 2026
8 of 9 checks passed
@pasin
pasin deleted the CBL-8744 branch August 26, 2026 17:00
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants