diff --git a/docs/drive.md b/docs/drive.md index b9e8ebfb..cba23819 100644 --- a/docs/drive.md +++ b/docs/drive.md @@ -10,7 +10,7 @@ driving is something a build opts into, never an API an app calls. ## The protocol and its transports shared/wire/headless.proto is the driver's protocol (ls, info, tap, type, -set, select, swipe, longpress, back, dismiss, wait, quit); shared/wire/session.proto a +set, select, swipe, longpress, back, dismiss, scene, wait, quit); shared/wire/session.proto a head's session (drawing the app). An app serves both as an `IsomerServer` (drive/serving), apart from any transport; adapters carry it: @@ -105,8 +105,12 @@ iso swipe "Weekly report" left # a list row's trailing swipe actions, listed as iso swipe "Weekly report" left Delete # or the one named, performed iso longpress "Weekly report" # its context menu: its items are the screen's buttons iso back | dismiss +iso scene background # the app left (its scene phase), then come back to: +iso scene active # .onChange(of: scenePhase) sees each step iso wait "Taps: 1" --timeout 10 iso drive # JSON lines on standard input +iso launch …/drive/launchers/ios_launcher.sh # a headful app of a launcher's (below) +iso launch stop ios ``` An element's name is its role and its place among that role's on the @@ -158,9 +162,29 @@ Apple's frameworks): with the flag, @isomer//apple/libraries/swift_ui links drive/apple, which a constructor starts (the app calls nothing). It answers the same commands from the app's accessibility tree, with the same names: on iOS it walks UIKit's accessibility containers once the app's -accessibility is on, scrolls to what is off screen as a person would, waits -out navigation transitions, and taps what takes no activation (a list's -cell, a control) as a tap would; on the Mac it asks its own process as an +accessibility is on (SwiftUI's own elements, a List's or a Form's rows and +the controls in them, give their identifiers to the selector, as XCUITest +reads them, not through UIAccessibilityIdentification; a multi-line +`TextField` is labelled by its placeholder), scrolls to what is off screen +as a person would, waits out navigation transitions and, after an action, +what it starts a moment later (a sheet presented from a sheet), and taps +what takes no activation (a list's cell, a control) as a tap would, at its +place in the window however the app is turned. `dismiss` closes what is +presented on top: escape, else a sheet a person may swipe away, as the +swipe does (its `onDismiss` runs), else its own Cancel, Close or Done. +`scene` moves SwiftUI's scene phase as leaving the app and coming back +does: on iOS, SwiftUI's own scene delegate is told each step UIKit tells it +(will resign active, did enter background, will enter foreground, did become +active) and UIKit's notifications are posted after each, the app staying on +screen and driven (posting the notifications alone moves nothing, and a +simulator's real trip to the home screen takes seconds and suspends the +app); on the Mac the app is hidden (background), its windows put in the +Dock (inactive) or shown and brought to the front (active). Isomer's own +driver sets its runtime's phase; a test's `HeadlessApp.scene` asks the +launcher first, and Android's sends the app home and brings it back, its +activity stopped and resumed, as `iso scene` does for an app `iso launch` +started. ISO_DRIVE_TRACE=1 in the app's environment prints each reading's walk of +the tree; on the Mac it asks its own process as an accessibility client would (AXUIElement), on the main thread. A list's row of texts is a button, as Isomer's renderers give it. Launch with simctl's `SIMCTL_CHILD_ISO_DRIVER=…`, or a Mac app with `ISO_DRIVER` in its @@ -180,11 +204,29 @@ try app.tap("Tap") try app.wait(for: "Taps: 1") ``` -`HeadlessApp(launcher:environment:)` drives a headful app with the same -test: the test holds a hub of its own (IsomerServing's `IsomerHub`, on a -loopback WebSocket), and the launcher, run as ` -[KEY=VALUE…]`, starts the app dialling out to it (ISO_DRIVER, -ISO_DRIVER_NAME) and ends it when stopped. A launcher a platform, beside the +`HeadlessApp(launcher:environment:arguments:fresh:)` drives a headful app +with the same test: the test holds a hub of its own (IsomerServing's +`IsomerHub`, on a loopback WebSocket), and the launcher, run as +` [KEY=VALUE…] [-- …]`, starts the app +dialling out to it (ISO_DRIVER, ISO_DRIVER_NAME), its environment the +pairs and its command line the arguments, and ends it when stopped (SIGTERM, +or a file named `stop` among its requests, which `quit()` leaves, then +signals: drive/launchers/requests.sh). + +- **Fresh by default.** A launch starts with none of the app's data from + before, as an app built headless does (`fresh: true`, the launcher's + ISOMER_FRESH=1): the iPhone's launcher clears the app's defaults (through + the simulator's cfprefsd), its files and its app groups' and the + simulator's keychain; Android's runs `pm clear`; the Mac's and Linux's + give the app a new home; the web's page has a new browser profile every + launch anyway. `fresh: false` (`iso launch --keep-data`) keeps it, for a + test of what lasts between launches. Windows' launcher keeps it, for now. +- **Pictures** (`picture`) are named by the platform's device as the app + stands (`ios/phone-landscape/…`, `android/phone-landscape/…` in a wide + window). An iPhone app that turns itself to landscape on an upright + simulator is drawn on its side in the simulator's frame; its driver says + how it stands (Info's `orientation`), and the iPhone's launcher turns the + picture to match. A launcher a platform, beside the Storybook's tests (examples' headless/storybook_headless_application): the web's (headless Chrome under browser_check `--hold`), the iPhone's (simctl on a booted simulator), the Mac's, Android's (`am start` with the extras, diff --git a/drive/apple/AXClientAccessibility.swift b/drive/apple/AXClientAccessibility.swift index ad7d98b9..0c90abff 100644 --- a/drive/apple/AXClientAccessibility.swift +++ b/drive/apple/AXClientAccessibility.swift @@ -340,6 +340,30 @@ extension AccessibilityDriver { return await activate(button) } + /// Moves SwiftUI's scene phase as the Mac moves it: background when the + /// app is hidden (NSApp.hide, as ⌘H), inactive when its windows are in + /// the Dock (miniaturized), active when shown and frontmost again. The + /// app's deactivation alone (another app frontmost) leaves it active. + func scene(_ phase: String) async -> String? { + let windows = NSApp.windows.filter { $0.isMiniaturized || $0.isVisible } + switch phase { + case "background": + NSApp.hide(nil) + case "inactive": + if NSApp.isHidden { NSApp.unhide(nil) } + for window in windows where !window.isMiniaturized && window.canBecomeMain { window.miniaturize(nil) } + case "active": + if NSApp.isHidden { NSApp.unhide(nil) } + for window in windows where window.isMiniaturized { window.deminiaturize(nil) } + NSApp.activate(ignoringOtherApps: true) + default: + return "no scene phase \(phase)" + } + // The window's animation into the Dock or out of it. + try? await Task.sleep(nanoseconds: 600_000_000) + return nil + } + /// A sheet's cancel, as Escape gives it; else its Cancel button. func dismiss(_ screen: AccessibilityScreen) async -> Bool { if let cancel = screen.elements.last(where: { $0.role == "button" && ["Cancel", "Done", "OK"].contains($0.label) }) { diff --git a/drive/apple/AccessibilityDriver.swift b/drive/apple/AccessibilityDriver.swift index b03a2407..b525d16a 100644 --- a/drive/apple/AccessibilityDriver.swift +++ b/drive/apple/AccessibilityDriver.swift @@ -186,7 +186,7 @@ struct AccessibilityScreen { /// Drives the app running in this process through its accessibility tree. /// Each platform's file gives it `screen()`, `transitioning()`, `reveal`, and /// the actions (`activate`, `type`, `set`, `select`, `swipe`, `back`, -/// `dismiss`). +/// `dismiss`, `scene`). @MainActor final class AccessibilityDriver { var info: Wire_Info @@ -205,26 +205,42 @@ final class AccessibilityDriver { /// Frames until the screen holds still (five readings alike, or the /// timeout): animations, transitions and the app's tasks run meanwhile. - func settle(timeout: Double = 3, after action: Bool = false) async -> AccessibilityScreen { + /// After an action (`unchanged`: the screen before it), a screen still as + /// it was is given a while longer to change: what a tap starts may begin + /// a few turns of the run loop later (a sheet presented from a sheet, + /// which SwiftUI presents once the one below has settled; Quick Look's + /// controls), and the answer is the screen it leads to. + func settle(timeout: Double = 3, after unchanged: AccessibilityScreen? = nil) async -> AccessibilityScreen { // What an action starts (a navigation's push, a sheet) begins on the // next turns of the run loop: a moment before reading the screen. - if action { try? await Task.sleep(nanoseconds: 300_000_000) } - let deadline = Date().addingTimeInterval(timeout) - // A navigation's push or pop, a sheet's coming or going: the screen - // is the one it ends on. - while Date() < deadline, transitioning() { - try? await Task.sleep(nanoseconds: 50_000_000) + if unchanged != nil { try? await Task.sleep(nanoseconds: 150_000_000) } + let start = Date() + let deadline = start.addingTimeInterval(timeout) + let before = unchanged?.wire + func busy() async { + // A navigation's push or pop, a sheet's coming or going: the + // screen is the one it ends on. + while Date() < deadline, transitioning() { + try? await Task.sleep(nanoseconds: 50_000_000) + } } + await busy() var current = await screen() var last = current.wire var still = 0 while Date() < deadline { try? await Task.sleep(nanoseconds: 50_000_000) + if transitioning() { + await busy() + still = 0 + } current = await screen() let reading = current.wire if reading == last { still += 1 - if still >= 5 { break } + // Still as it was before the action: up to a second and a + // half for what it started to show. + if still >= 5, before != reading || Date().timeIntervalSince(start) > 1.5 { break } } else { still = 0 last = reading @@ -235,6 +251,7 @@ final class AccessibilityDriver { func handle(_ command: Wire_Command) async -> Wire_Answer { var error: String? + var waited: AccessibilityScreen? let target = command.target let before = await settle() switch command.cmd { @@ -289,6 +306,14 @@ final class AccessibilityDriver { } case "longpress": error = await longPress(before, target) + case "scene": + // The app's scene phase, as leaving it and coming back moves it + // (each platform's file says how). + if ["active", "inactive", "background"].contains(command.value) { + error = await scene(command.value) + } else { + error = "scene: active, inactive or background, not \(command.value)" + } case "back": if !(await back(before)) { error = "nothing to go back from" } case "dismiss": @@ -301,13 +326,16 @@ final class AccessibilityDriver { let text = command.text let deadline = Date().addingTimeInterval(command.timeout > 0 ? command.timeout : 5) var shown = false - while Date() < deadline { - let frame = await settle(timeout: 0.5) + var frame = before + while true { if frame.texts.contains(where: { $0.text.contains(text) }) || frame.elements.contains(where: { $0.label.contains(text) || ($0.value ?? "").contains(text) }) { shown = true + waited = frame break } + if Date() >= deadline { break } + frame = await settle(timeout: 0.5) } if !shown { error = "\(text) didn't show" } default: @@ -316,8 +344,15 @@ final class AccessibilityDriver { var answer = Wire_Answer() answer.ok = error == nil if let error { answer.error = error } - let after = await settle(after: !["ls", "info", "quit", "wait"].contains(command.cmd)) - answer.screen = after.wire + // What only reads the screen answers with the reading it made; an + // action, with the screen it leads to. + if ["ls", "info", "inspect", "quit"].contains(command.cmd) { + answer.screen = before.wire + } else if command.cmd == "wait", let waited { + answer.screen = waited.wire + } else { + answer.screen = await settle(after: before).wire + } if command.cmd == "info" { answer.info = info } return answer } diff --git a/drive/apple/UIKitAccessibility.swift b/drive/apple/UIKitAccessibility.swift index 1c8608b6..ae88ef8c 100644 --- a/drive/apple/UIKitAccessibility.swift +++ b/drive/apple/UIKitAccessibility.swift @@ -10,8 +10,10 @@ import IsomerWire import UIKit extension AccessibilityScreen { - /// The screen's bounds, set as each screen is read. - @MainActor static var screenBounds: CGRect? + /// The window the screen is read in, and how accessibility's screen + /// coordinates (an element's frame, its activation point) map into it, + /// set as each screen is read (`AccessibilityDriver.toWindow`). + @MainActor static var window: (bounds: CGRect, fromScreen: CGAffineTransform)? /// The screen under `roots` (the key window, or what it presents). @MainActor @@ -55,14 +57,14 @@ extension AccessibilityScreen { /// content carries (a `List(selection:)` row found by its content's /// identifier, as Isomer's driver finds it). @MainActor - private mutating func visit(_ object: NSObject) { + private mutating func visit(_ object: NSObject, depth: Int = 0) { guard object is UICollectionViewCell || object is UITableViewCell else { - visitObject(object) + visitObject(object, depth: depth) return } let elementCount = elements.count let textCount = texts.count - visitObject(object) + visitObject(object, depth: depth) if elements[elementCount...].allSatisfy({ $0.role == "staticText" }), texts.count > textCount { // The texts' identifiers are the row's: not elements of their own. elements.removeSubrange(elementCount...) @@ -73,8 +75,68 @@ extension AccessibilityScreen { } } + /// An element's accessibility identifier, however it carries it: a + /// UIView adopts UIAccessibilityIdentification, but SwiftUI's own + /// elements (the nodes a List's, a Form's rows and its controls are) + /// answer `accessibilityIdentifier` without declaring the protocol, so a + /// cast to it finds nothing where XCUITest, which asks the selector, + /// finds them all. @MainActor - private mutating func visitObject(_ object: NSObject) { + static func identifier(_ object: NSObject) -> String? { + let id: String? + if let identified = object as? UIAccessibilityIdentification { + id = identified.accessibilityIdentifier + } else if object.responds(to: #selector(getter: UIAccessibilityIdentification.accessibilityIdentifier)) { + id = object.value(forKey: "accessibilityIdentifier") as? String + } else { + id = nil + } + return id.flatMap { $0.isEmpty ? nil : $0 } + } + + /// A text field's title where it has no accessibility label: a + /// UITextField's placeholder; a UITextView's (SwiftUI's multi-line + /// `TextField(_:text:axis:)`, which UIKit gives no placeholder of its + /// own) what its accessibility says of its placeholder while it is + /// empty, else the label it draws as one (kept, hidden, once there is + /// text), else its identifier. + @MainActor + static func fieldTitle(_ object: NSObject) -> String? { + if let field = object as? UITextField { return field.placeholder } + let placeholder = NSSelectorFromString("accessibilityPlaceholderValue") + if object.responds(to: placeholder), let text = object.value(forKey: "accessibilityPlaceholderValue") as? String, + !text.isEmpty { + return text + } + if let view = object as? UIView { + func label(_ view: UIView) -> String? { + for subview in view.subviews { + if let text = (subview as? UILabel)?.text, !text.isEmpty { return text } + if let text = label(subview) { return text } + } + return nil + } + if let text = label(view) { return text } + } + return identifier(object) + } + + /// ISO_DRIVE_TRACE=1: each reading's walk printed, an object a line + /// (its class, whether it is an element, its traits, label, identifier + /// and frame), for finding what the tree holds where the driver lists + /// nothing. + static let tracing = ProcessInfo.processInfo.environment["ISO_DRIVE_TRACE"] == "1" + + @MainActor + private mutating func visitObject(_ object: NSObject, depth: Int = 0) { + if Self.tracing { + let frame = object.accessibilityFrame + print("[iso trace] \(String(repeating: " ", count: depth))\(Swift.type(of: object))" + + " element=\(object.isAccessibilityElement) traits=\(object.accessibilityTraits.rawValue)" + + " label=\(object.accessibilityLabel ?? "-") id=\(Self.identifier(object) ?? "-")" + + " children=\((object.accessibilityElements?.count).map(String.init) ?? "-")/\(object.accessibilityElementCount())" + + " frame=\(Int(frame.minX)),\(Int(frame.minY)),\(Int(frame.width)),\(Int(frame.height))") + } if let view = object as? UIView, view.isHidden || view.alpha < 0.01 { return } if object.accessibilityElementsHidden { return } // A page a navigation slid away (beside the window) is not on the @@ -83,15 +145,15 @@ extension AccessibilityScreen { if let view = object as? UIView, let window = view.window { let frame = view.convert(view.bounds, to: window) if !frame.isEmpty, frame.maxX <= window.bounds.minX || frame.minX >= window.bounds.maxX { return } - } else if object.isAccessibilityElement, let screen = Self.screenBounds { - let frame = object.accessibilityFrame - if !frame.isEmpty, frame.maxX <= screen.minX || frame.minX >= screen.maxX { return } + } else if object.isAccessibilityElement, let window = Self.window { + let frame = object.accessibilityFrame.applying(window.fromScreen) + if !frame.isEmpty, frame.maxX <= window.bounds.minX || frame.minX >= window.bounds.maxX { return } } if object.isAccessibilityElement { let traits = object.accessibilityTraits let label = object.accessibilityLabel ?? "" let value = object.accessibilityValue - let id = (object as? UIAccessibilityIdentification)?.accessibilityIdentifier.flatMap { $0.isEmpty ? nil : $0 } + let id = Self.identifier(object) let kind = String(describing: Swift.type(of: object)) let frame = object.accessibilityFrame var params = ["traits": String(traits.rawValue), @@ -103,7 +165,7 @@ extension AccessibilityScreen { disabled: disabled) } if object is UITextField || (object as? UITextView)?.isEditable == true || traits.contains(.searchField) { - add("textField", label.isEmpty ? (object as? UITextField)?.placeholder ?? "" : label, + add("textField", label.isEmpty ? Self.fieldTitle(object) ?? "" : label, (object as? UITextField)?.text ?? (object as? UITextView)?.text ?? value) } else if let control = object as? UISegmentedControl { add("segmented", label, value, (0.. 0 { for index in 0.. CGAffineTransform { + func point(_ x: CGFloat, _ y: CGFloat) -> CGPoint { + let rect = UIAccessibility.convertToScreenCoordinates(CGRect(x: x, y: y, width: 0, height: 0), in: window) + return rect.origin + } + let origin = point(0, 0), right = point(100, 0), down = point(0, 100) + let toScreen = CGAffineTransform(a: (right.x - origin.x) / 100, b: (right.y - origin.y) / 100, + c: (down.x - origin.x) / 100, d: (down.y - origin.y) / 100, + tx: origin.x, ty: origin.y) + return toScreen.inverted() + } + + /// An interface orientation as headless.proto's Info names it. + static func orientationName(_ orientation: UIInterfaceOrientation) -> String { + switch orientation { + case .portrait: "portrait" + case .portraitUpsideDown: "portrait-upside-down" + case .landscapeLeft: "landscape-left" + case .landscapeRight: "landscape-right" + default: "" + } + } + /// The key window's topmost presented view controller (an alert, a /// sheet), or the window itself: what can be used. func roots() -> ([NSObject], [(style: String, title: String, message: String)]) { - let windows = UIApplication.shared.connectedScenes.compactMap { $0 as? UIWindowScene } - .flatMap(\.windows).filter { !$0.isHidden } - guard let window = windows.first(where: \.isKeyWindow) ?? windows.first else { return ([], []) } - AccessibilityScreen.screenBounds = window.screen.bounds + guard let window = Self.keyWindow() else { return ([], []) } + AccessibilityScreen.window = (window.bounds, Self.fromScreen(window)) info.width = Double(window.bounds.width) info.height = Double(window.bounds.height) + info.orientation = window.windowScene.map { Self.orientationName($0.effectiveGeometry.interfaceOrientation) } ?? "" // An open menu (a Menu, a picker's menu): UIKit's context menu, a view // of the window's own beside its content, which the window's // accessibility containers don't lead to. While it's open it is all @@ -278,8 +368,24 @@ extension AccessibilityDriver { return true } guard let window = roots().0.compactMap({ ($0 as? UIWindow) ?? ($0 as? UIView)?.window }).first else { return false } - let point = window.convert(object.accessibilityActivationPoint, from: window.screen.coordinateSpace) - var view = window.hitTest(point, with: nil) + let point = object.accessibilityActivationPoint.applying(Self.fromScreen(window)) + if let view = window.hitTest(point, with: nil), use(from: view, element, window) { return true } + // Where the point reaches nothing that takes a tap: the view the + // element belongs to (its accessibility containers, up to a view), + // whatever the coordinates say. + var container: NSObject? = object + while let current = container, !(current is UIView) { + let key = "accessibilityContainer" + container = current.responds(to: NSSelectorFromString(key)) ? current.value(forKey: key) as? NSObject : nil + } + if let view = container as? UIView, view.window === window { return use(from: view, element, window) } + return false + } + + /// The tap on `element` that `view` (where it stands) or a view around + /// it takes: a list's or a table's cell selected, a control's action. + private func use(from view: UIView, _ element: AccessibilityScreen.Element, _ window: UIWindow) -> Bool { + var view: UIView? = view while let current = view { if let cell = current as? UICollectionViewCell, let collection = Self.enclosing(UICollectionView.self, cell), let path = collection.indexPath(for: cell) { @@ -416,8 +522,9 @@ extension AccessibilityDriver { return nil } // A menu: opened, then its option activated. + let closed = await screen() guard await activate(element) else { return "\(element.name) didn't open" } - let menu = await settle(after: true) + let menu = await settle(after: closed) guard let choice = menu.elements.last(where: { $0.role == "button" && $0.label == option }) else { return "\(element.name) has no option \(option)" } @@ -463,6 +570,11 @@ extension AccessibilityDriver { navigation.popViewController(animated: false) return true } + // Nothing pushed in what is presented: back closes it, as Isomer's + // driver's back closes a sheet at its first page. + if let presented = presenting, presented !== window.rootViewController, !(presented is UIAlertController) { + return await dismiss(screen) + } return false } @@ -494,9 +606,117 @@ extension AccessibilityDriver { "a long press isn't driven on iOS yet" } + /// Closes what is presented on top, as a person would: escape (what + /// VoiceOver's two-finger scrub does: an alert's Cancel, a menu, a + /// popover); else a sheet a person may swipe away, closed as the swipe + /// closes it (its presentation controller's delegate asked and told, so + /// SwiftUI's `isPresented` and `onDismiss` follow); else its own Cancel, + /// Close or Done. Each tried until the presentation is gone. func dismiss(_ screen: AccessibilityScreen) async -> Bool { - guard let view = roots().0.first as? UIView else { return false } - return view.accessibilityPerformEscape() + guard let window = Self.keyWindow() else { return false } + var top: UIViewController? = window.rootViewController + while let next = top?.presentedViewController, !next.isBeingDismissed { top = next } + guard let presented = top, presented !== window.rootViewController else { + // An open menu, nothing presented: escape alone. + return (roots().0.first as? UIView)?.accessibilityPerformEscape() ?? false + } + func closed() async -> Bool { + let deadline = Date().addingTimeInterval(2) + while Date() < deadline { + if presented.presentingViewController == nil || presented.isBeingDismissed { return true } + try? await Task.sleep(nanoseconds: 50_000_000) + } + return false + } + if presented.view.accessibilityPerformEscape(), await closed() { return true } + if !(presented is UIAlertController), let controller = presented.presentationController, + !presented.isModalInPresentation, + controller.delegate?.presentationControllerShouldDismiss?(controller) != false { + let delegate = controller.delegate + delegate?.presentationControllerWillDismiss?(controller) + presented.presentingViewController?.dismiss(animated: true) { + delegate?.presentationControllerDidDismiss?(controller) + } + if await closed() { return true } + } + let names = ["cancel", "close", "done"] + for name in names { + if let button = screen.elements.last(where: { + $0.role == "button" && !$0.disabled && ($0.label.lowercased() == name || $0.id?.lowercased() == name) + }), await activate(button), await closed() { + return true + } + } + return false + } + + /// The scene phases the driver has moved its scenes to, by scene: UIKit's + /// own activation state stays as it was. + private static var phases: [ObjectIdentifier: Int] = [:] + + /// Moves SwiftUI's scene phase (`@Environment(\.scenePhase)`, its + /// `.onChange`) as leaving the app and coming back does: SwiftUI's own + /// scene delegate told, a step at a time, what UIKit tells it then + /// (`sceneWillResignActive`, `sceneDidEnterBackground`, + /// `sceneWillEnterForeground`, `sceneDidBecomeActive`), and UIKit's + /// notifications of each step posted after it for what observes them. + /// The app stays on screen and is still driven (a simulator's real trip + /// to the home screen takes seconds and suspends the app). Posting the + /// notifications alone moves nothing: SwiftUI's phase follows its + /// delegate. + func scene(_ phase: String) async -> String? { + let order = ["background": 0, "inactive": 1, "active": 2] + guard let wanted = order[phase] else { return "no scene phase \(phase)" } + let scenes = UIApplication.shared.connectedScenes.compactMap { $0 as? UIWindowScene } + guard !scenes.isEmpty else { return "no scene" } + let center = NotificationCenter.default + let application = UIApplication.shared + for scene in scenes { + let key = ObjectIdentifier(scene) + var current = Self.phases[key] ?? { + switch scene.activationState { + case .foregroundActive: return 2 + case .foregroundInactive: return 1 + default: return 0 + } + }() + while current != wanted { + let delegate = scene.delegate + switch (current, wanted > current) { + case (2, false): + delegate?.sceneWillResignActive?(scene) + center.post(name: UIScene.willDeactivateNotification, object: scene) + center.post(name: UIApplication.willResignActiveNotification, object: application) + current = 1 + case (1, false): + delegate?.sceneDidEnterBackground?(scene) + center.post(name: UIScene.didEnterBackgroundNotification, object: scene) + center.post(name: UIApplication.didEnterBackgroundNotification, object: application) + current = 0 + case (0, true): + delegate?.sceneWillEnterForeground?(scene) + center.post(name: UIScene.willEnterForegroundNotification, object: scene) + center.post(name: UIApplication.willEnterForegroundNotification, object: application) + current = 1 + default: + delegate?.sceneDidBecomeActive?(scene) + center.post(name: UIScene.didActivateNotification, object: scene) + center.post(name: UIApplication.didBecomeActiveNotification, object: application) + current = 2 + } + // A turn of the run loop between steps, as UIKit's come. + try? await Task.sleep(nanoseconds: 100_000_000) + } + Self.phases[key] = current + } + return nil + } + + /// The window the app is shown in: the key one, else the first. + static func keyWindow() -> UIWindow? { + let windows = UIApplication.shared.connectedScenes.compactMap { $0 as? UIWindowScene } + .flatMap(\.windows).filter { !$0.isHidden } + return windows.first(where: \.isKeyWindow) ?? windows.first } } #endif diff --git a/drive/apple/iso_apple_drive_attach.c b/drive/apple/iso_apple_drive_attach.c index 3b8280fa..4457aa77 100644 --- a/drive/apple/iso_apple_drive_attach.c +++ b/drive/apple/iso_apple_drive_attach.c @@ -1,8 +1,10 @@ // Starts the Apple accessibility driver (AccessibilityDriver.swift) once the // app is running: a constructor of the library --@isomer//drive:headful links -// into an app on Apple's SwiftUI, so the app's own code calls nothing. Half a -// second after launch, on the main queue (the app's first scene is up by -// then; the driver reads the screen afresh at every command anyway). +// into an app on Apple's SwiftUI, so the app's own code calls nothing. A +// tenth of a second after launch, on the main queue, once the app has +// finished launching (the driver reads the screen afresh at every command, +// waiting for it to settle, so a first scene not up yet costs nothing; half +// a second here was a tenth of a test's launch). #include extern void iso_apple_drive_attach(void); @@ -13,6 +15,6 @@ static void iso_apple_drive_start(void *context) { } __attribute__((constructor)) static void iso_apple_drive_schedule(void) { - dispatch_after_f(dispatch_time(DISPATCH_TIME_NOW, 500 * NSEC_PER_MSEC), dispatch_get_main_queue(), NULL, + dispatch_after_f(dispatch_time(DISPATCH_TIME_NOW, 100 * NSEC_PER_MSEC), dispatch_get_main_queue(), NULL, iso_apple_drive_start); } diff --git a/drive/driver/IsomerAppDriver.swift b/drive/driver/IsomerAppDriver.swift index 9699e7d7..d9ffbf43 100644 --- a/drive/driver/IsomerAppDriver.swift +++ b/drive/driver/IsomerAppDriver.swift @@ -25,6 +25,9 @@ // back, dismiss the page's Back (its bar's, the topmost // presentation's own navigation first), else the // system's; the topmost presentation +// scene the app's scene phase (active, inactive, +// background), as leaving the app and coming +// back moves it // wait [timeout] until the text shows // quit ends the call (and a headless app) // @@ -901,6 +904,17 @@ public final class IsomerAppDriver { } else { error = "nothing presented" } + case "scene": + // The app's scene phase, as its host sets it when the app is left + // and come back to (`@Environment(\.scenePhase)`, its `.onChange`): + // an app built headless has no other; a headful one's host sets it + // again at its next real change. + switch command.value { + case "active": runtime.setScenePhase(.active) + case "inactive": runtime.setScenePhase(.inactive) + case "background": runtime.setScenePhase(.background) + default: error = "scene: active, inactive or background, not \(command.value)" + } case "wait": let text = command.text let deadline = Date().addingTimeInterval(command.timeout > 0 ? command.timeout : 5) diff --git a/drive/launchers/android_launcher.sh b/drive/launchers/android_launcher.sh index fd47a8f0..0513668d 100755 --- a/drive/launchers/android_launcher.sh +++ b/drive/launchers/android_launcher.sh @@ -4,14 +4,19 @@ # started with ISO_DRIVER and ISO_DRIVER_NAME as its intent's ISO_ extras # (its environment), until this is stopped. The hub listens on this # machine's loopback, so its port is reversed onto the device first. -# Pictures (a test's `picture`) are adb's screencap. +# Pictures (a test's `picture`) are adb's screencap. It ends the app when +# stopped (SIGTERM, or HeadlessApp's `stop` request, requests.sh). # -# android_launcher.sh [KEY=VALUE…] +# android_launcher.sh [KEY=VALUE…] [-- …] # +# The arguments after `--` are the app's own, as ISO_ARGUMENTS (where +# Isomer's Foundation reads a program's arguments on Android: UserDefaults' +# `-key value`), separated by spaces. # From the test's environment: ISOMER_ANDROID_PACKAGE (the app), # ISOMER_ANDROID_ACTIVITY (.MainActivity, isomer_android_app's, by # default), ADB (adb; the PATH's by default), ANDROID_SERIAL (which device, -# when several). +# when several), ISOMER_FRESH (requests.sh: 1, the default, starts the app +# with none of its data from before: `pm clear`). set -uo pipefail # Nothing here reads standard input, and adb mustn't either: on a device # without adb's shell protocol (Android 6.0, API 23) `adb shell` copies @@ -27,8 +32,11 @@ activity="${ISOMER_ANDROID_ACTIVITY:-.MainActivity}" port="${url##*:}" port="${port%%/*}" "$adb" reverse "tcp:$port" "tcp:$port" > /dev/null || exit 1 +split_arguments "$@" extras=(-e ISO_DRIVER "$url" -e ISO_DRIVER_NAME "$name") -for pair in "$@"; do +# (Quoted for the device's shell, which `adb shell` hands the line to.) +[ ${#app_arguments[@]} -eq 0 ] || extras+=(-e ISO_ARGUMENTS "'${app_arguments[*]}'") +for pair in ${pairs[@]+"${pairs[@]}"}; do case "${pair%%=*}" in ISO_HEADLESS_WIDTH | ISO_HEADLESS_HEIGHT) ;; *) extras+=(-e "${pair%%=*}" "${pair#*=}") ;; @@ -46,6 +54,9 @@ stop() { } trap stop TERM INT "$adb" shell am force-stop "$package" > /dev/null 2>&1 || true +# Its data as an install leaves it (files, preferences, databases, caches, +# the permissions it was granted). +if fresh; then "$adb" shell pm clear "$package" > /dev/null 2>&1 || true; fi "$adb" logcat -c > /dev/null 2>&1 || true if [ -n "$logs" ]; then "$adb" logcat > "$logs/$name.logcat" 2> /dev/null & @@ -53,4 +64,24 @@ if [ -n "$logs" ]; then fi "$adb" shell am start -W -n "$package/$activity" "${extras[@]}" > /dev/null || exit 1 take() { "$adb" exec-out screencap -p > "$1" && [ -s "$1" ]; } +# HeadlessApp's `scene` (requests.sh) through the activity's own lifecycle: +# background, the app sent home (stopped); active, brought to the front again +# as the launcher's icon brings it (its task as it was, the activity not +# started again). Inactive (paused, still shown) has no such way: the +# driver sets it. +scene() { + case "$1" in + background) "$adb" shell input keyevent KEYCODE_HOME > /dev/null || return 1 ;; + active) "$adb" shell am start -n "$package/$activity" > /dev/null 2>&1 || return 1 ;; + *) return 1 ;; + esac + # Until the activity is stopped, or resumed (at most 5 s). + local resumed + for _ in $(seq 1 25); do + resumed="$("$adb" shell dumpsys activity activities 2> /dev/null | grep -E "mResumedActivity|topResumedActivity" | grep -c "$package/")" + if [ "$1" = active ]; then [ "$resumed" -gt 0 ] && return 0; else [ "$resumed" -eq 0 ] && return 0; fi + sleep 0.2 + done + return 0 +} serve_requests take diff --git a/drive/launchers/ios_launcher.sh b/drive/launchers/ios_launcher.sh index 90077b2a..2137b5b8 100755 --- a/drive/launchers/ios_launcher.sh +++ b/drive/launchers/ios_launcher.sh @@ -2,15 +2,21 @@ # An iPhone app on a booted simulator, headful, for a test's HeadlessApp # (docs/drive.md): built with --@isomer//drive:headful, so its driver dials # out, launched with ISO_DRIVER and ISO_DRIVER_NAME (simctl's -# SIMCTL_CHILD_), until this is stopped. Pictures (a test's `picture`) are -# simctl's, masked (the simulator's own status bar is held at 9:41 by -# whoever made the simulator). +# SIMCTL_CHILD_), until this is stopped (SIGTERM, or HeadlessApp's `stop` +# request). Pictures (a test's `picture`) are simctl's, masked (the +# simulator's own status bar is held at 9:41 by whoever made the +# simulator), turned to the app's interface: an app that turns itself to +# landscape on an upright simulator is pictured on its side, as a person +# turning the phone sees it. # -# ios_launcher.sh [KEY=VALUE…] +# ios_launcher.sh [KEY=VALUE…] [-- …] # +# The arguments after `--` are the app's own (simctl launch's). # From the test's environment: ISOMER_SIMULATOR (a booted simulator's -# UDID) and ISOMER_BUNDLE_ID (the app installed on it). ISO_HEADLESS_WIDTH -# and ISO_HEADLESS_HEIGHT are a headless window's, not a phone's: left out. +# UDID), ISOMER_BUNDLE_ID (the app installed on it) and ISOMER_FRESH +# (requests.sh: 1, the default, starts the app with none of its data from +# before). ISO_HEADLESS_WIDTH and ISO_HEADLESS_HEIGHT are a headless +# window's, not a phone's: left out. set -uo pipefail . "$(dirname "$0")/requests.sh" url="${1:?usage: ios_launcher.sh [KEY=VALUE…]}" @@ -18,8 +24,9 @@ name="${2:?}" shift 2 udid="${ISOMER_SIMULATOR:?ISOMER_SIMULATOR: a booted simulator}" bundle="${ISOMER_BUNDLE_ID:?ISOMER_BUNDLE_ID: the app installed on it}" +split_arguments "$@" environment=(SIMCTL_CHILD_ISO_DRIVER="$url" SIMCTL_CHILD_ISO_DRIVER_NAME="$name") -for pair in "$@"; do +for pair in ${pairs[@]+"${pairs[@]}"}; do case "${pair%%=*}" in ISO_HEADLESS_WIDTH | ISO_HEADLESS_HEIGHT) ;; *) environment+=("SIMCTL_CHILD_$pair") ;; @@ -27,6 +34,53 @@ for pair in "$@"; do done stop() { xcrun simctl terminate "$udid" "$bundle" > /dev/null 2>&1 || true; exit 0; } trap stop TERM INT -env "${environment[@]}" xcrun simctl launch --terminate-running-process "$udid" "$bundle" > /dev/null || exit 1 -take() { xcrun simctl io "$udid" screenshot --mask=black "$1" > /dev/null 2>&1; } +if fresh; then + # The app's data as an install leaves it: its defaults, each domain + # deleted through the simulator's cfprefsd by its file's path (the + # domain the app's container holds: by name, `defaults` reaches the + # simulator's own, and a file removed under cfprefsd comes back from its + # cache), then its files and its app groups' (the containers themselves + # kept, with their metadata), and the simulator's keychain (an app's + # items outlive it, as on a phone). The simulator is the test's: nothing + # else of its is touched. + xcrun simctl terminate "$udid" "$bundle" > /dev/null 2>&1 + clear_container() { + [ -n "$1" ] && [ -d "$1" ] || return 0 + local plist + for plist in "$1/Library/Preferences/"*.plist; do + [ -e "$plist" ] && xcrun simctl spawn "$udid" defaults delete "${plist%.plist}" > /dev/null 2>&1 + done + rm -rf "$1/Documents/"* "$1/Library/Application Support" "$1/Library/Caches" "$1/Library/Preferences/"* \ + "$1/Library/SplashBoard" "$1/tmp/"* 2> /dev/null + } + clear_container "$(xcrun simctl get_app_container "$udid" "$bundle" data 2> /dev/null)" + xcrun simctl get_app_container "$udid" "$bundle" groups 2> /dev/null | while IFS=$'\t' read -r group path; do + clear_container "$path" + done + xcrun simctl keychain "$udid" reset > /dev/null 2>&1 +fi +# The app's output kept with the test's (an app that never attached says why +# there; ISO_DRIVE_TRACE=1, simctl's SIMCTL_CHILD_ISO_DRIVE_TRACE, prints +# the driver's walk of its accessibility tree). +log="${TEST_UNDECLARED_OUTPUTS_DIR:-${TMPDIR:-/tmp}}/$name.log" +env "${environment[@]}" xcrun simctl launch --terminate-running-process --stdout="$log" --stderr="$log" "$udid" "$bundle" \ + ${app_arguments[@]+"${app_arguments[@]}"} > /dev/null || exit 1 +# A picture, turned from the simulator's upright frame to the interface's +# orientation (the app's driver says which, in the request) when the two +# differ: sips turns clockwise, so landscape-right (the home indicator on +# the right, the phone turned anticlockwise) is 270. +take() { + local path="$1" orientation="${2:-}" turn="" width height + xcrun simctl io "$udid" screenshot --mask=black "$path" > /dev/null 2>&1 || return 1 + case "$orientation" in + landscape-right) turn=270 ;; + landscape-left) turn=90 ;; + *) return 0 ;; + esac + width="$(sips -g pixelWidth "$path" 2> /dev/null | awk '/pixelWidth/ { print $2 }')" + height="$(sips -g pixelHeight "$path" 2> /dev/null | awk '/pixelHeight/ { print $2 }')" + # Already as the interface stands (a simulator turned with the app). + [ "${width:-0}" -gt "${height:-0}" ] && return 0 + sips -r "$turn" "$path" > /dev/null 2>&1 +} serve_requests take diff --git a/drive/launchers/linux_launcher.sh b/drive/launchers/linux_launcher.sh index e591fb1c..8980ae2e 100755 --- a/drive/launchers/linux_launcher.sh +++ b/drive/launchers/linux_launcher.sh @@ -6,9 +6,12 @@ # stopped. A test's pictures are the app's last frame: its renderer writes # each frame to ISO_SHOT, a PNG (linux/libraries/swift_ui/renderer.c). # -# linux_launcher.sh [KEY=VALUE…] +# linux_launcher.sh [KEY=VALUE…] [-- …] # -# From the test's environment: ISOMER_LINUX_APP, the app. +# From the test's environment: ISOMER_LINUX_APP, the app, and ISOMER_FRESH +# (requests.sh: 1, the default, empties the app's home, HOME and XDG_*, +# the same every launch, in the temporary directory, before the launch; 0 +# keeps it). set -uo pipefail . "$(dirname "$0")/requests.sh" url="${1:?usage: linux_launcher.sh [KEY=VALUE…]}" @@ -16,13 +19,18 @@ name="${2:?}" shift 2 app="${ISOMER_LINUX_APP:?ISOMER_LINUX_APP: the Linux app}" frames="$(mktemp -d)" +split_arguments "$@" environment=(ISO_DRIVER="$url" ISO_DRIVER_NAME="$name" LIBGL_ALWAYS_SOFTWARE=1 ISO_SHOT="$frames/frame.png") -for pair in "$@"; do +for pair in ${pairs[@]+"${pairs[@]}"}; do case "${pair%%=*}" in ISO_HEADLESS_WIDTH | ISO_HEADLESS_HEIGHT) ;; *) environment+=("$pair") ;; esac done +home="${TMPDIR:-/tmp}/isomer-linux-home-$(basename "$app")" +if fresh; then rm -rf "$home"; fi +mkdir -p "$home/.config" "$home/.local/share" "$home/.cache" +environment+=(HOME="$home" XDG_CONFIG_HOME="$home/.config" XDG_DATA_HOME="$home/.local/share" XDG_CACHE_HOME="$home/.cache") pids=() stop() { for pid in "${pids[@]}"; do kill "$pid" 2> /dev/null; done; rm -rf "$frames"; exit 0; } trap stop TERM INT @@ -35,7 +43,7 @@ if command -v Xvfb > /dev/null; then else environment+=(SDL_VIDEODRIVER=offscreen) fi -env "${environment[@]}" "$app" > "${TEST_UNDECLARED_OUTPUTS_DIR:-/tmp}/$name.log" 2>&1 & +env "${environment[@]}" "$app" ${app_arguments[@]+"${app_arguments[@]}"} > "${TEST_UNDECLARED_OUTPUTS_DIR:-/tmp}/$name.log" 2>&1 & pids+=($!) # A picture: the last frame drawn (none before the first). last_frame() { [ -s "$frames/frame.png" ] && cp "$frames/frame.png" "$1"; } diff --git a/drive/launchers/macos_launcher.sh b/drive/launchers/macos_launcher.sh index 184ad6e5..4c87cb63 100755 --- a/drive/launchers/macos_launcher.sh +++ b/drive/launchers/macos_launcher.sh @@ -2,11 +2,19 @@ # A Mac app, headful, for a test's HeadlessApp (docs/drive.md): built with # --@isomer//drive:headful, so its driver dials out, run with ISO_DRIVER and # ISO_DRIVER_NAME in its environment, light, with a home of its own, until -# this is stopped. Pictures (a test's `picture`) are its window's alone. +# this is stopped (SIGTERM, or HeadlessApp's `stop` request). Pictures (a +# test's `picture`) are its window's alone. # -# macos_launcher.sh [KEY=VALUE…] +# macos_launcher.sh [KEY=VALUE…] [-- …] # -# From the test's environment: ISOMER_MAC_APP, the .app. +# The arguments after `--` are the app's own, after the launcher's +# (`-NSTextInsertionPointBlinkPeriodOn 100000`, a `-key value` of +# UserDefaults' argument domain: a later one wins). From the test's +# environment: ISOMER_MAC_APP, the .app, and ISOMER_FRESH (requests.sh: 1, +# the default, starts it with none of its data from before: its defaults +# deleted, through cfprefsd, which keeps them by the user's own home +# whatever CFFIXED_USER_HOME says, and its home emptied; 0 keeps both). The +# home is the app's own, in the temporary directory, the same every launch. set -uo pipefail . "$(dirname "$0")/requests.sh" url="${1:?usage: macos_launcher.sh [KEY=VALUE…]}" @@ -14,18 +22,30 @@ name="${2:?}" shift 2 app="${ISOMER_MAC_APP:?ISOMER_MAC_APP: the Mac app}" binary="$app/Contents/MacOS/$(defaults read "$app/Contents/Info" CFBundleExecutable)" +split_arguments "$@" environment=(ISO_DRIVER="$url" ISO_DRIVER_NAME="$name") -for pair in "$@"; do +for pair in ${pairs[@]+"${pairs[@]}"}; do case "${pair%%=*}" in ISO_HEADLESS_WIDTH | ISO_HEADLESS_HEIGHT) ;; *) environment+=("$pair") ;; esac done -home="$(mktemp -d)" +bundle_id="$(defaults read "$app/Contents/Info" CFBundleIdentifier)" +home="${TMPDIR:-/tmp}/isomer-mac-home-$bundle_id" +if fresh; then + defaults delete "$bundle_id" > /dev/null 2>&1 + rm -rf "$home" +fi +mkdir -p "$home" env "${environment[@]}" CFFIXED_USER_HOME="$home" "$binary" -AppleInterfaceStyle Light \ - -ApplePersistenceIgnoreState YES -AppleShowScrollBars WhenScrolling > "$home/app.log" 2>&1 & + -ApplePersistenceIgnoreState YES -AppleShowScrollBars WhenScrolling ${app_arguments[@]+"${app_arguments[@]}"} \ + > "$home/app.log" 2>&1 & pid=$! -stop() { kill "$pid" 2> /dev/null || true; wait "$pid" 2> /dev/null || true; rm -rf "$home"; exit 0; } +stop() { + kill "$pid" 2> /dev/null || true + wait "$pid" 2> /dev/null || true + exit 0 +} trap stop TERM INT # The app's largest window, by CoreGraphics' window list (no permission # needed), pictured alone. diff --git a/drive/launchers/requests.sh b/drive/launchers/requests.sh index 227674de..74a72e24 100644 --- a/drive/launchers/requests.sh +++ b/drive/launchers/requests.sh @@ -1,19 +1,76 @@ -# Sourced by the launchers: the test's picture requests (HeadlessApp's -# `picture`), each a file .request in ISOMER_LAUNCHER_REQUESTS naming -# the picture's path. `serve_requests ` runs until the launcher is -# stopped, calling ` ` for each request and removing it once -# answered; a launcher that can't picture its platform answers none. +# Sourced by the launchers: the test's requests (HeadlessApp's `picture` +# and `quit`), files in ISOMER_LAUNCHER_REQUESTS. +# +# - .request: a picture, its first line the path to write it to, then +# lines of KEY=VALUE about the app as it stands (orientation=). +# Or, its first line `scene:` (HeadlessApp's `scene`): the app's +# scene moved by the platform itself (active, inactive, background), by a +# launcher that defines `scene ` (Android's: the app sent home and +# brought back); answered in .answer, `ok`, or why not, when the +# test's driver command moves it instead. +# - stop: the test is done with the app (`quit`), seen within a poll: the +# launcher ends, as SIGTERM ends it, so a test needn't wait out a signal +# that a platform (Windows) doesn't deliver. +# +# `serve_requests ` runs until the launcher is stopped, calling +# ` ` for each picture and removing the request +# once answered (a launcher that can't picture its platform answers none). +# Started in the background by a launcher that waits on its app, it stops +# that launcher (its $$) with SIGTERM, whose trap ends the app. serve_requests() { - local take="$1" request path + local take="$1" request path orientation while true; do if [ -n "${ISOMER_LAUNCHER_REQUESTS:-}" ]; then + if [ -e "$ISOMER_LAUNCHER_REQUESTS/stop" ]; then + kill -TERM $$ 2> /dev/null + return 0 + fi for request in "$ISOMER_LAUNCHER_REQUESTS"/*.request; do [ -e "$request" ] || continue - path="$(cat "$request")" - "$take" "$path" || echo "launcher: no picture for $path" >&2 + path="$(head -1 "$request")" + case "$path" in + scene:*) + if declare -F scene > /dev/null && scene "${path#scene:}"; then + echo ok > "${request%.request}.answer" + else + echo "the launcher doesn't move the scene to ${path#scene:}" > "${request%.request}.answer" + fi + ;; + *) + orientation="$(sed -n 's/^orientation=//p' "$request")" + "$take" "$path" "$orientation" || echo "launcher: no picture for $path" >&2 + ;; + esac rm -f "$request" done fi - sleep 0.2 + sleep 0.05 + done +} + +# Whether the app starts with none of its data from before: ISOMER_FRESH, +# 1 (the default, as HeadlessApp's own: a test's app starts as an app built +# headless does, with nothing kept from another test) or 0 (its files, +# defaults and caches kept, for a test of what lasts between launches). +fresh() { [ "${ISOMER_FRESH:-1}" != 0 ]; } + +# A launcher's arguments after the hub's URL and the app's name: +# KEY=VALUE pairs (the app's environment), then, after `--`, the app's own +# command-line arguments (`-NSTextInsertionPointBlinkPeriodOn 100000`, a +# UserDefaults argument domain's `-key value`). `split_arguments "$@"` sets +# `pairs` and `app_arguments`. +split_arguments() { + pairs=() + app_arguments=() + while [ $# -gt 0 ]; do + if [ "$1" = -- ]; then + shift + app_arguments=("$@") + return + fi + pairs+=("$1") + shift done } diff --git a/drive/launchers/web_launcher.sh b/drive/launchers/web_launcher.sh index e00cfcc2..447105f3 100755 --- a/drive/launchers/web_launcher.sh +++ b/drive/launchers/web_launcher.sh @@ -1,8 +1,11 @@ #!/bin/bash # A web app, headful, for a test's HeadlessApp (docs/drive.md): its site # (built with --@isomer//drive:headful) in headless Chrome under -# browser_check, which keeps the page open until this is stopped and takes -# the test's pictures (`--requests`). The page dials out to the test's hub +# browser_check, which keeps the page open until this is stopped (SIGTERM, +# or HeadlessApp's `stop` request) and takes the test's pictures +# (`--requests`). Each launch's Chrome has a profile of its own: the page +# starts with nothing kept, whatever ISOMER_FRESH says, and takes no +# arguments after `--`. The page dials out to the test's hub # (ISO_DRIVER, ISO_DRIVER_NAME: the page's ISO_ query, which the web host # reads as the app's environment). # @@ -21,6 +24,7 @@ width=390 height=844 query="${ISOMER_WEB_QUERY:-renderer=dom}&ISO_DRIVER=$url&ISO_DRIVER_NAME=$name" for pair in "$@"; do + [ "$pair" != -- ] || break case "${pair%%=*}" in ISO_HEADLESS_WIDTH) width="${pair#*=}" ;; ISO_HEADLESS_HEIGHT) height="${pair#*=}" ;; diff --git a/drive/launchers/windows_launcher.sh b/drive/launchers/windows_launcher.sh index 9c08d126..57328bb3 100755 --- a/drive/launchers/windows_launcher.sh +++ b/drive/launchers/windows_launcher.sh @@ -23,6 +23,8 @@ app="${ISOMER_WINDOWS_APP:?ISOMER_WINDOWS_APP: the Windows app}" dir="$(cd "$(dirname "$app")" && { pwd -W 2> /dev/null || pwd; })" environment=(ISO_DRIVER="$url" ISO_DRIVER_NAME="$name") for pair in "$@"; do + # (An app's arguments after `--` aren't passed on here yet.) + [ "$pair" != -- ] || break case "${pair%%=*}" in ISO_HEADLESS_WIDTH | ISO_HEADLESS_HEIGHT) ;; *) environment+=("$pair") ;; diff --git a/examples/apple/driver_probe_ios_application/BUILD.bazel b/examples/apple/driver_probe_ios_application/BUILD.bazel new file mode 100644 index 00000000..61bde66a --- /dev/null +++ b/examples/apple/driver_probe_ios_application/BUILD.bazel @@ -0,0 +1,41 @@ +load("@build_bazel_rules_apple//apple:ios.bzl", "ios_application") +load("@rules_swift//swift:swift.bzl", "swift_library") + +# The driver probe (//headless/driver_probe: views shaped as an app's are) on +# Apple's SwiftUI, for the iPhone's simulator: the app its tests drive +# through Apple's driver (drive/apple) when it is built with +# --@isomer//drive:headful, as an app's UI tests drive it +# (//headless/driver_probe:driver_probe_headful_apple_runner). Twice: upright, +# and turned to landscape by the app itself (its Info.plist allows +# landscape alone), on a simulator that stands upright, as an app's proof +# on its side runs. No profile: the simulator's alone. +# +# bazel build --@isomer//drive:headful --ios_multi_cpus=sim_arm64 //apple/driver_probe_ios_application/... +swift_library( + name = "driver_probe_ios_lib", + srcs = ["//headless/driver_probe:DriverProbe.swift"], + module_name = "DriverProbeIOS", + tags = ["manual"], + deps = [ + "@isomer//apple/libraries/swift_ui", + "@isomer//apple/libraries/swift_ui:isomer", + ], +) + +[ + ios_application( + name = name, + bundle_id = bundle_id, + bundle_name = "DriverProbe", + families = ["iphone"], + infoplists = [plist], + minimum_os_version = "17.0", + tags = ["manual"], + visibility = ["//visibility:public"], + deps = [":driver_probe_ios_lib"], + ) + for name, bundle_id, plist in [ + ("driver_probe_ios_application", "com.LoganShire.IsomerExamples.DriverProbe", "Info.portrait.plist"), + ("driver_probe_landscape_ios_application", "com.LoganShire.IsomerExamples.DriverProbeLandscape", "Info.landscape.plist"), + ] +] diff --git a/examples/apple/driver_probe_ios_application/Info.landscape.plist b/examples/apple/driver_probe_ios_application/Info.landscape.plist new file mode 100644 index 00000000..0461398f --- /dev/null +++ b/examples/apple/driver_probe_ios_application/Info.landscape.plist @@ -0,0 +1,20 @@ + + + + + CFBundlePackageType + APPL + CFBundleShortVersionString + 1.0 + CFBundleVersion + 1 + UILaunchScreen + + UIRequiresFullScreen + + UISupportedInterfaceOrientations + + UIInterfaceOrientationLandscapeRight + + + diff --git a/examples/apple/driver_probe_ios_application/Info.portrait.plist b/examples/apple/driver_probe_ios_application/Info.portrait.plist new file mode 100644 index 00000000..4a28fcec --- /dev/null +++ b/examples/apple/driver_probe_ios_application/Info.portrait.plist @@ -0,0 +1,20 @@ + + + + + CFBundlePackageType + APPL + CFBundleShortVersionString + 1.0 + CFBundleVersion + 1 + UILaunchScreen + + UIRequiresFullScreen + + UISupportedInterfaceOrientations + + UIInterfaceOrientationPortrait + + + diff --git a/examples/headless/driver_probe/BUILD.bazel b/examples/headless/driver_probe/BUILD.bazel index 31f894d1..5bf2f5b4 100644 --- a/examples/headless/driver_probe/BUILD.bazel +++ b/examples/headless/driver_probe/BUILD.bazel @@ -1,3 +1,5 @@ +load("@build_bazel_rules_apple//apple:macos.bzl", "macos_command_line_application") +load("@rules_swift//swift:swift.bzl", "swift_library") load("@rules_swift//swift:swift_binary.bzl", "swift_binary") load("@rules_swift//swift:swift_test.bzl", "swift_test") @@ -9,6 +11,12 @@ load("@rules_swift//swift:swift_test.bzl", "swift_test") # flag, as //headless/... is run: # # bazel test --@isomer//headless //headless/driver_probe:driver_probe_test +# Also the iPhone probe's source, on Apple's SwiftUI. +exports_files( + ["DriverProbe.swift"], + visibility = ["//apple/driver_probe_ios_application:__pkg__"], +) + swift_binary( name = "driver_probe", srcs = ["DriverProbe.swift"], @@ -30,3 +38,47 @@ swift_test( module_name = "DriverProbeTests", deps = ["@isomer//headless/isomer_headless"], ) + +# The same tests against the iPhone's probe on Apple's SwiftUI +# (//apple/driver_probe_ios_application, built with --@isomer//drive:headful), +# through Apple's driver, upright and turned to landscape by the app +# (:driver_probe_landscape_ios_application): a Mac command, built in CI's +# container and run on GitHub's Mac with the apps on a simulator, as the +# Storybook's (tools/ci/hosted_headful_test.sh): +# +# ISOMER_HEADFUL_LAUNCHER= ISOMER_SIMULATOR= \ +# ISOMER_BUNDLE_ID=com.LoganShire.IsomerExamples.DriverProbe \ +# ISOMER_LANDSCAPE_BUNDLE_ID=com.LoganShire.IsomerExamples.DriverProbeLandscape driver_probe_headful_apple_runner +# +# Swift Testing's main written out (as the Storybook's runner's), the +# frameworks from the Mac's Xcode at run time (DYLD_FRAMEWORK_PATH). +genrule( + name = "test_main", + outs = ["test_main/main.swift"], + cmd = """ +{ + echo 'import Testing' + echo 'await Testing.__swiftPMEntryPoint() as Never' +} > $@ +""", +) + +swift_library( + name = "driver_probe_headful_apple_tests", + testonly = True, + srcs = [ + "DriverProbeTests.swift", + ":test_main", + ], + module_name = "DriverProbeHeadfulAppleTests", + tags = ["manual"], + deps = ["@isomer//headless/isomer_headless"], +) + +macos_command_line_application( + name = "driver_probe_headful_apple_runner", + testonly = True, + minimum_os_version = "15.0", + tags = ["manual"], + deps = [":driver_probe_headful_apple_tests"], +) diff --git a/examples/headless/driver_probe/DriverProbe.swift b/examples/headless/driver_probe/DriverProbe.swift index a7c3a449..4a81e71f 100644 --- a/examples/headless/driver_probe/DriverProbe.swift +++ b/examples/headless/driver_probe/DriverProbe.swift @@ -3,8 +3,15 @@ // lost it before (an identifier outside `.padding`, `.background` or an app's // own modifier; a selection list's row; a combined card; a toggle; a bottom // bar's icon button; a disabled toolbar item; a page pushed in a sheet; a -// long press). PROBE_SCREEN picks the screen: the controls (the default), the -// list, or the sheet's. +// long press; controls in a Form's rows, a text field and a multi-line one, +// the navigation title's own button; sheets closed by `dismiss` alone, and +// one presented over another; the scene's phase, and what an app does on +// coming back to the foreground). PROBE_SCREEN picks the screen: the +// controls (the default), the list, the sheet's, the form, the sheets or the +// scene. The same +// source is the iPhone's probe on Apple's SwiftUI +// (//apple/driver_probe_ios_application), which its tests drive through +// Apple's driver, upright and turned to landscape by the app itself. import Foundation import SwiftUI @@ -16,6 +23,9 @@ struct DriverProbe: App { switch ProcessInfo.processInfo.environment["PROBE_SCREEN"] ?? "" { case "list": ListScreen() case "sheet": SheetScreen() + case "form": FormScreen() + case "sheets": SheetsScreen() + case "scene": SceneScreen() default: ControlsScreen() } } @@ -134,3 +144,141 @@ struct SheetScreen: View { } } } + +/// A form's rows as an app's settings are: controls with identifiers in +/// them, a text field, a multi-line field, and the navigation title's own +/// button. +struct FormScreen: View { + @State private var last = "nothing yet" + @State private var sync = false + @State private var host = "" + @State private var message = "" + @AppStorage("probe.launches") private var launches = 0 + /// A launch argument (`-probe.argument `, UserDefaults' argument + /// domain), shown. + private let argument = UserDefaults.standard.string(forKey: "probe.argument") ?? "none" + + var body: some View { + NavigationStack { + Form { + Section { + Text("Last: \(last)") + Text("Argument: \(argument)") + Text("Launches: \(launches)") + } + Section("Computers") { + Button("Add Computer") { last = "add computer" } + .accessibilityIdentifier("add-computer") + HStack { + Text("Mac Studio") + Spacer() + Button("Connect") { last = "connect" } + .accessibilityIdentifier("connect") + } + Toggle("Sync", isOn: $sync) + .accessibilityIdentifier("sync") + NavigationLink("Runner") { + Text("Runner settings") + .navigationTitle("Runner") + } + .accessibilityIdentifier("runner") + } + Section("Connection") { + TextField("Host name", text: $host) + .accessibilityIdentifier("host") + TextField("Message", text: $message, axis: .vertical) + Text("Host: \(host.isEmpty ? "none" : host), message: \(message.isEmpty ? "none" : message)") + } + } + .navigationTitle("Settings") + .toolbar { + ToolbarItem(placement: .principal) { + Button("Session title") { last = "title" } + .accessibilityIdentifier("session-title") + } + } + } + .onAppear { launches += 1 } + } +} + +/// Sheets an app presents: one with only Done, one with nothing to close it +/// but a swipe (closed by the driver's `dismiss`), and one presented from +/// inside another. +struct SheetsScreen: View { + @State private var done = false + @State private var question = false + @State private var outer = false + @State private var inner = false + @State private var last = "nothing yet" + + var body: some View { + NavigationStack { + VStack(spacing: 12) { + Text("Last: \(last)") + Button("Tool calls") { done = true } + Button("Question") { question = true } + Button("Settings…") { outer = true } + } + .navigationTitle("Sheets") + } + .sheet(isPresented: $done) { + NavigationStack { + Text("Every tool call") + .navigationTitle("Tool calls") + .toolbar { + ToolbarItem(placement: .confirmationAction) { + Button("Done") { done = false } + } + } + } + } + .sheet(isPresented: $question, onDismiss: { last = "question closed" }) { + VStack { + Text("Which model?") + Text("Pick one, or swipe down") + } + } + .sheet(isPresented: $outer) { + VStack(spacing: 12) { + Text("Outer sheet") + Button("Add Computer") { inner = true } + .accessibilityIdentifier("add-computer") + } + .sheet(isPresented: $inner) { + Text("Inner sheet") + } + } + } +} + +/// The scene's phase as the app sees it, each change it was told of, and +/// what an app does on coming back from the background (a sign-in prompt +/// opened on `.onChange(of: scenePhase)`). +struct SceneScreen: View { + @Environment(\.scenePhase) private var phase + @State private var changes: [String] = [] + @State private var welcome = false + + var body: some View { + NavigationStack { + VStack(alignment: .leading, spacing: 12) { + Text("Phase: \(name(phase))") + Text("Changes: \(changes.isEmpty ? "none" : changes.joined(separator: " "))") + if welcome { + Text("Welcome back") + Button("Sign in") { welcome = false } + .accessibilityIdentifier("sign-in") + } + } + .padding() + .navigationTitle("Scene") + } + .onChange(of: phase) { old, new in + changes.append(name(new)) + if old == .background, new != .background { welcome = true } + } + } + + private func name(_ phase: ScenePhase) -> String { "\(phase)" } +} diff --git a/examples/headless/driver_probe/DriverProbeTests.swift b/examples/headless/driver_probe/DriverProbeTests.swift index 6c2f6eb9..9677d450 100644 --- a/examples/headless/driver_probe/DriverProbeTests.swift +++ b/examples/headless/driver_probe/DriverProbeTests.swift @@ -2,6 +2,12 @@ // app's UI tests found them wanting: each finds an element by the identifier // or label SwiftUI gives it, and uses it, as XCUITest would on Apple's // SwiftUI. +// +// Headless (the probe built headless, :driver_probe_test), and headful when +// ISOMER_HEADFUL_LAUNCHER names a launcher: the iPhone's probe on Apple's +// SwiftUI through Apple's driver (:driver_probe_headful_apple_runner, on +// GitHub's Mac in a run by hand, tools/ci/hosted_headful_test.sh), with +// ISOMER_LANDSCAPE_BUNDLE_ID the same probe turned to landscape by itself. import Foundation import IsomerHeadless @@ -9,11 +15,26 @@ import Testing @Suite(.serialized) struct DriverProbeTests { - private func launch(_ screen: String = "") throws -> HeadlessApp { + private var headful: Bool { !(ProcessInfo.processInfo.environment["ISOMER_HEADFUL_LAUNCHER"] ?? "").isEmpty } + + /// The probe on `screen`: headful through the launcher the environment + /// names (`bundle`, another app than ISOMER_BUNDLE_ID's: the landscape + /// probe), else built headless. + private func launch(_ screen: String = "", arguments: [String] = [], fresh: Bool = true, + bundle: String? = nil) throws -> HeadlessApp { let environment = ProcessInfo.processInfo.environment + if let launcher = environment["ISOMER_HEADFUL_LAUNCHER"], !launcher.isEmpty { + // The launcher reads the app's bundle from its environment, which + // it takes from this process's as it starts. + let saved = environment["ISOMER_BUNDLE_ID"] + if let bundle { setenv("ISOMER_BUNDLE_ID", bundle, 1) } + defer { if bundle != nil { if let saved { setenv("ISOMER_BUNDLE_ID", saved, 1) } else { unsetenv("ISOMER_BUNDLE_ID") } } } + return try HeadlessApp(launcher: launcher, environment: ["PROBE_SCREEN": screen], arguments: arguments, + fresh: fresh) + } let root = "\(environment["TEST_SRCDIR"] ?? "")/\(environment["TEST_WORKSPACE"] ?? "_main")" return try HeadlessApp(executable: "\(root)/headless/driver_probe/driver_probe", - environment: ["PROBE_SCREEN": screen]) + environment: ["PROBE_SCREEN": screen], arguments: arguments) } /// An identifier or a label applied outside what wraps a button @@ -127,8 +148,160 @@ struct DriverProbeTests { @Test func aLongPressFiresTheGesture() throws { let app = try launch() defer { app.quit() } + // Apple's driver has no long press it can use in process (drive/apple). + guard try app.info().renderer != "SwiftUI" else { return } try app.longPress("Hold to pin") try app.wait(for: "Last: pinned") #expect(throws: (any Error).self) { try app.longPress("Mac Studio") } } + + /// Controls in a Form's rows, a text field and the navigation title's + /// own button, each found by its identifier, as XCUITest finds them. + @Test func aFormsRowsCarryTheirIdentifiers() throws { + let app = try launch("form") + defer { app.quit() } + let screen = try app.screen() + for (id, role) in [("add-computer", "button"), ("connect", "button"), ("sync", "toggle"), ("runner", "button"), + ("host", "textField"), ("session-title", "button")] { + #expect(screen.element(id)?.role == role, "\(id): \(screen.elements)") + } + try app.tap("add-computer") + try app.wait(for: "Last: add computer") + try app.tap("connect") + try app.wait(for: "Last: connect") + try app.tap("session-title") + try app.wait(for: "Last: title") + try app.type("host", "studio.local") + try app.wait(for: "Host: studio.local") + try app.tap("runner") + try app.wait(for: "Runner settings") + } + + /// A multi-line text field (`axis: .vertical`) is labelled by its + /// title, as a one-line field is. + @Test func aMultiLineFieldIsLabelled() throws { + let app = try launch("form") + defer { app.quit() } + let screen = try app.screen() + let field = try #require(screen.element("Message"), "\(screen.elements)") + #expect(field.role == "textField") + try app.type("Message", "Hello") + try app.wait(for: "message: Hello") + } + + /// `dismiss` closes the topmost sheet, whatever closes it for a person: + /// its Done, or only a swipe down. + @Test func dismissClosesSheets() throws { + let app = try launch("sheets") + defer { app.quit() } + var screen = try app.tap("Tool calls") + #expect(screen.presented.count == 1, "\(screen.presented)") + screen = try app.dismiss() + #expect(screen.presented.isEmpty, "\(screen.presented)") + #expect(screen.shows("Last: nothing yet")) + screen = try app.tap("Question") + #expect(screen.shows("Which model?")) + screen = try app.dismiss() + #expect(screen.presented.isEmpty, "\(screen.presented)") + // As a swipe closes it: the sheet's onDismiss runs. + try app.wait(for: "Last: question closed") + } + + /// A sheet presented from a sheet is on the screen once the tap that + /// presents it answers. + @Test func aSheetOverASheetIsThereWhenTheTapAnswers() throws { + let app = try launch("sheets") + defer { app.quit() } + try app.tap("Settings…") + let screen = try app.tap("add-computer") + #expect(screen.shows("Inner sheet"), "\(screen.texts)") + #expect(screen.presented.count == 2, "\(screen.presented)") + _ = try app.dismiss() + #expect(try app.screen().shows("Outer sheet")) + } + + /// The app's own arguments (`-key value`: UserDefaults' argument + /// domain) reach it. + @Test func launchArgumentsReachTheApp() throws { + let app = try launch("form", arguments: ["-probe.argument", "hello"]) + defer { app.quit() } + try app.wait(for: "Argument: hello") + } + + /// A launch starts fresh (the app's defaults and files from before + /// gone), unless the test asks to keep them. + @Test func aLaunchStartsFreshUnlessAskedToKeep() throws { + // Headless, the app's defaults are the building machine's: what + // lasts there is not the launcher's to clear. + guard headful else { return } + for (fresh, launches) in [(true, 1), (false, 2), (true, 1)] { + let app = try launch("form", fresh: fresh) + defer { app.quit() } + try app.wait(for: "Launches: \(launches)") + } + } + + /// `scene` moves the app's scene phase as leaving it and coming back + /// does: its `.onChange(of: scenePhase)` sees each step, and what it + /// opens on coming back (a sign-in prompt) is there. + @Test func sceneLeavesAndComesBack() throws { + let app = try launch("scene") + defer { app.quit() } + try app.wait(for: "Phase: active") + try app.scene(.background) + // Through inactive, as the platform goes (the launcher's or the + // driver's way alike). + try app.scene(.active) + let screen = try app.wait(for: "Welcome back") + #expect(screen.shows("Phase: active"), "\(screen.texts)") + #expect(screen.texts.contains { $0.hasPrefix("Changes: ") && $0.contains("background") && $0.hasSuffix("active") }, + "\(screen.texts)") + #expect(screen.element("sign-in") != nil, "\(screen.elements)") + try app.tap("sign-in") + // Inactive alone (a notification's banner pulled down, the app + // switcher) is no return from the background. + try app.scene(.inactive) + try app.wait(for: "Phase: inactive") + try app.scene(.active) + let back = try app.wait(for: "Phase: active") + #expect(!back.shows("Welcome back"), "\(back.texts)") + } + + /// An iPhone app that turns itself to landscape on an upright phone: + /// taps reach its rows, its info says how it stands, and its pictures + /// are turned as a person sees them. + @Test func anAppTurnedToLandscape() throws { + guard let bundle = ProcessInfo.processInfo.environment["ISOMER_LANDSCAPE_BUNDLE_ID"], headful else { return } + let app = try launch("form", bundle: bundle) + defer { app.quit() } + let info = try app.info() + #expect(info.width > info.height, "\(info)") + #expect(info.orientation.hasPrefix("landscape"), "\(info)") + try app.tap("add-computer") + try app.wait(for: "Last: add computer") + try app.tap("Runner") + try app.wait(for: "Runner settings") + try app.back() + let id = try #require(try app.picture("form")) + #expect(id.hasSuffix("/phone-landscape/form")) + let environment = ProcessInfo.processInfo.environment + let proof = environment["ISOMER_PROOF"] ?? "\(environment["TEST_UNDECLARED_OUTPUTS_DIR"] ?? NSTemporaryDirectory())/proof" + let png = try Data(contentsOf: URL(fileURLWithPath: "\(proof)/screens/\(id).png")) + // The PNG's IHDR: width, then height, big-endian, from byte 16. + func number(_ at: Int) -> Int { png[at.. number(20), "the picture stands on its side: \(number(16))×\(number(20))") + } + + /// A selection list's rows take no activation of their own: the tap + /// lands where the row stands, in a window the app turned to landscape + /// on an upright phone as in one upright. + @Test func aRowTakesATapInAnAppTurnedToLandscape() throws { + guard let bundle = ProcessInfo.processInfo.environment["ISOMER_LANDSCAPE_BUNDLE_ID"], headful else { return } + let app = try launch("list", bundle: bundle) + defer { app.quit() } + try app.tap("server-gamma") + try app.wait(for: "Selected: gamma") + try app.tap("Beta") + try app.wait(for: "Selected: beta") + } } diff --git a/examples/headless/storybook_headless_application/StorybookHeadlessTests.swift b/examples/headless/storybook_headless_application/StorybookHeadlessTests.swift index a2086eb1..b5fe2130 100644 --- a/examples/headless/storybook_headless_application/StorybookHeadlessTests.swift +++ b/examples/headless/storybook_headless_application/StorybookHeadlessTests.swift @@ -115,6 +115,21 @@ struct StorybookHeadlessTests { try app.picture("state.observableobject") } + /// A trip to the background and back (`scene`): the app comes back as + /// it was, still driven, whichever way its platform moves it (Android's + /// launcher sends it home and brings it back; iOS tells SwiftUI's scene + /// delegate; the Mac hides it; Isomer's own runtimes set the phase). + @Test func comesBackFromTheBackground() throws { + let app = try launch(story: "state.observableobject") + defer { app.quit() } + try app.tap("Tap") + try app.wait(for: "Taps: 1") + try app.scene(.background) + try app.scene(.active) + try app.tap("Tap") + try app.wait(for: "Taps: 2") + } + @Test func textFieldBindsTheModel() throws { let app = try launch(story: "state.observableobject") defer { app.quit() } @@ -328,6 +343,42 @@ struct StorybookHeadlessTests { #expect(throws: (any Error).self) { try app.swipe("Weekly report", "left", action: "Archive") } } + /// A headful launch starts with none of the app's data from before (the + /// launchers' ISOMER_FRESH, HeadlessApp's default), as an app built + /// headless does; asked to, it keeps it. + @Test func aLaunchStartsFreshUnlessAskedToKeep() throws { + // (Hosted in the iPhone app, there is no launcher to start one.) + #if !(os(iOS) || os(tvOS) || os(watchOS) || os(visionOS)) + let environment = ProcessInfo.processInfo.environment + guard let launcher = environment["ISOMER_HEADFUL_LAUNCHER"], !launcher.isEmpty, + HeadlessApp.hostedIfAny() == nil else { return } + let story = ["ISO_STORYBOOK_STORY": "state.appstorage"] + var platform = "" + do { + let app = try HeadlessApp(launcher: launcher, environment: story) + defer { app.quit() } + platform = try app.info().platform + // Windows' launcher keeps an app's data as it is, for now. + guard platform != "Windows" else { return } + try app.wait(for: "Starred 0 times") + try app.tap("Star") + try app.wait(for: "Starred 1 times") + } + // Kept where the platform's app keeps its defaults between launches + // for sure: not a web page (a browser profile of its own each + // launch), nor Linux's app (corelibs' UserDefaults, which a launcher + // stopping the app with SIGTERM gives no time to write). + if ["iOS", "Android", "macOS"].contains(platform) { + let app = try HeadlessApp(launcher: launcher, environment: story, fresh: false) + defer { app.quit() } + try app.wait(for: "Starred 1 times") + } + let app = try HeadlessApp(launcher: launcher, environment: story) + defer { app.quit() } + try app.wait(for: "Starred 0 times") + #endif + } + @Test func unknownTargetFailsWithTheScreen() throws { let app = try launch(story: "state.observableobject") defer { app.quit() } diff --git a/headless/isomer_headless/HeadlessApp.swift b/headless/isomer_headless/HeadlessApp.swift index a3b24a5c..6b6034d2 100644 --- a/headless/isomer_headless/HeadlessApp.swift +++ b/headless/isomer_headless/HeadlessApp.swift @@ -155,6 +155,9 @@ public struct HeadlessInfo: Sendable { public let renderer: String public let width: Double public let height: Double + /// The interface's orientation where the platform turns it apart from + /// the device (iOS: portrait, landscape-left, …), else empty. + public let orientation: String } /// An app driven a command at a time, over one call that lasts as long as @@ -307,15 +310,25 @@ public final class HeadlessApp: @unchecked Sendable { /// process (a WebSocket on the loopback address, its port the system's /// choice). The launcher is run as /// - /// [KEY=VALUE…] + /// [KEY=VALUE…] [-- …] /// /// with `environment` as its KEY=VALUE arguments (the app's launch hooks: /// ISO_STORYBOOK_STORY, ISO_HEADLESS_WIDTH and ISO_HEADLESS_HEIGHT for - /// its window), and must start the app with ISO_DRIVER set to the URL and - /// ISO_DRIVER_NAME to the name (a web page's ISO_ query, an Android - /// intent's extras), then run until it is stopped (SIGTERM), when it ends - /// the app. `quit()` stops it. The app has `timeout` seconds to attach. - public convenience init(launcher: String, environment: [String: String] = [:], timeout: Double = 120) throws { + /// its window) and `arguments` after `--` (the app's command line: a + /// UserDefaults `-key value`), and must start the app with ISO_DRIVER set + /// to the URL and ISO_DRIVER_NAME to the name (a web page's ISO_ query, + /// an Android intent's extras), then run until it is stopped (SIGTERM, + /// or a file named `stop` among its requests), when it ends the app. + /// `quit()` stops it. The app has `timeout` seconds to attach. + /// + /// `fresh` (the default) starts the app with none of its data from + /// before, as an app built headless always starts: the launcher's + /// ISOMER_FRESH (the iPhone's: its defaults, files, app groups and the + /// simulator's keychain cleared; Android's `pm clear`; the Mac's a new + /// home; the web's a new browser profile, whatever this says). A test of + /// what lasts between launches passes `fresh: false` for the second. + public convenience init(launcher: String, environment: [String: String] = [:], arguments: [String] = [], + fresh: Bool = true, timeout: Double = 120) throws { let hub = try HeadfulHub.shared() // A launch that never attaches is tried once more: a simulator now // and then loses a launch made just as the app before it quits. The @@ -323,7 +336,8 @@ public final class HeadlessApp: @unchecked Sendable { var failure: HeadlessError? for (attempt, wait) in [timeout / 2, timeout].enumerated() { let name = HeadfulHub.nextName() - let (process, requests) = try Self.startLauncher(launcher, hub: hub.url, name: name, environment: environment) + let (process, requests) = try Self.startLauncher(launcher, hub: hub.url, name: name, environment: environment, + arguments: arguments, fresh: fresh) let attached = try Self.blocking { () async -> Bool in let deadline = Date().addingTimeInterval(wait) while Date() < deadline { @@ -355,10 +369,11 @@ public final class HeadlessApp: @unchecked Sendable { /// The launcher run for one launch: the hub's URL, the app's name and /// `environment` as its arguments, and where the test asks it for /// pictures (`picture`). - private static func startLauncher(_ launcher: String, hub: String, name: String, - environment: [String: String]) throws -> (Process, URL) { + private static func startLauncher(_ launcher: String, hub: String, name: String, environment: [String: String], + arguments appArguments: [String], fresh: Bool) throws -> (Process, URL) { let process = Process() let arguments = [hub, name] + environment.sorted { $0.key < $1.key }.map { "\($0.key)=\($0.value)" } + + (appArguments.isEmpty ? [] : ["--"] + appArguments) #if os(Windows) // A launcher script through Git's bash (ISOMER_BASH, else Git for // Windows' own), as Windows runs no script by its #! line. @@ -371,7 +386,8 @@ public final class HeadlessApp: @unchecked Sendable { #endif let requests = FileManager.default.temporaryDirectory.appendingPathComponent("isomer-requests-\(name)") try? FileManager.default.createDirectory(at: requests, withIntermediateDirectories: true) - process.environment = ProcessInfo.processInfo.environment.merging(["ISOMER_LAUNCHER_REQUESTS": requests.path]) { $1 } + process.environment = ProcessInfo.processInfo.environment.merging( + ["ISOMER_LAUNCHER_REQUESTS": requests.path, "ISOMER_FRESH": fresh ? "1" : "0"]) { $1 } try process.run() return (process, requests) } @@ -379,6 +395,14 @@ public final class HeadlessApp: @unchecked Sendable { deinit { if let call { Task { await call.finish() } } + // A headful app's launcher a test left running (it threw before its + // `quit`): stopped, so it doesn't outlive the test with its app. + #if !(os(iOS) || os(tvOS) || os(watchOS) || os(visionOS)) + if let launched, launched.isRunning { + if let requests { try? Data().write(to: requests.appendingPathComponent("stop")) } + launched.terminate() + } + #endif } /// The screen as it is. @@ -390,7 +414,8 @@ public final class HeadlessApp: @unchecked Sendable { let answer = try exchange("info") let info = answer.info return HeadlessInfo(name: info.name, bundleIdentifier: info.bundleIdentifier, version: info.version, - platform: info.platform, renderer: info.renderer, width: info.width, height: info.height) + platform: info.platform, renderer: info.renderer, width: info.width, height: info.height, + orientation: info.orientation) } /// One element or text in full (`inspect`): its kind, actions, @@ -426,9 +451,9 @@ public final class HeadlessApp: @unchecked Sendable { /// A picture of the screen as it stands once settled, kept as a proof's /// screen (docs/drive.md, "Pictures"): its id is /// `//` (`ios/phone/chat`), the platform the - /// app's, the device `device` or the platform's own (ios: phone, or - /// phone-landscape in a landscape window; macos, linux, windows: window; - /// android: phone; web: desktop). The launcher takes it (simctl, the + /// app's, the device `device` or the platform's own (ios and android: + /// phone, or phone-landscape in a landscape window; macos, linux, + /// windows: window; web: desktop). The launcher takes it (simctl, the /// Mac's window, adb, Chrome) into ISOMER_PROOF (else the test's /// outputs' proof folder), screens/.png; a test hosted in its app /// draws the app's own window there (HostedPicture.swift) and hands it @@ -444,8 +469,7 @@ public final class HeadlessApp: @unchecked Sendable { let landscape = info.width > info.height let device = device ?? { switch platform { - case "ios": landscape ? "phone-landscape" : "phone" - case "android": "phone" + case "ios", "android": landscape ? "phone-landscape" : "phone" case "web": "desktop" default: "window" } @@ -461,8 +485,8 @@ public final class HeadlessApp: @unchecked Sendable { let file = proof.appendingPathComponent("screens/\(id).png") try FileManager.default.createDirectory(at: file.deletingLastPathComponent(), withIntermediateDirectories: true) try? FileManager.default.removeItem(at: file) - // The screen settled first. - _ = try screen() + // The screen settled first (the driver settles before it answers + // `info`, above). if hosted { // A test hosted in its app: the app's own window, drawn here. let png = Thread.isMainThread @@ -477,7 +501,10 @@ public final class HeadlessApp: @unchecked Sendable { // The launcher's picture: a request naming the file, which it writes. asked += 1 let request = requests.appendingPathComponent("\(asked).request") - try Data(file.path.utf8).write(to: request.appendingPathExtension("part")) + // The picture's path, then what the launcher may need to take it as + // the app stands (requests.sh). + let text = file.path + "\n" + (info.orientation.isEmpty ? "" : "orientation=\(info.orientation)\n") + try Data(text.utf8).write(to: request.appendingPathExtension("part")) try FileManager.default.moveItem(at: request.appendingPathExtension("part"), to: request) let deadline = Date().addingTimeInterval(60) while Date() < deadline { @@ -487,7 +514,7 @@ public final class HeadlessApp: @unchecked Sendable { return FileManager.default.fileExists(atPath: file.path) ? id : nil } if !launched.isRunning { break } - Thread.sleep(forTimeInterval: 0.1) + Thread.sleep(forTimeInterval: 0.02) } throw HeadlessError(message: "the launcher took no picture of \(id)", screen: nil) } @@ -551,6 +578,41 @@ public final class HeadlessApp: @unchecked Sendable { @discardableResult public func dismiss() throws -> HeadlessScreen { try send("dismiss") } + /// A scene's phase, as SwiftUI's `ScenePhase` has them. + public enum ScenePhase: String, Sendable { + case active, inactive, background + } + + /// Moves the app's scene phase as leaving the app and coming back does, + /// so what it does then (`.onChange(of: scenePhase)`) runs: `background` + /// then `active` is a trip to the home screen and back. The launcher's + /// way first, where the platform's own moves it (Android's: the app sent + /// home and brought back, its activity stopped and resumed); else the + /// app's driver's (headless.proto's `scene`: Isomer's runtime's phase, + /// SwiftUI's scene delegate told on iOS, the app hidden or its windows + /// in the Dock on the Mac). The screen once it settles. + @discardableResult + public func scene(_ phase: ScenePhase) throws -> HeadlessScreen { + if local == nil, let requests, let launched, launched.isRunning { + asked += 1 + let request = requests.appendingPathComponent("\(asked).request") + let answer = requests.appendingPathComponent("\(asked).answer") + try Data("scene:\(phase.rawValue)\n".utf8).write(to: request.appendingPathExtension("part")) + try FileManager.default.moveItem(at: request.appendingPathExtension("part"), to: request) + // Answered (requests.sh): `ok` in .answer when the launcher + // moved it; why not, or nothing, when it didn't. + let deadline = Date().addingTimeInterval(30) + while FileManager.default.fileExists(atPath: request.path), Date() < deadline, launched.isRunning { + Thread.sleep(forTimeInterval: 0.02) + } + try? FileManager.default.removeItem(at: request) + let said = (try? String(contentsOf: answer, encoding: .utf8))?.trimmingCharacters(in: .whitespacesAndNewlines) + try? FileManager.default.removeItem(at: answer) + if said == "ok" { return try screen() } + } + return try send("scene") { $0.value = phase.rawValue } + } + /// Waits until a text shows (or a control's label or value holds it). @discardableResult public func wait(for text: String, timeout: Double = 5) throws -> HeadlessScreen { @@ -568,14 +630,22 @@ public final class HeadlessApp: @unchecked Sendable { _ = try? exchange("quit") if let call { try? Self.blocking { await call.finish() } } if let launched, launched.isRunning { - // A `stop` file beside the picture requests first: a launcher on - // Windows, which has no SIGTERM to trap, ends its app on seeing it. + // A `stop` file beside the picture requests, which every launcher + // of isomer's watches (requests.sh): on Windows, which has no + // SIGTERM to trap, it is how the launcher ends its app. if let requests { try? Data().write(to: requests.appendingPathComponent("stop")) } + #if os(Windows) var deadline = Date().addingTimeInterval(10) - while launched.isRunning, Date() < deadline { Thread.sleep(forTimeInterval: 0.1) } + while launched.isRunning, Date() < deadline { Thread.sleep(forTimeInterval: 0.05) } if launched.isRunning { launched.terminate() } + #else + // Elsewhere SIGTERM at once too: a launcher that doesn't watch + // for the file (an app's own) isn't waited for. + launched.terminate() + var deadline = Date() + #endif deadline = Date().addingTimeInterval(10) - while launched.isRunning, Date() < deadline { Thread.sleep(forTimeInterval: 0.1) } + while launched.isRunning, Date() < deadline { Thread.sleep(forTimeInterval: 0.05) } #if !os(Windows) if launched.isRunning { kill(launched.processIdentifier, SIGKILL) } #endif diff --git a/shared/wire/headless.proto b/shared/wire/headless.proto index 6799fe83..9bfe2b35 100644 --- a/shared/wire/headless.proto +++ b/shared/wire/headless.proto @@ -17,7 +17,8 @@ option java_multiple_files = true; message Command { // ls, info, inspect, tap, type, set, select, swipe, longpress, back, - // dismiss, wait or quit (drive/driver/IsomerAppDriver.swift says each). + // dismiss, scene, wait or quit (drive/driver/IsomerAppDriver.swift says + // each). string cmd = 1; // The element: its name on the screen (`button-3`, `text-2`: its role and // its place among that role's, from 1), its accessibility identifier, its @@ -28,7 +29,15 @@ message Command { string text = 3; // `set`'s value (a toggle's "1" or "0", a picker's index); `select`'s // option, by its label; `swipe`'s action on a list row (one of its - // `.swipeActions`, by its title), performed as the swipe reveals it. + // `.swipeActions`, by its title), performed as the swipe reveals it; + // `scene`'s phase: active, inactive or background, the app's scene moved + // as leaving the app and coming back moves it (`@Environment(\.scenePhase)` + // and its `.onChange`): Isomer's runtime's phase set (headless, the web, + // Linux, Windows, and Android when no launcher took it); on iOS SwiftUI's + // scene delegate told each step UIKit tells it; on the Mac the app hidden + // (background), its windows in the Dock (inactive) or shown again (active). + // A test's HeadlessApp.scene asks its launcher first: Android's sends the + // app home and brings it back (the activity's own lifecycle). string value = 4; // `wait`'s timeout in seconds (5 when 0). double timeout = 5; @@ -71,6 +80,13 @@ message Info { // The window, in points. double width = 6; double height = 7; + // The interface's orientation where the platform turns it apart from the + // device (iOS): portrait, portrait-upside-down, landscape-left or + // landscape-right, as UIKit's interface orientations; empty elsewhere. A + // launcher turns its pictures by it (drive/launchers/ios_launcher.sh): an + // app that turns itself to landscape on an upright phone is drawn on its + // side in the device's frame. + string orientation = 8; } message Screen { diff --git a/tools/ci/container_build.sh b/tools/ci/container_build.sh index 32918972..7307d8a3 100755 --- a/tools/ci/container_build.sh +++ b/tools/ci/container_build.sh @@ -398,6 +398,23 @@ if has apple; then else results+=(1) fi + # The driver probe on the SwiftUI of Apple (views shaped as those of an + # app), upright and turned to landscape by itself, and its tests, a Mac + # command as for the Storybook: the checks of the Apple driver itself. + probe_apps=(//apple/driver_probe_ios_application:driver_probe_ios_application + //apple/driver_probe_ios_application:driver_probe_landscape_ios_application) + probe_runner=//headless/driver_probe:driver_probe_headful_apple_runner + if bazel build "${headful[@]}" --ios_multi_cpus=sim_arm64 "${probe_apps[@]}" && bazel build "$probe_runner"; then + mkdir -p "$platforms_out/headful/probe" + for app in "${probe_apps[@]}"; do + for f in $(files "${headful[@]}" --ios_multi_cpus=sim_arm64 "$app"); do + case "$f" in *.ipa) cp "$exec_root/$f" "$platforms_out/headful/probe/" ;; esac + done + done + for f in $(files "$probe_runner"); do cp "$exec_root/$f" "$platforms_out/headful/probe/driver_probe_headful_apple_runner"; done + else + results+=(1) + fi # And the tests hosted in the iPhone app: their XCTest bundle and the # app built to host them (headful, for the simulator), which the Mac # runs with an xctestrun of its own (tools/ci/hosted_headful_test.sh), diff --git a/tools/ci/hosted_apple_test.sh b/tools/ci/hosted_apple_test.sh index b8d067cb..091bcef1 100755 --- a/tools/ci/hosted_apple_test.sh +++ b/tools/ci/hosted_apple_test.sh @@ -52,30 +52,52 @@ steady() { } failures=0 -# The iPhone: a simulator of the run's own. -device="${HOSTED_IOS_DEVICE:-iPhone 17}" -runtime="$(xcrun simctl list runtimes iOS available | sed -n 's/^iOS .* - \(com\.apple\.CoreSimulator\.SimRuntime\.iOS-[0-9-]*\)$/\1/p' | tail -1)" -[ -n "$runtime" ] || { echo "hosted_apple_test: no iOS runtime on this Mac" >&2; exit 1; } -udid="$(xcrun simctl create "isomer-test-$$" "$device" "$runtime")" || exit 1 -xcrun simctl boot "$udid" && xcrun simctl bootstatus "$udid" -b >/dev/null || exit 1 -xcrun simctl status_bar "$udid" override --time 9:41 --dataNetwork wifi --wifiMode active --wifiBars 3 \ - --cellularMode active --cellularBars 4 --operatorName "" --batteryState charged --batteryLevel 100 -xcrun simctl ui "$udid" appearance light +# The iPhone: a simulator of the run's own (tools/ci/hosted_simulator.sh, +# which starts it again should GitHub's image take it down). +. "$ROOT/tools/ci/hosted_simulator.sh" mkdir -p "$work/ios" && unzip -q "$ipa" -d "$work/ios" || exit 1 -xcrun simctl install "$udid" "$work/ios/Payload/"*.app || exit 1 +prepare_ios() { + xcrun simctl status_bar "$udid" override --time 9:41 --dataNetwork wifi --wifiMode active --wifiBars 3 \ + --cellularMode active --cellularBars 4 --operatorName "" --batteryState charged --batteryLevel 100 + xcrun simctl ui "$udid" appearance light + xcrun simctl install "$udid" "$work/ios/Payload/"*.app || return 1 + [ ! -d "$work/webview/Payload" ] || xcrun simctl install "$udid" "$work/webview/Payload/"*.app +} +hosted_sim_start "isomer-test-$$" prepare_ios || exit 1 ios_shot() { xcrun simctl io "$udid" screenshot --mask=black "$1" >/dev/null 2>&1; } -for story in "${stories[@]}"; do - xcrun simctl terminate "$udid" "$BUNDLE" >/dev/null 2>&1 - SIMCTL_CHILD_ISO_STORYBOOK_STORY="$story" xcrun simctl launch "$udid" "$BUNDLE" >/dev/null || - { echo "hosted_apple_test: ios $story: didn't launch" >&2; failures=$((failures + 1)); continue; } +# A story opened and pictured; once more if the simulator went down under it. +ios_story() { + local bundle="$1" story="$2" picture="$3" wait="$4" + xcrun simctl terminate "$udid" "$bundle" >/dev/null 2>&1 + SIMCTL_CHILD_ISO_STORYBOOK_STORY="$story" xcrun simctl launch "$udid" "$bundle" >/dev/null || return 2 sleep 4 - if steady ios_shot "$out/ios-$story.png" && drawn "$out/ios-$story.png"; then - echo "hosted_apple_test: ios $story" - else - echo "hosted_apple_test: ios $story: nothing drawn" >&2 - failures=$((failures + 1)) - fi -done + # The web-view app's reactor compiled by WebKit first: a picture every + # 2 s until one has drawn, for at most seconds. + local waited=0 + until [ "$wait" -eq 0 ] || { ios_shot "$picture" && drawn "$picture"; }; do + [ "$waited" -lt "$wait" ] || break + sleep 2 + waited=$((waited + 2)) + done + steady ios_shot "$picture" && drawn "$picture" +} +ios_stories() { + local kind="$1" bundle="$2" wait="$3" story status + for story in "${stories[@]}"; do + ios_story "$bundle" "$story" "$out/$kind-$story.png" "$wait" + status=$? + if [ $status -ne 0 ] && ! hosted_sim_healthy && hosted_sim_recover "$out"; then + ios_story "$bundle" "$story" "$out/$kind-$story.png" "$wait" + status=$? + fi + case $status in + 0) echo "hosted_apple_test: $kind $story" ;; + 2) echo "hosted_apple_test: $kind $story: didn't launch" >&2; failures=$((failures + 1)) ;; + *) echo "hosted_apple_test: $kind $story: nothing drawn" >&2; failures=$((failures + 1)) ;; + esac + done +} +ios_stories ios "$BUNDLE" 0 # The web-view app (examples/apple/webview_ios_application, docs/modes.md), # when given: the Storybook as wasm in a web view out of sight, drawn by the # tree renderer over the session; each story opened as the Storybook's are, @@ -86,31 +108,13 @@ if [ -n "$webview_ipa" ]; then WEBVIEW_BUNDLE=com.LoganShire.IsomerExamples.WebView xcrun simctl terminate "$udid" "$BUNDLE" >/dev/null 2>&1 mkdir -p "$work/webview" && unzip -q "$webview_ipa" -d "$work/webview" && - xcrun simctl install "$udid" "$work/webview/Payload/"*.app || + { xcrun simctl install "$udid" "$work/webview/Payload/"*.app || + { ! hosted_sim_healthy && hosted_sim_recover "$out"; }; } || { echo "hosted_apple_test: the web-view app didn't install" >&2; failures=$((failures + 1)); webview_ipa=""; } fi if [ -n "$webview_ipa" ]; then - for story in "${stories[@]}"; do - webview_count=$((webview_count + 1)) - xcrun simctl terminate "$udid" "$WEBVIEW_BUNDLE" >/dev/null 2>&1 - SIMCTL_CHILD_ISO_STORYBOOK_STORY="$story" xcrun simctl launch "$udid" "$WEBVIEW_BUNDLE" >/dev/null || - { echo "hosted_apple_test: ios-webview $story: didn't launch" >&2; failures=$((failures + 1)); continue; } - # The reactor compiled by WebKit first: a picture every 2 s until one - # has drawn, for at most HOSTED_WEBVIEW_WAIT seconds (60), then two - # alike as for the others. - waited=0 - until ios_shot "$out/ios-webview-$story.png" && drawn "$out/ios-webview-$story.png"; do - [ "$waited" -lt "${HOSTED_WEBVIEW_WAIT:-60}" ] || break - sleep 2 - waited=$((waited + 2)) - done - if steady ios_shot "$out/ios-webview-$story.png" && drawn "$out/ios-webview-$story.png"; then - echo "hosted_apple_test: ios-webview $story" - else - echo "hosted_apple_test: ios-webview $story: nothing drawn" >&2 - failures=$((failures + 1)) - fi - done + webview_count=${#stories[@]} + ios_stories ios-webview "$WEBVIEW_BUNDLE" "${HOSTED_WEBVIEW_WAIT:-60}" fi xcrun simctl shutdown "$udid" >/dev/null 2>&1 xcrun simctl delete "$udid" >/dev/null 2>&1 diff --git a/tools/ci/hosted_headful_test.sh b/tools/ci/hosted_headful_test.sh index 946c486a..6b716fa2 100755 --- a/tools/ci/hosted_headful_test.sh +++ b/tools/ci/hosted_headful_test.sh @@ -46,17 +46,40 @@ failures=0 apple_runner() { local name="$1" out="$2" shift 2 - local developer platform + local developer platform log status attempt=0 filters=() failed=() test developer="$(xcode-select -p)" platform="$developer/Platforms/MacOSX.platform/Developer" mkdir -p "$out/$name" chmod +x "$runner" - env DYLD_FRAMEWORK_PATH="$platform/Library/Frameworks:$platform/Library/PrivateFrameworks" \ - DYLD_LIBRARY_PATH="$platform/usr/lib" TEST_UNDECLARED_OUTPUTS_DIR="$out/$name" "$@" \ - "$runner" > "$out/$name/test.log" 2>&1 - local status=$? - cat "$out/$name/test.log" - [ "$status" -eq 0 ] || failures=$((failures + 1)) + log="$out/$name/test.log" + # A simulator that went down before (under the tests before these) up first. + [ -z "$udid" ] || hosted_sim_healthy || hosted_sim_recover "$out" + while :; do + # ISOMER_SIMULATOR the simulator as it is now (another after a recovery). + env DYLD_FRAMEWORK_PATH="$platform/Library/Frameworks:$platform/Library/PrivateFrameworks" \ + DYLD_LIBRARY_PATH="$platform/usr/lib" TEST_UNDECLARED_OUTPUTS_DIR="$out/$name" "$@" \ + ${udid:+ISOMER_SIMULATOR="$udid"} "$runner" ${filters[@]+"${filters[@]}"} > "$log" 2>&1 + status=$? + cat "$log" + [ "$status" -ne 0 ] || return 0 + # The simulator taken down under the tests by GitHub's image + # (tools/ci/hosted_simulator.sh): up again, and the tests that failed + # run again (Swift Testing's --filter), at most twice. A test that + # fails with the simulator up still fails. + [ -n "$udid" ] && [ "$attempt" -lt 2 ] || break + if hosted_sim_healthy && ! grep -qE "CoreSimulatorService: Connection interrupted|device is not booted" "$log"; then + break + fi + hosted_sim_recover "$out" || break + failed=($(sed -n 's/^✘ Test \([A-Za-z0-9_]*\)(.*) failed after .*/\1/p' "$log")) + [ ${#failed[@]} -gt 0 ] || break + filters=() + for test in "${failed[@]}"; do filters+=(--filter "$test"); done + attempt=$((attempt + 1)) + log="$out/$name/test.again-$attempt.log" + echo "hosted_headful_test: $name: ${#failed[@]} tests again on the simulator started again" + done + failures=$((failures + 1)) } # The tests hosted in the iPhone app as CI's container built them (their # XCTest bundle and the app hosting it, beside the .ipa in hosted/): run @@ -137,21 +160,40 @@ apple) hosted_dir="$(dirname "$ipa")/hosted" [ -d "$hosted_dir" ] || hosted_dir="" if [[ $parts == *" ios "* || $parts == *" hosted "* ]]; then - # The iPhone: a simulator of the run's own, the app installed. - runtime="$(xcrun simctl list runtimes iOS available | sed -n 's/^iOS .* - \(com\.apple\.CoreSimulator\.SimRuntime\.iOS-[0-9-]*\)$/\1/p' | tail -1)" - [ -n "$runtime" ] || { echo "hosted_headful_test: no iOS runtime" >&2; exit 1; } - udid="$(xcrun simctl create "isomer-headful-$$" "${HOSTED_IOS_DEVICE:-iPhone 17}" "$runtime")" || exit 1 - xcrun simctl boot "$udid" && xcrun simctl bootstatus "$udid" -b > /dev/null || exit 1 - xcrun simctl ui "$udid" appearance light - mkdir -p "$work/ios" && unzip -q "$ipa" -d "$work/ios" && xcrun simctl install "$udid" "$work/ios/Payload/"*.app || exit 1 + # The iPhone: a simulator of the run's own, the apps installed + # (tools/ci/hosted_simulator.sh, which starts it again should + # GitHub's image take it down). + . "$ROOT/tools/ci/hosted_simulator.sh" + probe="$(dirname "$ipa")/probe" + mkdir -p "$work/ios" && unzip -q "$ipa" -d "$work/ios" || exit 1 + prepare_ios() { + local app_ipa + xcrun simctl ui "$udid" appearance light + xcrun simctl install "$udid" "$work/ios/Payload/"*.app || return 1 + [[ $parts == *" ios "* && -f "$probe/driver_probe_headful_apple_runner" ]] || return 0 + for app_ipa in "$probe"/*.ipa; do + rm -rf "$work/probe" && mkdir -p "$work/probe" && unzip -q "$app_ipa" -d "$work/probe" && + xcrun simctl install "$udid" "$work/probe/Payload/"*.app || return 1 + done + } + hosted_sim_start "isomer-headful-$$" prepare_ios || exit 1 fi if [[ $parts == *" ios "* ]]; then if [ -n "$runner" ]; then - apple_runner storybook_headful_ios_test "$out" ISOMER_SIMULATOR="$udid" \ + apple_runner storybook_headful_ios_test "$out" \ ISOMER_BUNDLE_ID=com.LoganShire.IsomerStorybook ISOMER_HEADFUL_LAUNCHER="$ROOT/drive/launchers/ios_launcher.sh" else run storybook_headful_ios_test "$out" --test_env=ISOMER_SIMULATOR="$udid" fi + # The driver probe's tests (Apple's driver's own checks), when CI's + # container built the probe and its runner: upright, and turned to + # landscape by the app itself on this upright simulator. + if [ -f "$probe/driver_probe_headful_apple_runner" ]; then + runner="$probe/driver_probe_headful_apple_runner" apple_runner driver_probe_headful_ios_test "$out" \ + ISOMER_BUNDLE_ID=com.LoganShire.IsomerExamples.DriverProbe \ + ISOMER_LANDSCAPE_BUNDLE_ID=com.LoganShire.IsomerExamples.DriverProbeLandscape \ + ISOMER_HEADFUL_LAUNCHER="$ROOT/drive/launchers/ios_launcher.sh" + fi fi if [[ $parts == *" hosted "* ]]; then # The same tests hosted in the app: an XCTest bundle in the headful @@ -160,13 +202,18 @@ apple) # took over a minute to boot on GitHub's runner, xcodebuild hung, and # the Mac's tests after it timed out. if [ -n "$hosted_dir" ]; then + hosted_sim_healthy || hosted_sim_recover "$out" hosted_xctest "$out" else run storybook_hosted_ios_test "$out" --@isomer//drive:headful --ios_multi_cpus=sim_arm64 \ --test_env=ISOMER_SIMULATOR="$udid" fi fi - [ -z "$udid" ] || xcrun simctl shutdown "$udid" > /dev/null 2>&1 + if [ -n "$udid" ]; then + xcrun simctl shutdown "$udid" > /dev/null 2>&1 + xcrun simctl delete "$udid" > /dev/null 2>&1 + udid="" + fi if [[ $parts == *" macos "* ]]; then # The Mac: this VM's own screen. ditto -x -k "$zip" "$work/mac" || exit 1 diff --git a/tools/ci/hosted_simulator.sh b/tools/ci/hosted_simulator.sh new file mode 100644 index 00000000..fcf0f190 --- /dev/null +++ b/tools/ci/hosted_simulator.sh @@ -0,0 +1,98 @@ +# The iPhone simulator of a job on GitHub's own `xcode-27` Mac runner, +# sourced by tools/ci/hosted_apple_test.sh and hosted_headful_test.sh. +# +# GitHub's xcode-27-arm64 image 20261112.0361.1 (actions/runner-images#15334, +# #14946) has simdiskimaged mount an iOS runtime twice and then unmount it +# while a simulator runs from it: SimLaunchHost crashes, every booted +# simulator goes down, and CoreSimulatorService answers every later simctl +# with "Connection interrupted" for the rest of the job. 20261104.0337.1 did +# not. So a job here waits for the runtimes to be mounted before it makes +# its simulator, and, when the simulator goes down under it, starts +# CoreSimulator again and boots it again (a new one if its runtime is gone), +# prepared as before, so the caller can try again what failed. +# +# hosted_sim_start sets udid: made, booted, prepared +# hosted_sim_healthy the simulator answers +# hosted_sim_recover [] back up (udid may change); the +# crash reports kept in +# +# is called with the simulator booted, its udid in $udid +# (its appearance, status bar, apps installed); it returns non-zero if it +# could not. + +hosted_sim_name="" +hosted_sim_prepare="" +hosted_sim_recoveries=0 + +# Every runtime simdiskimaged is mounting, mounted (at most two minutes): +# `simctl runtime list` also starts CoreSimulator. +hosted_sim_settle() { + local waited=0 pending + command -v jq > /dev/null || { xcrun simctl runtime list > /dev/null 2>&1; return 0; } + while :; do + pending="$(xcrun simctl runtime list -j 2> /dev/null | jq -r ' + [.[] | select((.state // "Ready") != "Ready" and ((.state // "") | test("^Unusable") | not))] | length' 2> /dev/null)" + [ "$pending" = 0 ] && return 0 + [ "$waited" -lt 120 ] || { echo "hosted_simulator: runtimes still not ready after 120 s" >&2; return 0; } + sleep 2 + waited=$((waited + 2)) + done +} + +# The newest iOS runtime simctl has. +hosted_sim_runtime() { + xcrun simctl list runtimes iOS available | + sed -n 's/^iOS .* - \(com\.apple\.CoreSimulator\.SimRuntime\.iOS-[0-9-]*\)$/\1/p' | tail -1 +} + +hosted_sim_boot() { + xcrun simctl boot "$udid" > /dev/null 2>&1 + xcrun simctl bootstatus "$udid" -b > /dev/null 2>&1 && hosted_sim_healthy +} + +hosted_sim_create() { + local runtime + runtime="$(hosted_sim_runtime)" + [ -n "$runtime" ] || { echo "hosted_simulator: no iOS runtime on this Mac" >&2; return 1; } + udid="$(xcrun simctl create "$hosted_sim_name" "${HOSTED_IOS_DEVICE:-iPhone 17}" "$runtime")" || return 1 + hosted_sim_boot || return 1 + "$hosted_sim_prepare" +} + +hosted_sim_start() { + hosted_sim_name="$1" + hosted_sim_prepare="$2" + hosted_sim_settle + hosted_sim_create +} + +# CoreSimulator answering, and the simulator booted. +hosted_sim_healthy() { + [ -n "$udid" ] && xcrun simctl list devices 2> /dev/null | grep -qF "($udid) (Booted)" +} + +hosted_sim_recover() { + local out="${1:-}" + hosted_sim_recoveries=$((hosted_sim_recoveries + 1)) + [ "$hosted_sim_recoveries" -le 3 ] || { echo "hosted_simulator: down a fourth time, not recovered" >&2; return 1; } + echo "hosted_simulator: the simulator went down (actions/runner-images#15334): CoreSimulator started again" >&2 + if [ -n "$out" ]; then + mkdir -p "$out/simulator-crashes" + cp ~/Library/Logs/DiagnosticReports/*CoreSimulator* ~/Library/Logs/DiagnosticReports/*SimLaunchHost* \ + ~/Library/Logs/DiagnosticReports/*simdiskimage* "$out/simulator-crashes/" 2> /dev/null + xcrun simctl runtime list > "$out/simulator-crashes/runtimes-$hosted_sim_recoveries.txt" 2>&1 + fi + # CoreSimulatorService is this job's (the first simctl started it); launchd + # starts it again at the next simctl. + killall -9 com.apple.CoreSimulator.CoreSimulatorService 2> /dev/null + sleep 2 + hosted_sim_settle + if [ -n "$udid" ] && xcrun simctl list devices | grep -q "($udid)"; then + xcrun simctl shutdown "$udid" > /dev/null 2>&1 + if hosted_sim_boot; then + "$hosted_sim_prepare" && return 0 + fi + xcrun simctl delete "$udid" > /dev/null 2>&1 + fi + hosted_sim_create +} diff --git a/tools/testing/browser_check/main.swift b/tools/testing/browser_check/main.swift index 62de8af4..7827f973 100644 --- a/tools/testing/browser_check/main.swift +++ b/tools/testing/browser_check/main.swift @@ -19,10 +19,14 @@ // exceptions are printed either way. CHROME in the environment names Chrome // when `--chrome` doesn't. `--hold` keeps the last page open after its // picture, its console still printed, until the process is told to stop -// (SIGTERM or SIGINT): a headful app a test drives, launched this way. While -// it holds, `--requests ` takes a picture for each `.request` file -// that appears there (the file holds the picture's path; the request is -// removed once the picture is written): a test's `picture`. +// (SIGTERM or SIGINT, or a file named `stop` in `--requests`' folder): a +// headful app a test drives, launched this way. While it holds, `--requests +// ` takes a picture for each `.request` file that appears there (its +// first line the picture's path; the request is removed once the picture is +// written): a test's `picture`. A `scene:` request (a test's `scene`) +// is removed unanswered: the test's driver moves the page's scene. Each run's Chrome has a profile of its own, +// removed after: a page starts with nothing kept (its IndexedDB, its +// localStorage), as ISOMER_FRESH's default has the other launchers start. // // For comparing two renderings of a site (the web's DOM renderer against // React's, story by story): `--dom` also writes each page's DOM beside its @@ -288,13 +292,24 @@ if options.hold, let last = options.pages.last { let name = URL(fileURLWithPath: last.out).deletingPathExtension().lastPathComponent log("\(name): held open until stopped") while Held.stopping == 0 { - usleep(200_000) + usleep(50_000) + // The test done with the app (HeadlessApp's quit: a `stop` file). + if !options.requests.isEmpty, FileManager.default.fileExists(atPath: options.requests + "/stop") { break } // A test's pictures, as asked. if !options.requests.isEmpty, let names = try? FileManager.default.contentsOfDirectory(atPath: options.requests) { for request in names.sorted() where request.hasSuffix(".request") { let path = options.requests + "/" + request - guard let target = try? String(contentsOfFile: path, encoding: .utf8) else { continue } + // Its first line (the rest says what the app is, for + // launchers that turn a picture: a page has nothing to turn). + guard let target = (try? String(contentsOfFile: path, encoding: .utf8))? + .split(whereSeparator: \.isNewline).first.map(String.init) else { continue } + // A scene's phase (HeadlessApp's `scene`): not the page's to + // move; removed unanswered, the test's driver moves it. + if target.hasPrefix("scene:") { + try? FileManager.default.removeItem(atPath: path) + continue + } if let data = (chrome.send("Page.captureScreenshot", ["format": "png"], session: session)?["data"] as? String) .flatMap({ Data(base64Encoded: $0) }) { try? data.write(to: URL(fileURLWithPath: target.trimmingCharacters(in: .whitespacesAndNewlines))) diff --git a/xplat/iso/main.swift b/xplat/iso/main.swift index d7344aa4..266786f7 100644 --- a/xplat/iso/main.swift +++ b/xplat/iso/main.swift @@ -22,6 +22,13 @@ // iso longpress its context menu (its items then the // screen's buttons), or its long press // iso back | dismiss +// iso scene +// the app's scene phase, as leaving the +// app (background) and coming back +// (active) moves it: an app `iso launch` +// started, by its launcher where the +// platform's own moves it (Android's), +// else by its driver // iso wait [--timeout ] // iso drive commands as JSON lines on standard input // @@ -32,6 +39,17 @@ // iso apps the apps attached to the hub on this machine // iso hub stop ends the hub iso started // +// iso launch [--name ] [--keep-data] [KEY=VALUE…] [-- …] +// a headful app started by one of isomer's +// launchers (drive/launchers: ios_launcher.sh, +// android_launcher.sh, …, with their environment: +// ISOMER_SIMULATOR, ISOMER_BUNDLE_ID, …), dialling +// the hub on this machine under (the +// launcher's platform by default), fresh (none of +// its data from before) unless --keep-data; +// KEY=VALUE its environment, after -- its arguments +// iso launch stop ends it +// // Options for the commands: --url (an app by its URL), --app (an // application kept by name, not the selected one), --json (each answer as // protobuf's JSON). URLs: grpc://, grpcs://, ws://, unix:, exec: Wire_Comma } case "longpress": command.target = argument(0, "longpress ") + case "scene": + command.value = argument(0, "scene ") + guard ["active", "inactive", "background"].contains(command.value) else { + fail("usage: iso scene ") + } case "wait": command.text = argument(0, "wait [--timeout ]") command.timeout = options.timeout @@ -458,6 +481,13 @@ func run(_ commands: [Wire_Command], _ options: Options) async throws -> Bool { let results = await withTaskGroup(of: (String, [Wire_Answer], String?).self) { group in for target in targets { group.addTask { + // A scene an app's launcher moved: the screen it leads to. + let commands = commands.map { command in + guard command.cmd == "scene", launcherScene(target, command.value) else { return command } + var ls = Wire_Command() + ls.cmd = "ls" + return ls + } let (answers, error) = await send(commands, over: endpoint, context: endpoint.context(token: nil, metadata: ["iso-target": target])) return (target, answers, error) } @@ -511,6 +541,101 @@ func driveInteractively(_ options: Options) async throws { await call.finish() } +// MARK: - Apps started by a launcher + +/// `scene` for an app `iso launch` started, by its launcher where the +/// platform's own moves the scene (drive/launchers/requests.sh: Android's +/// sends the app home and brings it back): true if it did, false for the +/// app's driver to. +func launcherScene(_ app: String, _ phase: String) -> Bool { + let requests = Registry.directory.appendingPathComponent("run/launch-\(app).requests") + guard running(runFiles("launch-\(app)").pid) != nil, FileManager.default.fileExists(atPath: requests.path) else { return false } + let id = "scene-\(getpid())" + let request = requests.appendingPathComponent("\(id).request") + let answer = requests.appendingPathComponent("\(id).answer") + let part = request.appendingPathExtension("part") + guard (try? Data("scene:\(phase)\n".utf8).write(to: part)) != nil, + (try? FileManager.default.moveItem(at: part, to: request)) != nil else { return false } + let deadline = Date().addingTimeInterval(30) + while FileManager.default.fileExists(atPath: request.path), Date() < deadline { usleep(20_000) } + try? FileManager.default.removeItem(at: request) + let said = (try? String(contentsOf: answer, encoding: .utf8))?.trimmingCharacters(in: .whitespacesAndNewlines) + try? FileManager.default.removeItem(at: answer) + return said == "ok" +} + +/// `iso launch`: a launcher (drive/launchers) run in the background, its app +/// dialling the hub on this machine, kept by name until `iso launch stop`. +func launch(_ arguments: [String]) async throws { + if arguments.first == "stop" { + guard arguments.count >= 2 else { fail("usage: iso launch stop ") } + let files = runFiles("launch-\(arguments[1])") + let requests = Registry.directory.appendingPathComponent("run/launch-\(arguments[1]).requests") + try? Data().write(to: requests.appendingPathComponent("stop")) + if let pid = running(files.pid) { + kill(pid, SIGTERM) + print("stopped \(arguments[1])") + } else { + print("no app of iso's launched as \(arguments[1])") + } + try? FileManager.default.removeItem(at: files.pid) + return + } + let usage = "usage: iso launch [--name ] [--keep-data] [KEY=VALUE…] [-- …] | iso launch stop " + var rest = arguments[...] + guard let launcher = rest.popFirst() else { fail(usage) } + var name: String? + var fresh = true + var pairs: [String] = [] + var appArguments: [String] = [] + while let argument = rest.popFirst() { + switch argument { + case "--name": name = rest.popFirst() + case "--keep-data": fresh = false + case "--": + appArguments = Array(rest) + rest = [] + default: + guard argument.contains("=") else { fail(usage) } + pairs.append(argument) + } + } + // ios_launcher.sh: `ios`, as the app would name itself dialling the hub. + let platform = URL(fileURLWithPath: launcher).deletingPathExtension().lastPathComponent + .replacingOccurrences(of: "_launcher", with: "") + let app = name ?? platform + _ = await ensureHub() + let files = runFiles("launch-\(app)") + if let pid = running(files.pid) { fail("\(app) is running already (pid \(pid)): iso launch stop \(app)") } + let requests = Registry.directory.appendingPathComponent("run/launch-\(app).requests") + try? FileManager.default.removeItem(at: requests) + try FileManager.default.createDirectory(at: requests, withIntermediateDirectories: true, attributes: [.posixPermissions: 0o700]) + let process = Process() + process.executableURL = URL(fileURLWithPath: launcher) + process.arguments = [hubURL, app] + pairs + (appArguments.isEmpty ? [] : ["--"] + appArguments) + var environment = ProcessInfo.processInfo.environment + environment["ISOMER_FRESH"] = fresh ? "1" : "0" + environment["ISOMER_LAUNCHER_REQUESTS"] = requests.path + process.environment = environment + FileManager.default.createFile(atPath: files.log.path, contents: nil) + let log = try FileHandle(forWritingTo: files.log) + process.standardInput = FileHandle.nullDevice + process.standardOutput = log + process.standardError = log + try process.run() + try String(process.processIdentifier).write(to: files.pid, atomically: true, encoding: .utf8) + let deadline = Date().addingTimeInterval(120) + while Date() < deadline { + if (await attachedApps() ?? []).contains(where: { $0.name == app }) { + print("launched \(app)\(fresh ? "" : ", its data kept") (iso --app \(app) ls; iso launch stop \(app))") + return + } + if !process.isRunning { fail("\(launcher) ended (\(process.terminationStatus)) before \(app) attached (\(files.log.path))") } + try? await Task.sleep(nanoseconds: 250_000_000) + } + fail("\(app) didn't attach in 120 s (\(files.log.path)); iso launch stop \(app) ends its launcher") +} + // MARK: - The hub headful apps dial out to func hub(_ arguments: [String]) async throws { @@ -579,7 +704,7 @@ func hub(_ arguments: [String]) async throws { let parsed = Options(CommandLine.arguments.dropFirst()) let arguments = parsed.rest[...] guard let verb = arguments.first else { - print("iso ls|info|tap|type|set|select|swipe|longpress|back|dismiss|wait|drive [--app [,…] | --all], iso apps, iso hub [stop], iso application add|remove|select|list|stop …") + print("iso ls|info|tap|type|set|select|swipe|longpress|back|dismiss|scene|wait|drive [--app [,…] | --all], iso apps, iso hub [stop], iso launch | stop , iso application add|remove|select|list|stop …") exit(2) } @@ -623,6 +748,8 @@ do { } case "hub": try await hub(Array(arguments.dropFirst())) + case "launch": + try await launch(Array(arguments.dropFirst())) case "apps": _ = await ensureHub() for app in await attachedApps() ?? [] {