Basalt

Testing

All of it, in the order CI runs it:

All of it, in the order CI runs it:

make ts             # the type check, which CI runs first and gates the rest on
make test           # everything this machine can run
make test-quick     # unit and Node suites only

The type check

make ts builds the TypeScript packages and then checks everything that reads them: the three host packages, and the demo apps in e2e/. The apps are the half worth having. They call basalt's published types the way an app does, so a prop renamed in src/ and not in the apps fails here and nowhere else.

In CI it is a job of its own, the TypeScript, on the pinned React Native, and the three build jobs wait on it. Before that it ran inside each of them, which meant one bad declaration failed nine shards after nine builds. See docs/ci-performance.md.

It can disagree with your machine, and CI is the one that is right. build_ts.sh resolves react-native to the first of three files it finds, and which one that is changes what is declared: a checkout with generated types_generated/ pulls in React Native's hand-written globals, and a release checkout falls back to ReactNativeApi.d.ts, which declares none. To get CI's answer here, move the generated types aside:

mv "$RN_DIR/packages/react-native/types_generated" /tmp/aside && make ts
mv /tmp/aside "$RN_DIR/packages/react-native/types_generated"

Not npx tsc. There is an unrelated package on npm called tsc, and from a directory with no local TypeScript npx fetches that one, which prints a banner and exits 0. It reported a clean e2e/ while checking nothing at all.

The version can disagree too, and that is the sharper trap. The check is only as good as the checkout it runs against, and the two differ on purpose: scripts/react-native.pin is a release, because applications pin releases, and development happens against main. So a type that exists only on main passes here and fails the CI job that gates every other job.

Which happened on 2026-10-09, and took three tries to pin down because the Flow types and the TypeScript ones disagree. main declares backgroundSize in both and its C++ reads that raw prop, falling back to experimental_backgroundSize. On v0.87.1 the Flow types declare only the prefixed spelling -- and ReactNativeApi.d.ts, which is the file a release checkout is actually type-checked against, declares neither. So both spellings failed the pinned job, twenty seconds in, while both passed here.

The lesson in it is which file to read: not Libraries/StyleSheet/StyleSheetTypes.js, which is Flow and is what a search finds first, but the .d.ts that build_ts.sh resolves to -- and at the pinned tag rather than in the checkout. e2e/views.tsx now writes those three through one cast, with the reason beside it, and backlog/upstream.md carries the omission.

build_ts.sh now prints which React Native it is checking against whenever that is not the pin. It is a warning rather than an error, because working against main is the normal case -- the point is that the next red is obvious rather than mysterious.

Running one thing

The whole suite is about a quarter of an hour per host, and most changes want one part of it. Everything here narrows, and each narrows the same way: a substring of a name, and --list to see the names.

The make targets above are these with the arguments filled in, and they pass BASALT_BUILD_DIR through, so make test BUILD=build-087 tests the tree it built. Narrowing is what the scripts are for, and is below.

scripts/test_all.sh --list                  # the step names
scripts/test_all.sh --only unit             # steps whose name contains "unit"
scripts/test_all.sh --host gtk --only end   # the other host, end-to-end only

python3 scripts/integration_test.py --list                    # scenario names
python3 scripts/integration_test.py -k window                 # one scenario
python3 scripts/integration_test.py -k window -k menu         # or several
python3 scripts/integration_test.py --platform macos -k hover # and which host

./build/basalt_gtk_tests --list             # test names
./build/basalt_gtk_tests limits             # tests whose name contains "limits"
./build/basalt_gtk_tests window switch      # or several

A filter that matches nothing says so and exits non-zero, rather than reporting that everything passed. That is not hypothetical: the unit binaries used to accept an argument and ignore it, so basalt_gtk_tests hover ran all 185 tests and printed a pass, which looks exactly like a filter that worked.

It builds nothing but the demo bundle, and skips what is not built or not installed rather than failing, so on a machine with one host it runs that host's suites and says which it left out. CI's jobs and that script are two spellings of one list; a suite added to one belongs in the other.

The pieces, for when one of them is the thing being worked on. The Linux ones need a display, because everything in them is a GTK program, and CI runs them on Linux, which is where they matter: see .github/workflows/ci.yml.

./build/basalt_gtk_tests            # unit, Linux
./build/basalt_appkit_tests         # unit, on a Mac
./build/basalt_win32_tests.exe      # unit, on Windows

make bundle                         # a production bundle to test against
make e2e                            # end to end, on whichever host is built

make compare-hosts                  # every host built, same app, diffed
make compare                        # every app in e2e/, through every host
node --test scripts/test_cli.js     # run-linux, run-macos and run-windows
node scripts/check_includes.js      # a header used and not included, and a
                                    # spelling one toolchain lacks
node scripts/check_backlog.js       # the backlog's counts against its entries
node scripts/check_support.js      # the support page, from the scraped prop list
python3 scripts/scrape_props.py    # what React Native declares, from the pin

The last two need nothing built, and run in both CI jobs. check_includes.js exists because a header that reached <cstdint> through windows.h took the Linux job down for twenty-three commits and could not be seen from the machine the work was on; it reads text, so it cannot fail on one platform and pass on another.

It does two things now, for the same reason. The second is a short list of spellings that one of the three toolchains does not have, M_PI first among them: MSVC defines the POSIX math constants only behind _USE_MATH_DEFINES, which core/cmake/ReactNativeCore.cmake sets for React Native's targets and deliberately not for this project's, so a shared header using it compiled on the two hosts its author could build and took the Windows job down. A file that mentions _WIN32 is skipped, having already thought about the question.

Those spellings are all unused in the tree on purpose, which means nothing here would notice if the rules stopped matching. So the guard has its own tests, node --test scripts/test_check_includes.js, which feed text through the rules directly and run before it in both places.

check_support.js is the third, and the only one that reads React Native. website/docs/support.mdx has one row per attribute, and the attributes are not this repository's to list: scripts/scrape_props.py --from-pin reads BaseViewProps, AccessibilityProps and TextAttributes out of the pinned release and writes docs/react-native-props.json, and the statuses live beside it keyed by Struct.prop. So a prop added upstream arrives as a row with no status and fails the check until somebody says what this platform does with it, which a hand-written table cannot do: a missing row looks exactly like a row nobody filled in.

It also checks the statuses against the code -- a host claiming a prop has to read it, and a host disclaiming one has to not -- which found two wrong cells the day it was written, one of them accessibilityValue claiming Windows support that was never implemented. The scraper needs the network and CI does not run it; what CI runs is the renderer's --check against the committed inventory.

check_backlog.js is the same shape of guard pointed at prose. docs/BACKLOG.md is a table of areas and how much is open in each, every file repeats its own count in an **Open (n):** header, and both are maintained by hand in a repository where a commit routinely strikes one entry and adds another. Counting them for the first time on 2026-10-08 found accessibility.md claiming two with four open, wrong for an unknown number of commits, and the commit that found it had nearly shipped a wrong count of its own.

The rule it applies is the one these files are written to: an entry is closed when its own text is struck, and a struck note inside an entry leaves the entry open. It also has its own tests, for the reason above and because the shapes that matter -- the wrapped entry, the struck note, the prose file with no list -- are easier to feed as text than to find.

integration_test.py picks whichever host it finds and runs the same scenarios against it, because they are about React and Fabric rather than about a toolkit, --platform overrides the guess. Every scenario runs on every platform; the packager the two development-mode ones need is started through scripts/metro.js, in Node, because a shell script is not a way to start one on Windows.

Read the first line of the output before believing a pass. It names the host that actually ran, and the guess is first-found rather than this machine's own: on a Mac with both hosts built, a bare run says on linux and exercises GTK. Chasing a macOS-only failure that way produces a green run that proves nothing about macOS, which has happened. Pass --platform macos when the host is the point.

And --platform linux on a Mac is not Linux. It is the GTK host built against this machine's C library, so anything whose behaviour comes from libc rather than from GTK behaves as macOS, not as glibc. A green GTK run here is a statement about the GTK code and not about the platform under it.

That is not a hypothetical. backtrace_symbols_fd, which the crash handler uses, formats a frame differently in each of the three, and only one of the three can be seen from here:

macOS      0   basalt_appkit   0x0000000104e8c040 _ZN6basalt... + 156
glibc      build/basalt_gtk(+0x875a5c)[0x55cef4472a5c]
Windows     0  00007FF621EE0094

A leading index on two and none on glibc; 0x on two and absent on the third. A scenario matching the shape of the one visible here passed locally on both hosts and failed on CI twice, once per platform, against a handler that was printing a perfectly good stack each time. It counts the lines between the marker and the blank line now, which is the one thing all three agree on.

On a headless machine, run either under a virtual display:

xvfb-run -a ./build/basalt_gtk_tests
xvfb-run -a scripts/integration_test.py

basalt_gtk_tests exits 77 rather than failing when it cannot reach a display, which is the convention for "skipped".

Screenshotting a GTK window on macOS

screencapture -l <window-id> wants a CGWindow id, which nothing in the shell exposes. A five-line Swift program over CGWindowListCopyWindowInfo, filtered by kCGWindowOwnerPID, produces one. Compile it once with swiftc: swift <file> interprets slowly enough to miss the window's first frame.

basalt_appkit_tests needs neither a display nor a window: nothing in it is presented. It is built only on a Mac, and CMake omits the target everywhere else.

basalt_win32_tests needs neither either, and unlike the other two it makes real pictures while not needing them: Direct2D renders into a WIC bitmap with no window, no device and no display connection. That is why the rendering assertions this project has wanted since GTK live there and nowhere else. Built only on Windows; CMake omits the target elsewhere.

FileWhat it pins down
native/tests/test_win32_view.cppThe view layer as a tree. Tags and frames, insert indices, that a Remove detaches without destroying, that an Insert reparents, that destroying a view leaves nothing pointing at it, that zIndex does not touch the child list, and that describeTree is byte-for-byte what the other two hosts print.
native/tests/test_win32_paint.cppWhat reached the pixels. Placement, opacity as a subtree layer, clipping under overflow, a quarter turn about the centre and in the right direction, zIndex paint order, the scroll offset, and the corner radius. Each is something a tree dump cannot show.
native/tests/test_win32_hittest.cppPicking. Depth, sibling order, zIndex, misses, scroll offsets, hidden views, and that a press follows a transform, which this platform has to invert itself, having no toolkit picking to inherit.
native/tests/test_win32_text.cppDirectWrite measurement. Wrapping, numberOfLines, that a bigger font measures bigger, that alignment does not change the measured size, and two regression tests carried over from the other hosts: that font sizes are absolute rather than points, and that the default ellipsize mode does not collapse a wrapping paragraph. Plus one they cannot make: that the glyphs actually move when the alignment does.
native/tests/test_win32_image.cppThe four resize modes, against a real bitmap and real pixels rather than against the destination rect: cover that scales correctly and forgets to clip looks right in a rect and wrong on a screen. Also the WIC decode, round-tripped through the snapshot encoder.
native/tests/test_win32_accessibility.cppThe UI Automation mapping, asked the way a screen reader asks it: build the provider and read property ids. Roles, the group fallback, that a plain <View> stays out of the tree, labels and hints, the tri-state states, and that a role can change after mount, which GTK cannot do, because a GtkAccessible role is construct-only.
native/tests/test_win32_mounting.cppThe mutation walk, through the real Win32MountingManager. Deliberately the same nine questions asked of GTK and AppKit, in the same order with the same tags, so a divergence in the shared walk fails here rather than in an app. Built only when the build was pointed at a React Native.
native/tests/test_win32_imageloader.cppThe image loader. Decoding a file URI, the cache answering without going back to disk, the three ways a load can fail, and that a decoded image survives the loader that produced it. Built only when the build was pointed at a React Native, because the fetch is core/ImageBytes.cpp.

Watching an event go past

native/tests/EventRecorder.h is how a unit test sees that an event fired. Neither suite could: the managers ask eventEmitterForTag for an emitter and the hand-built shadow views carry none, so what the text-input tests pinned was, in their own words, "that the round trip did not throw".

A stub emitter cannot fix that, TextInputEventEmitter contains the word virtual zero times, so the recorder builds the real thing and listens underneath it. EventDispatcher takes a std::function<bool(const RawEvent &)> and consults it before the logger and before the queue; returning true stops the default dispatch, so no RuntimeScheduler ever has to run.

What it sees is the event's type and its position in the order. Payload values need a jsi::Runtime to read, so what a value ends up as stays the end-to-end suite's question.

build/basalt_gtk_tests: the unit suite

Everything reachable without a JavaScript runtime. No React, no Metro, no Hermes: mutations are hand-built, the way mount_harness_gtk builds them.

FileWhat it pins down
native/tests/test_mounting.cppThe mutation walk. Create/Insert/Remove/Delete/Update ordering, insert indices, nesting, that Remove detaches without destroying, that Delete unregisters, and that a stray Insert for a deleted tag is ignored rather than fatal.
native/tests/test_view.cppThe widget layer. Frames, child order, that a scroll offset moves children, that measure reports zero so GTK never competes with Yoga, and that dispose unparents children.
native/tests/test_text.cppPango measurement. Wrapping, numberOfLines, that a bigger font measures bigger, and two regression tests: that font sizes are absolute rather than points, and that the default ellipsize mode does not collapse a wrapping paragraph to one line.
native/tests/test_hittest.cppHit testing. Depth, sibling order, misses, that it follows a scroll offset, and that hitSlop grows a target without moving it.
native/tests/test_gtk_paint.cppWhat reached the pixels, through tests/GtkPixels.h: placement, opacity as a subtree layer, clipping under overflow, zIndex paint order, the scroll offset, corner radii, and a border inside the box and over the children. The same eleven questions test_win32_paint.cpp asks of Direct2D, in the same order. Each is something a tree dump cannot show.
native/tests/test_image.cppThe image loader. Decoding, the cache answering synchronously, and the three ways a load can fail.
native/tests/test_textinput.cppThe controlled-value loop. That applying a prop is not reported back as typing, that the caret survives a prop arriving mid-word, that a command carrying a stale eventCount is dropped, and that the GtkText peer is allocated inside the content inset. Props are built through React Native's own RawProps parser, because the fields that matter are const and only reachable that way. Two tests watch the events rather than the widget, through native/tests/EventRecorder.h.

build/basalt_appkit_tests: the macOS suite

Seventeen files, no window and no toolkit initialisation. The binary prints its own total, which was 824 on 2026-10-14; the lead here said "two files, sixteen tests" for a long time after it was neither, which is what a count kept by hand in prose does.

Four of them carry an argument about how they test, and those are described after this table. These are the rest:

FileWhat it pins down
native/tests/test_appkit_text.mmThe Core Text layer, against RnTextLayout directly: whether a paragraph measures and truncates the way React Native means it to.
native/tests/test_appkit_textinput.mmThe controlled-value loop, the same questions test_textinput.cpp asks of GTK.
native/tests/test_appkit_borders.mmWhat a border is drawn as, against a real bitmap: four filled edges or one dashed stroke. A view that remembered dashed and painted solid would pass any test that asked it what it holds.
native/tests/test_appkit_image.mmHow an <Image> fills its frame. The four resize modes are what two platforms are most likely to disagree about while both looking plausible.
native/tests/test_appkit_scrollview.mmWhat a scroll view does that needs no display: props arriving as mutations, and the offset that results.
native/tests/test_appkit_scrollbar.mmThe overlay scrollbar, which on this host has to be drawn over a content view that covers it.
native/tests/test_appkit_selection.mmSelecting a paragraph with a pointer: the same three questions the GTK suite asks, so that the two hosts agree.
native/tests/test_appkit_key_events.mmAn NSEvent as a key combination, which is the half of key handling that is this host's alone and where the silent failures live.
native/tests/test_appkit_droptarget.mmWhich view a drag is offered to. Added because a change that stopped reading the nativeID marker altogether left the whole suite green.
native/tests/test_appkit_titlebar.mmThe title bar's metrics, which are the whole contract src/TitleBar.tsx has with the host.
native/tests/test_appkit_titlebar_drag.mmWhich parts of an app-drawn header drag the window: the same four questions test_titlebar_drag.cpp asks of GTK, in the same order, because the rule in core/TitleBarRegions.h is shared and the answers have to match.
native/tests/test_appkit_fonts.mmA font loaded while the app is running, which is the seam expo-font arrives through, and whether the name the app chose keeps working.
native/tests/test_appkit_services.mmThe clipboard and URL halves of the platform-services seam.
native/tests/test_appkit_skia.mmThat Skia renders rather than merely links; see the Skia section below.

native/tests/test_appkit_view.mm is against RnAppKitView alone, with no Fabric and no React Native in it. It pins the tag and frame, the child order including insertion into the middle, that removing a view that is not yours is a no-op, that background colour, opacity, clipping and corner radius reach the CALayer, that the background is in sRGB rather than the display's space, and that describeTree emits the same text the GTK side does.

native/tests/test_appkit_accessibility.mm asserts the AppKit role, not the React Native name: the name is what describeTree reports and what the cross-platform diff compares, and whether the mapping onto NSAccessibility actually happened is a platform question. The GTK suite's test_accessibility.cpp asserts the same thing about GtkAccessibleRole.

native/tests/test_appkit_input.mm is hit testing and the touch state machine. It covers what test_win32_hittest.cpp does, including that a press follows a transform: a translation, a quarter turn, and a scale: 0 that has no inverse and is skipped. Hit testing is a pure function of the view tree, so it needs no mouse, no window server, and no permission to synthesise an event, which matters, because a real click on macOS means CGEvent and accessibility permission an automated run does not have. Unlike the GTK equivalent none of it needs a window: gtk_widget_pick skips unmapped widgets, so those tests show a window and pump the frame clock, and RnAppKitHitTest walks frames and answers straight away.

native/tests/test_appkit_mounting.mm is against AppKitMountingManager, and is deliberately the same questions native/tests/test_mounting.cpp asks of GTK, in the same order, with the same tags and frames. Since both managers now share their mutation walk, most of what these test is whether that sharing still holds. Built only when the build was pointed at a React Native; without one the view tests still run.

The one worth naming is mac_view_is_flipped_so_frames_are_top_left. AppKit's origin is bottom-left and every frame Fabric produces is top-left, so a child at y=32 in a 420-tall parent belongs 32 from the top; unflipped it would sit at 228 and the layout would look plausible and be wrong.

native/appkit/mount_harness_appkit.mm is the mounting equivalent: the same two transactions gtk/mount_harness_gtk.cpp runs, driven through the real mounting manager with no JavaScript anywhere. BASALT_SNAPSHOT_DIR=/tmp/out ./build/mount_harness_appkit writes mount-1.png and mount-2.png and prints the tree after each; with no directory set it opens a window and applies the second transaction two seconds in, the way the GTK harness does.

scripts/compare_all.sh runs every app in e2e/ through every host that is built and prints where the platforms stand. Windows against Linux in WSL:

hosts: windows linux (linux in Ubuntu-24.04)

  views        identical, frames included
  press        identical, frames included
  scroll       identical, frames included
  image        identical, frames included
  text         identical, frames ignored
  a11y         identical, frames ignored
  input        identical, frames ignored
  appearance   identical, frames included
  blob         identical, frames included
  modules      identical, frames included
  probe        identical, frames included
  index        identical, frames ignored

the 2 hosts agree on all 12 apps

An app that only passes with frames ignored is one whose layout depends on text measurement. Everything else (tree shape, strings, colours, roles, flags) is still compared there.

scripts/compare_hosts.sh is the single-app version, and the one that cannot run in CI. It runs e2e/views.tsx (a real React app made only of <View>) through every host that is built, and checks two things that fail differently: that the view trees match, and that each host's log reports the Platform.OS its bundle was built for. A bundle built for the wrong platform can render perfectly and be wrong about everything Platform.OS guards.

It takes an app name rather than a bundle path per platform, and finds build/<app>.<platform>.jsbundle.js for each host it is going to run. Three paths on a command line was one too many.

It needs at least two hosts, so it runs on a developer's machine rather than on CI, where each host only exists on its own side. On a Mac with GTK installed, build the bundles first:

scripts/bundle.sh --platform linux --entry views --out views.linux.jsbundle
scripts/bundle.sh --platform macos --entry views --out views.macos.jsbundle
scripts/compare_hosts.sh

On Windows the second host is the GTK one, running inside WSL. Ubuntu 24.04, because the GTK host needs GTK 4.10 and 22.04 ships 4.6, installed beside any distro already there:

wsl --install -d Ubuntu-24.04 --no-launch
wsl -d Ubuntu-24.04 -u root -e bash scripts/wsl_setup.sh
BASALT_COMPARE_WSL=Ubuntu-24.04 scripts/compare_all.sh

wsl_setup.sh installs what the Linux CI job installs, clones this checkout onto the distro's own filesystem; building across /mnt/c is too slow to be worth it, and builds the host there the way CI does. So it compares the committed state: rerun it after committing. compare_all.sh builds any missing Linux bundles inside the distro, and runs the host under Xvfb rather than WSLg, because Xvfb is what CI uses.

It also registers something to open https with. Linking.canOpenURL asks GIO whether anything handles the scheme, a bare distro has nothing, and e2e/modules.tsx then fails a check on Linux that a GitHub runner, which has a browser, passes, which is exactly what the first comparison through WSL reported as a difference.

BASALT_COMPARE_TAP="x,y;x,y" forwards taps to every host; they read the same variable, so the input path is compared too, not just the initial render. e2e/press.tsx is the app for that, and needs a longer quit than the default since the first tap is at 1500ms:

BASALT_COMPARE_TAP="400,100;400,100;400,100" BASALT_COMPARE_QUIT_AFTER_MS=5000 \
  scripts/compare_hosts.sh press BasaltPress

e2e/scroll.tsx scrolls itself through scrollTo on a ref, which is how both hosts can be driven identically from one file: a wheel has to be injected per platform and a command does not, and is the only test of the command path:

scripts/bundle.sh --platform linux --entry scroll --out scroll.linux.jsbundle
scripts/bundle.sh --platform macos --entry scroll --out scroll.macos.jsbundle
BASALT_COMPARE_QUIT_AFTER_MS=3000 scripts/compare_hosts.sh scroll BasaltScroll

For e2e/text.tsx and e2e/image.tsx, frames have to be ignored:

scripts/bundle.sh --platform linux --entry text --out text.linux.jsbundle
scripts/bundle.sh --platform macos --entry text --out text.macos.jsbundle
BASALT_COMPARE_IGNORE_FRAMES=1 scripts/compare_hosts.sh text BasaltText

Pango over the system sans and Core Text over San Francisco are different shapers over different fonts, so the same paragraph is a few points taller on one than the other and every frame below it shifts. Demanding equality there would mean the check could never be turned on for text at all. What is still compared is everything that must match: the tree shape, the strings, the colours, the clip and opacity flags.

As of phase 23 that comparison differs by exactly one thing, GTK emits role=label on a paragraph and macOS emits nothing, because macOS has no accessibility yet.

native/appkit/demo_layout_appkit.mm is the visual half of the view layer. It carries the same boxes at the same coordinates as the GTK demo_layout_gtk, so the two can be compared directly, and BASALT_SNAPSHOT=out.png ./build/demo_layout_appkit renders them offscreen to a PNG. BASALT_DUMP_TREE=1 prints the tree instead: useful, but it would print the same thing whether or not the flip works, which is what the snapshot is for.

The harness

native/tests/TestHarness.h, about sixty lines, no dependencies, shared by all three. TestHarness.cpp is the runner and knows about no toolkit; each suite brings its own main: tests/main_gtk.cpp initialises GTK, and the macOS one sits at the bottom of test_appkit_view.mm. Its assertions are EXPECT, EXPECT_EQ and EXPECT_NEAR rather than CHECK* because glog (which React Native pulls in almost everywhere) already defines CHECK and CHECK_EQ. Those win the preprocessor silently, and a failing assertion then aborts the process instead of being reported, taking every later test with it. That happened; hence the naming.

Hit tests put their widgets in a real window. gtk_widget_pick skips widgets that are not mapped, and a widget is only mapped inside a shown window. Allocation then happens in the frame clock's layout phase, so the tests build the tree first, show the window, and spin the main loop until a frame has actually gone through. Draining the context without waiting is not enough: with no frame due it returns immediately and the children are still unallocated.

scripts/integration_test.py: the end-to-end suite

Runs the real host against the real Metro bundle, injects taps, and asserts on the widget tree the host writes on the way out.

This is what BASALT_DUMP_TREE exists for. The path this project exists to provide (JavaScript, React, Fabric, the mounting manager, GTK) can only be exercised by running the whole thing, and before the dump the only way to check the result was to look at a screenshot.

What the scenarios are is not listed here, and deliberately: SCENARIOS at the bottom of scripts/integration_test.py is the list, and every one of them has a docstring saying what it asserts and why that is the thing worth asserting. A copy in this file was five entries long while the suite had twenty, which is what a hand-maintained list of something the code already states turns into.

What is worth writing down is what is not in the code.

BASALT_SNAPSHOT=<path> writes the window as a PNG, on all three hosts, on the way out and beside the tree dump. Windows had it from the start, since Direct2D renders into a WIC bitmap with no window and no display; GTK and AppKit grew it on 2026-10-10, each through the renderer its own suite already uses -- gsk_renderer_render_texture with a NULL surface, and RnAppKitWriteSnapshot -- so none of the three needs a display server or a screenshot permission.

BASALT_SNAPSHOT_SCALE=2 doubles the Windows render, and is for pictures rather than for tests. Win32Snapshot pins the render target to 96 dpi so a snapshot is the size the frames say whatever the machine is set to, which is what every pixel assertion depends on; the scale multiplies both the bitmap and the dpi, so the drawing is identical and the output is twice as many pixels. Unset, or outside [1, 4], it is 1 and nothing changes. CI uses it for the window in the README, because macOS and Linux are photographed at 2x and a 96 dpi Windows shot beside them is visibly the soft one.

BASALT_WINDOW_CAPTURE=<path> photographs the live window on Windows, through PrintWindow with PW_RENDERFULLCONTENT, which asks DWM for what it composited. That is the picture with the things the offscreen pass cannot see in it: a <TextInput>, whose peer is a child window; a <Canvas>, which holds the last frame delivered to the real window; and the caption buttons, which DWM draws over the app's own header.

Use the render for a test and the photograph for a picture somebody will look at. The render needs no window, is the size the frames say and scales; the photograph needs a real window and is the size that window actually is, so BASALT_SNAPSHOT_SCALE does not apply to it. Both can be asked for in one run, which is what CI does.

It exists because a tree dump says a view has a colour, a frame and a transform, and cannot say that any of it reached the screen. The two text bugs found on 2026-10-10 were both of that kind, and both were found by looking at a picture of React Native's own LogBox rather than by any assertion.

What a picture cannot show, which is worth knowing before one is read as a bug. On macOS the snapshot is [root.layer renderInContext:], and Core Animation composites nothing in that path: a boxShadow, a mixBlendMode and an <ActivityIndicator color> -- which is a CIFalseColor filter on the indicator's layer -- are all absent from the file while being perfectly visible on screen. GTK and Windows render their own node trees and have no such gap. The tree dump is what those three are asserted through instead.

The suite can read the picture back: Snapshot in scripts/integration_test.py decodes a PNG with the standard library alone -- no Pillow and no numpy on three CI images -- and answers at(x, y). a snapshot has each view's colour where its frame says is what that is for, and the pairing is the point: the dump says where React put a view, the picture says what is actually painted there, and either half alone passes while the other is wrong.

They read coordinates from each app's own layout. Most tap fixed points: e2e/index.tsx for the demo, and its own file for each of the apps written for a scenario. Change a demo's spacing and the coordinates need changing too. The alternative, searching the dumped tree for a button by its label and tapping its centre, would be more robust and is worth doing; the reason it has not been is that a wrong coordinate fails loudly and immediately.

Several apps set fixed heights on their labels for this reason. Text measurement is the one thing Pango, Core Text and DirectWrite will never agree on, so an app whose rows are positioned by measured text has different coordinates on each host. A height on a label is what keeps one set of taps working on all three.

Fast Refresh is the only scenario that runs in dev mode, which is why it exists: "development still works" was being taken on trust, and a wrong claim about it reached the README. It starts its own Metro on port 8099, edits e2e/index.tsx while the app is on screen, and restores the file after the host has exited, not before, because putting the original back while the app still has a Metro connection triggers a second refresh that undoes the edit before the tree is dumped, which looks exactly like Fast Refresh being broken. A kill -9 of the test would leave the demo edited; nothing softer will.

The edit does not work in CI. Metro on a GitHub runner never notices one: the file changes on disk with a fresh mtime, a newly requested bundle still carries the old text, and Metro's log is empty of complaint. fs.watch sees the same edit on the same runner, the inotify limits are already generous, both sides run the same Node, and the upstream Watchman build reports "watcher": "inotify" over exactly the root Metro was given and changes nothing. It passes on developer machines, on macOS and on Linux.

BASALT_SKIP_FAST_REFRESH therefore drops the edit, not the scenario. It used to drop the whole thing, which meant no machine other than somebody's laptop checked dev mode at all, and what CI cannot do is one specific half of it. Everything before the edit is the host's own code: dev mode, the dev server helper, the websocket, the DevSettings TurboModule, and a bundle fetched from Metro rather than the release one sitting on disk. Two bugs have already lived in that half: the host read its Metro entry from an argument nothing passed, and every platform reported a dev script URL naming linux, and both would have been caught with no edit made. When the variable is set the scenario passes with a note saying what it did not check, which is the point: a scenario that quietly checks less than its name says is worse than one that skips outright.

Input: real events, and where they are not

The end-to-end suite delivers taps one of two ways, and says which at the top of its output.

real: xdotool moves the pointer and clicks, so the event goes through the X server and GDK exactly as a person's would. This is the only mode that exercises event delivery itself. Chosen automatically when DISPLAY is set and xdotool is installed.

injected, BASALT_TEST_TAP enters at the touch dispatcher, skipping the window system. The fallback where a real event cannot be synthesised, which is both of the other desktops: macOS needs accessibility permission an automated run does not have, and Windows needs SendInput, which moves the real cursor and so cannot run beside anything else on the machine.

Force either with --input real or --input injected.

Typing splits the same way. In real mode xdotool type sends key events through the X server, so GDK and the input method see them. In injected mode BASALT_TEST_TYPE reaches whatever field has focus: through GtkEditable on Linux, the field editor on macOS, and real WM_CHAR messages on Windows, which is the closest of the three; what it skips there is the keyboard driver and nothing above it.

Windows is the exception to the exception in one place. A real click on a <TextInput> never reaches the touch dispatcher at all, because the field's peer is a child window and USER32 routes the click to it; so BASALT_TEST_TAP focuses the field under the point as well as dispatching the touch. That step exists only for the injected path.

Hover has no real mode on any of the three. BASALT_TEST_HOVER takes the same "x,y;x,y" as BASALT_TEST_TAP and moves the pointer without pressing it; a negative point is the pointer leaving the surface. There is no real-event equivalent because a real hover means moving the machine's actual cursor onto the window and leaving it there, which takes the pointer away from whoever is using the machine, xdotool mousemove included.

Keyboard focus is injected everywhere too, for the same reason. BASALT_TEST_FOCUS takes actions separated by ;: tab, shift-tab, activate and escape, and enters at each host's focus manager. A real Tab needs a window the display server considers focused, which an automated run does not reliably have on any of the three; what the instrument skips is the delivery of the keystroke and nothing above it.

escape and devmenu are the odd ones in that list and are there on purpose: neither is a focus action. escape closes the topmost <Modal>, and devmenu opens React Native's developer menu, the thing Ctrl+D (Cmd+D on macOS) opens. This is the instrument for "a key was pressed and nothing on the window has to be focused for it to arrive", which is what both of those are.

BASALT_TEST_FONT_SCALE is the odd one in a different direction: it is a setting rather than an event, and it stands in for something two of the three desktops do not have. GTK reads a real text scale from GtkSettings:gtk-xft-dpi, which is what GNOME's "Large Text" moves; macOS publishes no scalar at all, and Win32 would need WinRT for the one Windows publishes. So allowFontScaling and maxFontSizeMultiplier could otherwise only be tested where the machine running the suite happened to be set to large text. The variable goes in where the platform's own scale goes, so nothing above it is skipped, and because nothing is scheduled for it, it is deliberately not in core/TestSettle.h's list; scripts/test_harness.py records that with its reason.

A popup menu is the second thing an automated run cannot get past, and for a sharper reason than a dialog: on macOS popUpMenuPositioningItem runs the menu's own tracking loop on the main thread, so a menu nobody dismisses stops the process where it stands. BASALT_TEST_MENU answers it: an entry index, or dismiss: exactly as BASALT_TEST_DIALOG answers an alert, and with the same limit: only the presentation is skipped, and whatever the entry does still happens.

A file dialog is the third, and the one with the most behind it: what an app does with a path cannot be reached without a path. BASALT_TEST_FILE_DIALOG answers one: cancel, or a list of paths separated by : (; on Windows, where a path starts C:\ and a colon would cut every one of them in two).

A tap may name a window: BASALT_TEST_TAP="134,110;134,110@3" taps in the app's own window and then in the window whose surface is 3. Without it a second window could not be reached at all: each has its own touch dispatcher, and the app's would happily hit-test a tree that is not on screen and report a press on whatever was at those coordinates. A point with no @ means the app's own window, so every spec written before windows existed still means what it did.

BASALT_TEST_SECONDARY_TAP takes the same spec as BASALT_TEST_TAP and clicks the other button. Its own variable rather than a suffix on a point, because the two assert opposite things, one presses what it lands on and the other must not, and a scenario that mixed them in one string would be harder to read than to write.

BASALT_TEST_CLOSE_WINDOW takes a surface id and closes that window the way a person would, its own close button rather than the app asking. The two are different paths through the host, and only that one can leave the host holding a record whose window is gone while the app still believes it is open.

It is also the only thing an app's interception ever refuses, which is what makes it the instrument for onCloseRequest: an app closing its own window is not asking anybody. 1 names the app's own window, and a host whose main window refuses to close still has to shut down when BASALT_QUIT_AFTER_MS says so: on Windows that meant the quit timer stopping going through WM_CLOSE, because otherwise the app would have refused the harness too and the failure would have been a hang rather than a test.

A locked screen is not a broken host

Leave the macOS suite running long enough for the Mac to lock itself and two scenarios fail, neither of them with a message about screens:

click a TextInput with a real mouse and see it focus
    could not find the host window after 20 seconds
scrollTo({animated: true}) moves rather than jumps
    no scroll offsets were reported at all

A locked session realises no windows. The host still runs, still renders and still dumps a tree, but its accessibility window list is empty, so the point to click cannot be computed; and CADisplayLink does not fire, because there is nothing on a screen to be in step with, so an animated scrollTo arrives with no offsets in between. The second one is the dangerous shape: no offsets is exactly what a host that dropped the animated flag would report.

Both scenarios now ask, and skip with the reason. screen_is_locked() reads CGSSessionScreenIsLocked out of ioreg -n Root -d1, and answers false anywhere it cannot be sure, a skip that fires wrongly being worse than a failure. CI is never locked, so nothing is lost there.

What this cost before it was written down: two scenarios chased as regressions, including one that was blamed on a commit that had only touched text.

The suite is a background app, and macOS treats those differently

A host the harness launches does not get focus on a developer's Mac: the terminal running the suite keeps it. CI's macOS runner has nothing else on screen, so there the host does become active, and that difference hid two real bugs for as long as this suite has existed.

NSApp.keyWindow and NSApp.mainWindow are both nil for an app that is not active. Two things asked only those two:

  • windowBounds() answered 0x0, and because the bounds cache is primed once at startup and rewritten only on a move or a resize, an app that was launched behind another window believed it was 0x0 for its whole life. getBounds() returned zeros and setSize silently did nothing.
  • performMenuRole sent through [NSApp sendAction:to:nil], which walks the responder chain from the key window's first responder, so a paste role reached nothing.

Both now go through appWindow() (appkit/AppKitAppWindow.h), which is GTK's rule: prefer the active window, fall back to the first visible one that is not a panel. Both are covered by unit tests that work because the suite is a background app, which is the useful half of this: basalt_appkit_tests has no key window either, so a test that passes there has exercised the case a developer's machine is always in.

The lesson for the next one: a scenario that passes on CI's Mac and fails on yours is not automatically flaky. It may be the only machine where the app is not frontmost.

A real click lands where the window server says, not where the test aimed

click a TextInput with a real mouse and see it focus posts a CGEvent at a screen point, computed from the host window's position. The window server then delivers that click to whichever window is topmost at that point, which is not necessarily the window the point came from. An editor or a Finder window over that corner of the screen takes the click instead, the field under test stays unfocused, and the scenario reports the feature broken.

That is measured, not reasoned. With another application's window placed over the field the scenario failed 3 of 3 runs; without it, 0 of 5. It is why the scenario failed once on a developer Mac in a run whose only change was a text shadow.

So the scenario raises the host before it measures it: frontmost on the process and AXRaise on the window, then half a second for the raise to reach the window server. With the raise in place the same covering window produces 3 of 3 passes. The raise is not what focuses the field, which is worth stating because it would be a comfortable way to make the scenario pass for the wrong reason: raise the window and post no click, and the scenario still fails.

The host is also addressed by pid rather than by name. every process whose name contains "basalt" matches basalt_appkit_tests and any host left over from an earlier scenario, and item 1 of that list is whichever the window server answered with first. The Linux path had the same shape, xdotool search --name basalt-core taking the first match, and now prefers the window whose _NET_WM_PID is the host's. Neither bug was the cause of the failure above: every host opens a 900x700 window at the same default position, so the measured rectangle is identical and the click lands in the same place either way, which a run with a leftover host confirmed by passing. They are fixed because the next scenario that moves a window would have been debugging the wrong thing.

Skia, where the tests do not run in CI

Two suites, for two renderers, and neither runs in CI because the package is borrowed from the app rather than vendored and CI has no app. That is a real gap and is the price of not vendoring a 300MB dependency; the rest of the suite is unaffected either way, and with nothing pointed at, none of it is compiled.

Which suite a build gets is decided by what it was pointed at, and the two are mutually exclusive by construction:

# 2.x, Ganesh over Metal: tests/test_appkit_skia.mm, six tests.
cmake -B build-skia2 -DBASALT_SKIA=<app>/node_modules/@shopify/react-native-skia

# 3.x, Graphite over Dawn: basalt-skia's three, nine tests in all.
cmake -B build-skia3 -G Ninja \
  -DBASALT_SKIA_DIR=<app>/node_modules/react-native-skia \
  -DBASALT_SKIA_LIBS=<app>/node_modules/react-native-skia-graphite-apple-macos \
  -DBASALT_SKIA_DAWN=<app>/node_modules/react-native-webgpu-dawn

The three variables of the second are defaults, not requirements: basalt-skia looks for all three in the app's node_modules on its own, and they are here because the only app on this machine with a 3.x install is kino, which is not where this repository's build runs. BASALT_SKIA is the 2.x path and cmake/Skia.cmake refuses a 3.x install by version, so pointing both at the same install is not a thing that can happen; scripts/test_skia_cmake.js is that refusal's own test and does run in CI, since it needs no Skia.

Both were run against this repository's AppKit suite on 2026-10-11, separately: 786 tests with 2.14.0 and 789 with 3.4.0, all passing in both. The counts differ because the suites are different files, not because one skipped anything.

The canvas, and what a test of one can see. test_skia_canvas.mm drives the AppKit peer with a real RnAppKitView, a real CAMetalLayer and a real Dawn swapchain, which the 2.x canvas could never be tested with. Two of its assertions were vacuous when first written and the sabotage pass is what said so, which is worth recording because both failures are invisible from reading the test:

  • Points and pixels are the same number here. RNSkWindowSurface::attach takes pixels where 2.x took points, and the scale is 1 in a test -- it comes from currentDeviceMetrics(), which answers from what a host reported, and there is no host. So passing frame.size straight through passed every assertion. The multiplication is now a function, skiaCanvasPixelSize, called with the density the test chooses; the one line that feeds the real density into it is still not covered and cannot be until something can set the scale.
  • CAMetalLayer ignores a drawableSize of zero and keeps the last one, so attaching a zero-sized layout -- the case the peer guards, because Yoga hands out zero sizes before the first real layout -- looked identical from the layer. The surface's own size is what says, and skiaCanvasAttachedSize exists for that.

And a frame, which is the claim none of the rest makes. skia_canvas_draws_a_picture_and_reads_it_back goes the way an app goes: SkiaApi builds a picture in JavaScript, SkiaViewApi.setJsiProperty pushes it to the view by id, and SkiaViewApi.makeImageSnapshot reads back what the view rendered. The same three calls e2e/skia.tsx makes, with no app and no window, in the Hermes runtime the module test already needed.

Green over the left half of a 32x24 canvas and red over the right, sampled at (8,12) and (24,12): two rectangles rather than one fill, so that "drew something" and "drew it where it was asked to" are separate claims. Verified falsifiable by swapping the two samples, which prints the colours the other way round, and by removing the peer's registerSkiaView, which turns it into No Skia View currently available. -- the exception an app would get.

What the 3.x suite proves that is new. skia_module_installs_skias_javascript_api builds a Hermes runtime, calls install through the module's own property the way JavaScript does, and reads SkiaApi back off the global object -- which is the only thing that can say Skia's whole JavaScript API arrived, because arriving is all install does. And skia_context_makes_a_surface_that_draws is now a GPU round trip: the canvas records, the recording is submitted, the texture is shared as an image and read back. It found out it had become a GPU surface by peekPixels answering false, which is worth knowing as a symptom -- a Graphite surface has no CPU pixels to peek, and the test that used to peek them passed for a year against raster.

Dawn gets a Metal device on a machine with no window and no display, which is why that round trip can run at all, and would run in CI if CI had an app.

What they assert is pixels, not objects. RNSkiaModule: Skia installed says the bindings were constructed, and that is a long way from a pixel: a Metal device that fails to create, a GrDirectContext that comes back null, a surface wrapping a texture nothing can read, each installs perfectly well and then draws nothing. So a known shape goes into an offscreen surface and the pixels come back out, which is also the path an app rendering frames for export takes.

Verified falsifiable by drawing the rectangle somewhere else, which failed both halves at once:

expected inside.describe()  == "rgba(255,0,0,255)" but got rgba(0,0,0,0)
expected outside.describe() == "rgba(0,0,0,0)"     but got rgba(255,0,0,255)

"Drew something" and "drew it where it was asked to" are separate claims, and that is the pair that tells them apart. Corners rather than edges, for the reason test_appkit_image.mm gives: antialiasing puts a boundary half a pixel either way and pinning it fails on a future Skia for no useful reason.

A sabotage that does not build is not a passing test

Breaking the thing a test covers and watching it fail is the only way to know the test covers it. The hazard is that the two ways a sabotage can end look identical in a grep: the test failed, or the sabotage never ran. Three times on 2026-10-12:

what was brokenhow the sabotage died
the Win32 registry's includes-Werror=unused-parameter on a name the gutted branch stopped using
the Skia TurboModule's factory name checkthe same, one argument over
localModulesMissingDesktopTypeScript lost missing's inferred type once nothing was pushed to it, so dist/ kept the good build and the suite passed on it

The third is the nastiest, because the suite genuinely passed: scripts/test_cli.js requires the built package, a failed build_ts.sh leaves the previous dist/ in place, and the test then exercises code with no sabotage in it at all. The first run of the Skia CLI sabotages did exactly this three times over and proved nothing.

Two habits make it visible. Keep the sabotage compiling -- change a condition's threshold rather than deleting its body, so every variable stays used and every type stays inferable -- and read the build's exit status rather than only the test's output. A sabotage that prints no failing test and no error is the case to look at twice.

Knowing whether CI actually passed

scripts/ci_status.py says what each job last decided, which is not what gh run list shows. gh shows the latest run; when that run was cancelled by a newer push there is no verdict in it, and nothing distinguishes "not tested" from "tested and fine"; a cancelled job is not a failed one.

That cost five runs once. The Linux job was cancelled on five consecutive pushes to main while Windows failed on every one of them, and what exposed it was a sixth push adding a macOS job, whose failure was then assumed to be new. Five red runs read as four cancellations and a mystery.

  build and test           success   ace5dd3  2026-09-26T06:25:27Z
  the AppKit host          success   ace5dd3  2026-09-26T06:25:27Z
  the whole Windows stack  success   ace5dd3  2026-09-26T06:25:27Z

Each row is the newest run that job reached a verdict in, and a row gains <- and N newer run(s) never finished, so this is older than it looks when there are later runs it said nothing in. That count is the blind spot, and it is the number that would have exposed the day above.

Shards fold together (the AppKit host (shard 2) is the AppKit host) and a job counts as decided for a run only when every shard decided. Three green shards and one cancelled is not a tested commit. It exits non-zero when any job's last verdict was a failure, so it works as a check and not only a report.

The root cause is fixed too: cancel-in-progress is now ${{ github.ref != 'refs/heads/main' }}, so main never loses a result and branches still free their runners. scripts/test_ci_status.py checks the logic against that five-run history, because a bug in a tool like this points the same comfortable way as the problem it was written for.

near is a macro, and it is in <minwindef.h>

A test helper called near compiles on both desktops and fails on Windows with error: expected unqualified-id, because near and far are legacy no-op macros the Windows headers still define. The declaration becomes bool (float, float).

It does not fail in every file, which is what makes it confusing: it fails in the ones whose includes reach windows.h, which <react/renderer/attributedstring/AttributedString.h> does. Three core tests spell it near today and compile, and test_text_shadows.cpp did not. Call it closeTo. It cost a CI round trip, which is the only reason it is written down here.

Windows, on a machine you own

The Windows job is the one nothing here can check. No development machine in this project runs Windows, so a Windows-only file is written, pushed, and answered fifteen minutes later, and if it is wrong, that is fifteen minutes per attempt. It is also the least of the problem: the Windows implementation of the subprocess capability passed on its first run, and the two things that failed around it were a test that assumed my machine and a missing include.

So this is worth setting up when there is Windows work in flight, and not otherwise.

scripts/setup_windows_runner.ps1, run as administrator on the Windows machine, enrols it under the label basalt-windows and installs the runner as a service. It checks first and installs nothing: Visual Studio with the C++ workload and C++ Clang tools for Windows (the core is built with clang-cl, not cl: see docs/PORTING.md), vcpkg with VCPKG_INSTALLATION_ROOT pointing at it, Git for Windows for the bash that runs bootstrap.sh, cmake, ninja and node. A machine missing any of them is told all of them at once.

Enrolling a machine does not move the job to it. That is a separate step:

gh variable set WINDOWS_RUNNER --body basalt-windows   # use it
gh variable delete WINDOWS_RUNNER                      # go back to GitHub's image

Two decisions rather than one, because a self-hosted runner that is switched off does not fail a job: it queues it, for the job's six-hour timeout, and a run that never reports looks exactly like a run that is still going. Deleting one variable is a fix that can be made from a phone; taking a label out of a workflow and pushing it is not.

What it buys is that ccache, vcpkg's binaries and the vendored third_party survive on local disk between runs rather than being restored from a cache, so a warm run is minutes rather than a quarter of an hour. What it costs is that the machine's state is now part of the build: a green run on a machine with something left over from last month is not the same evidence as a green run on a fresh image. When a Windows failure looks impossible, unset the variable and let GitHub's image answer.

Sharding

--shard I/N runs the Ith of N shards of the end-to-end suite, and BASALT_COMPARE_SHARD=I/N does the same for compare_all.sh. CI runs three of each, on the same index, so a shard does a third of both.

Why three, what each job actually costs, and the ccache fault that hid inside those numbers are recorded in docs/ci-performance.md, which is in the repository but not on the site. None of it is needed to run a shard.

The crash reports a passing run leaves on macOS

A full run leaves a crash report named after the host, and it is a test. test_crash_handler sets BASALT_TEST_CRASH, which makes installCrashHandler raise the signal on purpose, because a handler that has never run is a guess. macOS reports any process that dies on a signal, so every run of the end-to-end suite writes one .ips into ~/Library/Logs/DiagnosticReports. There is no way to opt a process out of that reporting, so this is the note instead.

Telling it from a real one takes one line of the report's faulting thread:

raise <- basalt::(anonymous namespace)::crashIfAsked <- basalt::installCrashHandler

That is the deliberate one, and it happens before a window exists. A real crash is anywhere else. One report per run, named after whichever host ran: measured by running the scenario on each, which leaves a basalt_appkit report and a basalt_gtk one. Reading the stack out of a report, rather than opening the Console:

python3 - <<'EOF'
import glob, json, os
reports = os.path.expanduser("~/Library/Logs/DiagnosticReports/basalt*.ips")
for path in sorted(glob.glob(reports)):
    header, body = open(path).read().split("\n", 1)
    head, report = json.loads(header), json.loads(body)
    images = report.get("usedImages", [])
    thread = report["threads"][report.get("faultingThread", 0)]
    print(head["timestamp"], head["app_name"],
          report.get("exception", {}).get("signal"))
    for frame in thread["frames"][:10]:
        image = images[frame["imageIndex"]] if "imageIndex" in frame else {}
        print("   ", image.get("name", "?"), frame.get("symbol", ""))
EOF

Worth doing after a session of work rather than never: every one of the twenty-four reports on this machine on 2026-10-08 was either that deliberate crash or a bug that the same day's commits fixed -- nineteen of them the use-after-free in ~GtkFocusManager, which the GTK suite's own crash handler had already named, and two from GSK assertions that the box shadow and accessibilityLabelledBy work turned into clamps and a different call. The check is cheap and it is the only place a crash that happens after a test's assertions pass would show up.

Test instruments, and what they cannot reach

Dragging out has no automated coverage at all, and cannot have: once a drag begins the system owns it, and no instrument can put a file manager on the other end to receive the drop. What the suite does prove is that a drag source is inert when no view is marked: every press, scroll and context-menu scenario passes with one attached, which is worth knowing and is not the same as proving a drag works. The verification is e2e/drop.tsx's green row and a person dragging it somewhere.

An instrument goes on all three hosts, or the scenario that uses it skips where it is missing. Written down because it has been got wrong twice, the same way both times: BASALT_TEST_QUIT and then BASALT_TEST_DROP were added to GTK and AppKit, the scenario was written against those two, and Windows failed for a reason that had nothing to do with what the scenario tests. The failure is especially misleading: "the inner target was not told" reads as a broken feature rather than a missing instrument, and the feature was fine.

BASALT_TEST_DROP takes x,y:path and reports a file dropped at that point. It enters below the toolkit, the way BASALT_TEST_TAP does and for the same reason: a real drag needs a source outside the process, and no test can conjure one. So what it exercises is the hit test, the walk up to the nearest marked ancestor, the accept check, the event and React's half; every part that could be wrong about which view is told and what it is told, and not GtkDropTarget or NSDraggingDestination themselves. Worth stating, because a scenario that looks like it drives a real drag and does not is worse than one that admits it.

BASALT_TEST_QUIT is the same idea one level up: it asks the application to quit the way a person does (Cmd-Q on macOS, the session ending on Linux) so a quit an app refuses can be driven from a script. A count rather than a flag, because one ask cannot show both halves: a refusal is proved by the process still being there afterwards, and the agreement that follows is proved by it going before its own timer.

That the harness's shutdown is exempt is not a detail. BASALT_QUIT_AFTER_MS quits AppKit by calling terminate:, which arrives at the very handler an app uses to refuse, so the demo for this feature would have refused the harness, and every scenario running it would have hung until its own timeout and reported something other than what it was testing. The same trap as the Windows one above, two years of API apart. GTK and Win32 need no exemption: their session signals are not on the path g_application_quit and PostQuitMessage take.

BASALT_QUIT_WHEN_SETTLED=<ms> ends a run when there is nothing left to wait for, rather than on a budget chosen before the process started.

No scenario opts into it today, and the reason is worth reading before re-enabling any. A settle waits for the first mount and for scripted input to be delivered. It cannot wait for the round trip to JavaScript, and most scenarios assert on what JavaScript logged, so the host quits while the evidence is still in flight. Alert.alert failed on CI with "the alert's callback never reached JavaScript"; the hover scenario failed locally with a truncated event list. The opt-in list was built by opting everything in, running the suite and keeping what passed, which is unsound: passing four times does not establish the absence of a race. It was trimmed twice before the method rather than the entries was accepted as the problem, and the suite went back to 445 seconds from 298. Those 147 seconds were not real.

The right shape is per scenario, and the tool already exists: wait for the line the scenario is about to assert on, then write BASALT_TEST_QUIT_FILE. That is what stop_host does for Fast Refresh. It is correct rather than probabilistic, and it is faster than a settle, because it ends at the evidence rather than at a fixed delay after input. Settled means the first mount has been applied and any scripted input has been delivered, plus a short settle for React's next commit. BASALT_QUIT_AFTER_MS stays as the backstop, so nothing can get slower and a host that never settles still ends.

Measured on the whole macOS suite: 445s to 210s, twice, to the second. The developer menu scenario went from 39.5s to 8.2s, test_controls from 43.1s, the application menu from 7.6s to 1.0s.

The first mount is recorded in MountingWalk::applyMutations, which is the one place all three hosts mount through, so this is one code site rather than three, for the reason core/TestQuitFile.h gives about shared spellings. Only the first mount counts: taking the latest would push the settle out on every commit, and an app that re-renders steadily would never settle at all.

The first version saved nothing on the scenarios it was written for. It measured 2.96s against a 3s budget, because the hosts report their scripted schedule whether or not any input was asked for, and that total is never zero, it starts at 1500ms so a tap has a window to land on. Core ignores an empty schedule now, which is what took the no-input case to 1.4s. Deciding whether a schedule is real needs a list of the instruments that make one, and that list must be scheduled instruments: BASALT_TEST_MENU and BASALT_TEST_DIALOG look like they belong and do not, because they answer something when it appears rather than being laid out on a timer. That error was caught by the check in scripts/test_harness.py, which holds the list against what the three host mains actually read, in both directions.

Which scenarios opt in is a list of scenarios that have been run, not of scenarios that looked fine. Every one was opted in at once and the suite run; seven failed, and all seven for the same reason, work after the last input with nothing scheduling it. An animation is time. A refusal followed by an agreement is two events deliberately separated by time. A trace update that takes itself down again is measuring a decay. A settle cannot stand in for a duration that is the thing under test, and those keep their budgets. The list and each exclusion's failure text live in scripts/integration_test.py.

Off entirely in real input mode. A settle accounts for what the host scheduled, and real input is driven from outside the process: the harness waits four seconds for a window and then moves the pointer with xdotool, having set none of the BASALT_TEST_* variables. Nothing is requested, so the host settles at the first mount and has quit before the click lands. Linux CI is the only place that mode runs, which is why every local run on a Mac passed and all three Linux shards failed on the first push. The rule underneath is the exclusion list's rule one step further out: a settle cannot stand in for a duration it cannot see, and it cannot see another process. So the saving is a macOS and Windows saving; Linux CI keeps its budgets.

Guessing is only safe because getting it wrong fails loudly: everything opted in asserts on the dumped tree or a logged line, so quitting early produces a missing assertion rather than a quiet success. BASALT_NO_SETTLE=1 turns the whole thing off, which is how the saving was measured and how to tell "this scenario is broken" from "this scenario needed longer than it was given".

The suite also prints each scenario's duration now. Its cost is the thing most often being worked on here and was only ever visible as a total, which is why every estimate in this area had been a guess.

BASALT_TEST_KEY presses a key: "m;j;q;z+meta", presses separated by ; and modifiers by +. Parsed once in core/TestKeys.h rather than in three hosts, because three parsers is three chances to disagree quietly about what z+meta means: the wrong answer is a combination that matches nothing, which reads as a shortcut that did not fire rather than as a bad instrument.

It enters after the platform translation, not before it, so what it does not exercise is the NSEvent, keyval and WM_KEYDOWN mapping, which is the half most likely to be wrong, and is unit-tested per host instead (twenty-seven tests across the three). A scenario passing here says the registry, the focus path and the round trip to JavaScript work. It says nothing about whether pressing the left arrow produces "ArrowLeft".

On all three hosts, which is the rule below rather than a coincidence.

BASALT_TEST_QUIT_FILE is how the harness ends a run at a moment it picks, rather than on a budget fixed before launch. The host polls for the path and shuts down as soon as it exists, through exactly the path BASALT_QUIT_AFTER_MS takes, so the tree is dumped and the exit code is the app's. Existence is the whole message; nothing is read from the file, which is why there is no race between creating it and seeing it.

It exists because the portable way to end a host used to be a signal, and there is no portable signal: Windows has no SIGTERM and TerminateProcess cannot be handled. The scenario that needed it is Fast Refresh, whose length is not known in advance: an edit reaches the window in a second on a warm laptop and in up to two and a half minutes on a cold CI machine, so no fixed budget is both quick and safe.

Polled at 250ms rather than watched, on the timer each host already runs for BASALT_QUIT_AFTER_MS. A file watcher is three APIs with three failure modes on network and virtual filesystems, and a quarter-second poll cannot fail to be delivered. Measured end to end on GTK and AppKit: 0.24s from writing the file to the process exiting 0 with its tree on disk.

The skip path asserts that quitting this way wrote the widget tree, not merely that the process ended. Without that line CI proves the poll fires and says nothing about the half the instrument was added for, since the only scenario that reads the tree after a quit-file shutdown is the edit path, and the edit path cannot run on a CI machine. It costs a stat and it is the only place the whole chain is checked by a machine rather than by hand.

A host that does not answer within 30 seconds fails the scenario rather than being quietly killed. It was quietly killed at first, and the consequence was immediate: the first Windows run reported green while proving nothing, because a dead poll and a working one are indistinguishable when the fallback is silent. Verifying it fired meant grepping a CI log for the shared line, and the grep matched a compiler diagnostic quoting the header rather than a host logging it. So the check moved into the harness, where it cannot be forgotten.

Not refusable, for the reason the timed quit is not, see below. The variable name, the poll interval and the log line all live in core/TestQuitFile.h so the three hosts cannot disagree about them; BASALT_QUIT_AFTER_MS elapsed; quitting had to be retrofitted onto Windows after a scenario asserted on a string only two hosts printed, and scripts/test_quit_file.cpp asserts the shared spellings rather than trusting them.

BASALT_DUMP_TREE writes every window, each under a --- window <n> --- header, for the same reason it already appended the error inspector: they are separate trees on screen, and nesting one inside another would say something untrue.

A menu bar cannot be reached by any of them, because a menu cannot be opened without a person. BASALT_DUMP_MENU writes the menu the platform actually installed instead, read back from AppKit or from the HMENU rather than from the description that was sent, so it says a <Menu> became a real menu with the shortcuts the platform attached to its roles. It is written even when it is empty, because "this platform has no menu bar" is what a test on Linux asserts.

The wheel has its own: BASALT_TEST_SCROLL takes "x,y,lines" triples separated by ;, a wheel over a point, in surface-root coordinates, positive lines scrolling down, which is the direction contentOffset reads. Each host enters it at a different depth, deliberately: Windows sends a real WM_MOUSEWHEEL through the real window procedure, macOS builds a real NSEvent and hands it to the view its own hit test found, and Linux enters at the scroll manager where GtkEventControllerScroll would. It is also the only way to reach <RefreshControl>, whose gesture on a desktop is a wheel that keeps asking to go up after the list has already reached its top.

A modal dialog is the one thing no instrument could reach and no person is present for, so BASALT_TEST_DIALOG answers it instead of showing it: a button index, or dismiss for the last button. It covers Alert.alert and the share picker on every host, and it is the one place this project deliberately does not drive the real thing: see native/core/TestDialog.h, which sets out why (Apple's own addUIInterruptionMonitor is documented as not firing on recent iOS versions, and reaching into the dialog through accessibility needs a permission an automated run does not have).

What it skips is the presentation and nothing else. The request is built by the same module from the same JavaScript, the answer travels back through the same callback, and on the desktops whose share picker is built from a clipboard and a mail client, a scripted "Copy" really copies, which is what the end-to-end suite asserts on.

Without it, a run that opens a dialog hangs rather than failing: on macOS a sheet that is up stops [NSApp terminate:] outright. That is a real bug rather than a testing inconvenience (a person pressing Cmd-Q would see the same) so the host now closes any open sheet before it tries to quit.

That cuts the other way too, and it is worth knowing before a tree looks wrong: a window that opens under someone's cursor is hovered, before either host has drawn anything. e2e/hover.tsx is left out of scripts/compare_all.sh for that reason, and the end-to-end suite asserts the order events arrive in rather than that nothing else happens.

Animations the suite drives itself

An animated <Image> does not animate on its own. The view advances by a number of milliseconds its caller passes, and the mounting manager is what starts a clock, so a test says "a hundred and twenty milliseconds later" and renders.

That split exists because of a real failure rather than a preference. The first version drove itself from the GTK frame clock, and renderView spins the main loop to get a frame out of a widget, so every pixel assertion found the animation several frames past where it left it: four samples of the same GIF reported red, red, red, red with the frame changing underneath. Both hosts are split the same way, even though an AppKit timer is inert in a suite that never runs its run loop.

Fonts loaded at runtime, and the font map on a Mac

Three of the four font tests in each host's suite register a font the machine already has, under a second name of the app's choosing, which is the mapping expo-font needs and all a portable test can arrange. The fourth registers one the machine does not have, because that is the case a loader exists for, and it gets there without a fixture font: tests/RenamedFontFile.h copies a real file and renames the family inside it, one byte per occurrence of the name, so no offset in the file moves and no checksum has to be recomputed.

The GTK suite stops early for that fourth test on a Mac, and it says so rather than failing. Homebrew's Pango answers pango_cairo_font_map_get_default with a PangoCairoCoreTextFontMap, which never consults fontconfig, so a font added to fontconfig is registered and invisible. Running the GTK host next to the AppKit one is a comparison arrangement rather than a target, and the Linux fact is proved on Linux.

Expo, and the apps that need it

Two of the apps in e2e/ import an Expo package, notifications.tsx imports expo-notifications, and this directory is not an npm package, so there is nothing for Metro to resolve them against. Both halves have to be pointed at an app that does have them installed:

cmake -B build -DBASALT_EXPO_MODULES_CORE=/path/to/app/node_modules/expo-modules-core
BASALT_EXPO_APP=/path/to/app scripts/bundle.sh --entry notifications --out notifications.jsbundle
BASALT_EXPO_APP=/path/to/app scripts/integration_test.py

The same app for both, so that Expo's C++ and its JavaScript are the same version, the same reason the host builds against the app's React Native. Without them the host has no Expo at all and the scenario skips saying so, which is what a plain checkout and every CI job do.

e2e/fonts.tsx needs the host's half and not the app's: it reaches expo.modules.ExpoFontLoader through the global rather than importing expo-font, so only the build flag matters and no package is borrowed.

cmake -B build-expo -G Ninja -DRN_DIR=../react-native/packages/react-native \
  -DBASALT_EXPO_MODULES_CORE=/path/to/app/node_modules/expo-modules-core
cmake --build build-expo --target basalt_appkit
scripts/bundle.sh --prod --platform macos --entry index.tsx --out main.jsbundle \
  --build-dir build-expo
scripts/integration_test.py --build-dir build-expo -k "font loaded at runtime"

That is how it was first run, against Expo 57 and a host built on React Native main, on 2026-10-09.

Notifications, and the daemon that is never there

GtkNotifications.cpp sends Notify over the session bus, and neither a developer's Mac nor a CI runner has a desktop's notification daemon listening for it. So the suite brings its own: basalt_notification_stub owns org.freedesktop.Notifications, answers Notify with an id and prints what it was asked to show, and scripts/integration_test.py starts it on a session bus of its own for the length of one scenario.

dbus-daemon directly rather than dbus-run-session: on macOS the latter insists on launchd's socket and fails with "DBUS_LAUNCHD_SESSION_BUS_SOCKET is empty". brew install dbus is enough on a Mac; a Linux runner has it already.

Without dbus-daemon the scenario still runs and asserts the other half: that the host reports why it cannot send, which is also what macOS and Windows report on every machine, daemon or not.

Running on Linux

The project targets Linux and is developed on a Mac, so everything here should be checked on Linux before it is believed. A VM is enough:

brew install lima
limactl start --name=basalt <a template with vz, 8 cpus, 16GiB, 120GiB>
limactl shell basalt

Apple Virtualization rather than QEMU: the guest is aarch64 like the host, so it runs at native speed. Building React Native's core and Hermes under emulation would take hours.

Inside, on Ubuntu 24.04:

sudo apt install build-essential clang cmake ninja-build pkg-config git curl \
  libgtk-4-dev libpango1.0-dev libglib2.0-dev libgoogle-glog-dev \
  libboost-dev libboost-regex-dev libfmt-dev libdouble-conversion-dev \
  libgflags-dev libssl-dev libcurl4-openssl-dev
# Node 24: React Native needs ^22.13 || ^24.3 || >= 26, and 24.04 ships 18.
curl -fsSL https://deb.nodesource.com/setup_24.x | sudo -E bash - && sudo apt install nodejs

Then the usual bootstrap, cmake, build.

Rendering, on Wayland

A real compositor, headless, with no GPU:

sudo apt install sway grim mesa-vulkan-drivers libgl1-mesa-dri
printf 'output HEADLESS-1 resolution 1400x1000\ndefault_border none\n' > /tmp/sway.conf
XDG_RUNTIME_DIR=/run/user/$(id -u) WLR_BACKENDS=headless WLR_RENDERER=pixman \
  sway -c /tmp/sway.conf &
WAYLAND_DISPLAY=wayland-1 GSK_RENDERER=gl ./build/basalt_gtk build/main.jsbundle.js
grim shot.png

Two renderer choices, for two different layers, and they are not the same one.

WLR_RENDERER=pixman is the compositor's. Without a GPU, wlroots' default fails with drmGetDevices2 failed and sway starts but draws nothing.

GSK_RENDERER=gl is GTK's, and it must not be cairo. GTK 4.14's cairo renderer draws a transformed widget subtree unrotated and in the wrong colour; the demo's rotated card comes out as a flat pink square. The GL renderer gets it right, and works headless through Mesa's software rasteriser, so there is no reason to reach for cairo. This cost an hour of suspecting the transform code, which is fine on macOS and fine here under GL.

Input, on X11

Real pointer events need a seat with input devices, which a headless Wayland compositor does not have: sway sees no inputs, and swaymsg seat … cursor silently does nothing. Getting one means logind, a session and uinput, which is a lot of machinery for one assertion. X11 gives the same guarantee for far less:

sudo apt install xvfb xdotool
Xvfb :99 -screen 0 1400x1000x24 &
DISPLAY=:99 GDK_BACKEND=x11 GSK_RENDERER=cairo scripts/integration_test.py

That is a genuine event through the X server and GDK. The wheel path checks out too: five xdotool click 5 notches move the content exactly 265 points, which is the 53-point step in GtkScrollViewManager times five.

Dragging a window, which no suite can do

<TitleBar.DragRegion> says which parts of an app-drawn header move the window, and three things about it are tested in three different places, because they fail differently:

  • Which points drag is a question about the view tree, and each host answers it with its own walk: test_titlebar_drag.cpp, test_win32_titlebar.cpp and test_appkit_titlebar_drag.mm ask the same four questions in the same order, for the reason the paint tests do.
  • That a press on one is taken is the half that can be written and never called, which is what macOS did until 2026-10-19: the rule existed in core/TitleBarRegions.h, two hosts applied it and the third never asked. a_press_on_a_drag_region_moves_the_window_instead_of_pressing in test_appkit_input.mm is the test that fails if the call goes away.
  • That the window actually moves is nobody's test. It needs a real pointer, and a real pointer on macOS means CGEvent and accessibility permission an automated run does not have. e2e/titlebar.tsx exists to be dragged by hand, and scripts/integration_test.py -k "title bar" runs the same app to check the arithmetic around it: the style the host reports, the metrics reaching a render, and the header being as tall as the metric says.

What is still not covered

Wayland input. Rendering is verified on Wayland and input on X11, but not both at once. Closing it needs a compositor with a real seat: a desktop session in the VM, or a physical Linux machine.

Rendering. All three hosts assert on pixels now, each with no window on screen and no reference image: Direct2D into a WIC bitmap (test_win32_paint.cpp), Core Graphics into a CGBitmapContext through the view's own drawRect: (test_appkit_image.mm and two others), and GSK into a GdkTexture (test_gtk_paint.cpp, over tests/GtkPixels.h). The three ask the same eleven questions in the same order, because the answers are supposed to be the same picture.

That gap is what let GTK's cairo renderer mangle every transform in the demo without a single test noticing. What is still not asserted is the thing a reference image would catch and these cannot: whether a glyph is the right shape. Font rendering differs enough between machines that reference images would not travel, so the pixel suites ask about colour, placement, clipping and order -- all of which are exact -- and text is asserted by where its ink lands rather than by what it looks like.

One caveat lives in the GTK helper and it logs which renderer answered: the GL one is what an application uses and the cairo one is the fallback on a machine without GL, and they do not agree about transforms. So the GTK pixel suite sticks to questions both answer the same way, and transforms are checked through gtk_widget_compute_point.

Threading. The mounting manager asserts it is on the main thread, and text measurement takes a mutex, but nothing exercises the JS thread and the main thread concurrently.

Nothing, as of the day this repository went public. All three hosts are built and tested on every push: Linux and Windows have been for a while, and macOS joined when standard runners became free and the ten-times billing multiplier stopped being a reason to leave it out. The macOS job also runs compare_all.sh, which no other runner can, because no other has both hosts installed.

This paragraph used to say "CI is Linux only ... macOS is covered by whoever is working; it is not covered by a machine", which was already wrong about Windows and stopped being true of macOS on the same day. Left here as the answer to the obvious question rather than deleted.

CI

.github/workflows/ci.yml. Installs the dependencies, builds, and runs both suites under Xvfb with xdotool present, so the end-to-end suite is in real input mode.

It builds against a pinned React Native commit, in scripts/react-native.pin. This project tracks main, which is right for development and wrong for CI: a build that follows main turns every upstream change into a red tick on an unrelated pull request. A separate drift job builds against main weekly and is allowed to fail, so upstream churn is noticed without blocking anything.

Three caches carry the cost: React Native's node_modules and third_party/ (which holds the Hermes build) are keyed on the pin, and ccache on the commit with a rolling fallback.

They are the difference between a 42 minute run and a 2 minute one. Cold, a run installs React Native's dependencies (949MB of node_modules), builds Hermes, and compiles 400-odd objects of React Native core. Warm, it restores 304MB of third_party, hits ccache, and goes straight to the tests.

Two things about those caches that are easy to lose an hour to:

  • The third_party key hashes the whole of bootstrap.sh, so editing a comment in it costs a Hermes rebuild. That is deliberate: over-invalidating is the safe direction, and the script changes rarely now.
  • A cache saved on a branch is invisible to main. GitHub scopes caches to the branch that created them, and shares them only downwards, to that branch's descendants. So the first run after merging a branch is cold all over again, and only then do the caches land somewhere every later run can reach.
  • actions/cache will not save a path that escapes the workspace. It says nothing about it either; the cache simply never appears. React Native is therefore cloned to react-native-src/ inside the checkout rather than beside it. Getting this wrong is quiet until the day third_party hits: then bootstrap skips yarn install because codegen is already there, and bundling fails for want of node_modules.

On this page