# How we benchmark a desktop app

> Launch time, memory footprint and CPU, measured the same way on every build. The method behind SolKube's benchmark numbers, and the mistakes it is designed to avoid.

The SolKube team · 2026-11-07 · https://solkube.com/blog/how-we-benchmark-a-desktop-app

Performance claims about desktop apps are easy to make and hard to check. A screenshot of Activity Monitor says little about how the number was taken, what the app was doing, or whether the other app was measured the same way. So SolKube's numbers come from one script, run the same way on every build, with every run committed to the repository next to the code it measured.

## What we measure

The script, `bun run bench`, takes a commit and a machine label. It clones the commit into a cache folder, builds it once without timing so downloads never land in a sample, and then runs three suites.

- **Launch**: time from spawn to the first on-screen window.
- **Runtime**: memory and CPU at rest, while idle, and during a minute of light use.
- **Compile**: clean and incremental builds, for us rather than for you.

Each result is stored as a median plus every sample, so a reader can see the spread and not only the middle.

## A clean sandbox for every launch

The first rule is that the app under test must not carry state from your Mac. Every launch runs with `HOME` pointing at a sandbox folder: setup already done, an empty kubeconfig, no saved tabs and no network. That makes each number the build's own baseline cost, comparable across machines, and it means no real cluster is ever contacted during a benchmark.

## Launch time

A small Swift tool spawns the app and polls `CGWindowListCopyWindowInfo` every 2 ms until the process owns an on-screen window larger than 200 by 200. We launch ten times, three seconds apart. The first launch is cold, right after the build, so we keep it separately; the reported median covers the other nine.

The window threshold matters. Some apps put up a tiny splash or an offscreen window first, and counting that would flatter them.

## Memory: footprint, not RSS

We record two memory figures and lead with one.

The **physical footprint**, from macOS's `footprint` tool, is the memory the app owns, including GPU memory it allocates. It's the number in Activity Monitor's Memory column, so it's the number people see. **RSS**, from `ps`, counts every resident page the process maps, including shared frameworks such as AppKit that every app maps anyway. RSS is useful for spotting leaks but bad for comparing apps, because shared pages make every app look about the same.

Both are summed over the app's whole process tree. An app that does its work in helper processes doesn't get to hide them.

## CPU: idle and light use

Each runtime launch settles for 10 seconds, idles for 60 seconds with the window open, then gets 60 seconds of light use from a JXA script: the cursor sweeps the window at 30 Hz, scrolls every third tick, and every four seconds toggles a panel or opens the cluster switcher, types and closes it. We record CPU time over each 60-second window as a share of one core.

Light use is noisy. The same build has measured 5.7% and 9.3% an hour apart on one Mac, because other processes, thermal state and the window server's own work all vary. So we only compare light-use CPU between runs made back to back on the same machine, and the [benchmarks page](/benchmarks) never mixes hardware in one comparison.

## Rules we hold ourselves to

- **Same machine, same day** for any comparison we publish.
- **Medians, with every sample kept.** A single bad sample can't move the headline.
- **Losses stay visible.** If another build wins a metric, the card says so.
- **No hand-picked runs.** Every run file stays in the repository, including the ones that didn't go our way.

When we add Lens, Freelens and Headlamp to the page, they'll go through the same script on the same Macs.

## Short answers

### What is the difference between memory footprint and RSS on macOS?

RSS (resident set size) counts every page the process has in memory, including shared system frameworks that every app maps. Physical footprint counts the memory the app itself owns, including GPU memory on Apple silicon. Activity Monitor's Memory column shows the footprint, so it is the better measure of what an app costs.

### How do you measure app launch time on macOS?

Spawn the app and poll CGWindowListCopyWindowInfo every 2 ms until the process owns an on-screen window larger than 200 by 200 points. Take the median of nine warm launches after one cold launch.

### Why report medians instead of averages?

A single slow sample, such as a launch that lands while Spotlight indexes, moves an average a lot and a median hardly at all. The median is what you get on a typical run.