docs(capture): document stream behavior and performance checks
This commit is contained in:
1 parent
533cd74c94
commit
835aadc3eb
3 files changed
+204
-5
No files matched your search
@@ -78,6 +78,7 @@ With release binaries built, run isolated integration checks:
|
|||||||
```sh
|
```sh
|
||||||
FERESE_TEST_SHORTCUTS=1 python3 scripts/tests/test_shortcuts_isolated.py
|
FERESE_TEST_SHORTCUTS=1 python3 scripts/tests/test_shortcuts_isolated.py
|
||||||
FERESE_TEST_WINDOW_CAPTURE=1 python3 scripts/tests/test_window_capture_isolated.py
|
FERESE_TEST_WINDOW_CAPTURE=1 python3 scripts/tests/test_window_capture_isolated.py
|
||||||
|
FERESE_TEST_WINDOW_CAPTURE_SCALE=1 python3 scripts/tests/test_window_capture_scale_isolated.py
|
||||||
FERESE_TEST_INHIBIT=1 python3 scripts/tests/test_inhibit_isolated.py
|
FERESE_TEST_INHIBIT=1 python3 scripts/tests/test_inhibit_isolated.py
|
||||||
FERESE_TEST_RESTORE=1 python3 scripts/tests/test_restore_isolated.py
|
FERESE_TEST_RESTORE=1 python3 scripts/tests/test_restore_isolated.py
|
||||||
FERESE_TEST_WINDOW_STREAM=1 python3 scripts/tests/test_window_stream_isolated.py
|
FERESE_TEST_WINDOW_STREAM=1 python3 scripts/tests/test_window_stream_isolated.py
|
||||||
|
|||||||
@@ -87,3 +87,188 @@ window-capture readback separately. Current effect metrics are placeholders;
|
|||||||
`capture_us` covers monitor capture batches only. Exact cursor-only KMS updates
|
`capture_us` covers monitor capture batches only. Exact cursor-only KMS updates
|
||||||
and detailed scanout rejection causes require additional upstream observability.
|
and detailed scanout rejection causes require additional upstream observability.
|
||||||
Broader primary-plane and overlay policies still need hardware validation.
|
Broader primary-plane and overlay policies still need hardware validation.
|
||||||
|
|
||||||
|
## Capture scene reuse
|
||||||
|
|
||||||
|
With `FERESE_TRACE_PERFORMANCE=1`, both backends emit `capture scene outcomes`
|
||||||
|
through the `ferese::render` log target. Each report covers one retained output
|
||||||
|
view, with separate counts for cursor-hidden and cursor-embedded capture, and
|
||||||
|
resets its counters after roughly five seconds of capture activity.
|
||||||
|
|
||||||
|
`preparations` counts successful filtered-scene preparations, `scene_reuses`
|
||||||
|
counts checks that skipped rebuilding that scene, and `compositions` counts
|
||||||
|
renders with nonempty capture damage. A pending damage-waiting frame may be
|
||||||
|
checked repeatedly without a delivery. A preparation can also find unchanged
|
||||||
|
pixels, so it need not increase the composition count. These counters describe the
|
||||||
|
capture source; they do not count PipeWire deliveries, measure GPU time or
|
||||||
|
describe window-only capture. For a mirror, they describe its filtered source
|
||||||
|
before fitting it to the target output.
|
||||||
|
|
||||||
|
An unchanged view reuses its retained GPU image before constructing capture
|
||||||
|
render elements. Surface commits, layout changes, animation ticks and privacy
|
||||||
|
changes invalidate it, including the final animation frame. Cursor updates
|
||||||
|
invalidate only views that embed the cursor. Output identity, scale, transform
|
||||||
|
and geometry are checked as well, so a remapped or reconfigured output cannot
|
||||||
|
reuse the previous view by mistake. Capture requests alone do not invalidate
|
||||||
|
the image, and multiple mutations before a request uses it are coalesced.
|
||||||
|
|
||||||
|
This is conservative invalidation: an unrelated redraw may still cause a
|
||||||
|
preparation, after which damage tracking can find unchanged pixels. It does not
|
||||||
|
reuse the display swapchain or partially update consumer buffers. DMA-BUF and
|
||||||
|
shared-memory delivery still write complete frames, which keeps a recycled
|
||||||
|
buffer from exposing pixels left by a previous use.
|
||||||
|
|
||||||
|
## IPC snapshot CPU benchmark
|
||||||
|
|
||||||
|
The ignored release test below measures construction of unchanged desktop event
|
||||||
|
snapshots with the default theme. It warms up the state, then reports three runs
|
||||||
|
of 2,000 snapshots. Keep the runtime and state directories private so the test
|
||||||
|
cannot connect to your desktop or write its configuration.
|
||||||
|
|
||||||
|
```sh
|
||||||
|
ipc_benchmark_runtime=$(mktemp -d)
|
||||||
|
env -u FERESE_SOCKET \
|
||||||
|
XDG_RUNTIME_DIR="$ipc_benchmark_runtime" \
|
||||||
|
XDG_CONFIG_HOME="$ipc_benchmark_runtime" \
|
||||||
|
XDG_STATE_HOME="$ipc_benchmark_runtime" \
|
||||||
|
XDG_CACHE_HOME="$ipc_benchmark_runtime" \
|
||||||
|
FERESE_STARTUP_TEST_CHILD=ipc_events::tests::profile_unchanged_event_snapshots \
|
||||||
|
cargo test --release --locked -p ferese profile_unchanged_event_snapshots -- --ignored --nocapture
|
||||||
|
rm -rf -- "$ipc_benchmark_runtime"
|
||||||
|
```
|
||||||
|
|
||||||
|
Compare the reported `elapsed_us` values on the same machine and build profile.
|
||||||
|
These timings cover CPU snapshot construction, while the DRM procedure above
|
||||||
|
measures rendering and presentation under real desktop workloads.
|
||||||
|
|
||||||
|
For populated desktops, replace `profile_unchanged_event_snapshots` with
|
||||||
|
`profile_populated_event_publication` in both the test filter and
|
||||||
|
`FERESE_STARTUP_TEST_CHILD`. This test creates one output and private Wayland
|
||||||
|
clients, then measures three runs of 2,000 unchanged publication checks with
|
||||||
|
0, 1, 20 and 100 windows. Setup and the initial subscription event happen before
|
||||||
|
timing. Window records stay typed through comparison and serialize only when
|
||||||
|
an event is sent, avoiding JSON objects for checks that produce no update.
|
||||||
|
The benchmark includes construction, comparison and disposal of each snapshot,
|
||||||
|
while socket serialization and subscriber processing are excluded. Keep CPU
|
||||||
|
affinity identical when comparing binaries on machines with mixed core types.
|
||||||
|
|
||||||
|
## Shell snapshot handoff benchmark
|
||||||
|
|
||||||
|
The shell's ignored release test measures delivery of a completed desktop
|
||||||
|
snapshot from the control thread to the UI. It uses a private socket pair and
|
||||||
|
tests snapshots with 1, 20 and 100 windows, reporting three runs of 2,000
|
||||||
|
handoffs for each size.
|
||||||
|
|
||||||
|
```sh
|
||||||
|
cargo test --release --locked -p ferese-shell profile_snapshot_handoff -- --ignored --nocapture
|
||||||
|
```
|
||||||
|
|
||||||
|
Snapshot preparation happens before timing; delivery, polling and destruction
|
||||||
|
are included. Compare runs on the same machine without concurrent builds.
|
||||||
|
The benchmark excludes protocol decoding, UI layout and drawing, so its results
|
||||||
|
describe this handoff's CPU cost rather than overall shell responsiveness.
|
||||||
|
|
||||||
|
## Monitor capture benchmark
|
||||||
|
|
||||||
|
Monitor captures keep their own damage history alongside the capture texture.
|
||||||
|
The first frame fills the texture, and later frames redraw only damaged regions
|
||||||
|
before readback. Cursor inclusion, disappearing surfaces and output changes are
|
||||||
|
resolved against the capture scene, independently of the displayed image.
|
||||||
|
Cursor-included and cursor-hidden streams retain separate targets, so alternating
|
||||||
|
between them does not invalidate otherwise unchanged captures. Mirrored captures
|
||||||
|
also track damage in their filtered source composition before fitting it to the
|
||||||
|
mirror output.
|
||||||
|
|
||||||
|
Run the ignored release benchmark on a hardware EGL device:
|
||||||
|
|
||||||
|
```sh
|
||||||
|
cargo test --release --locked -p ferese profile_output_capture -- --ignored --nocapture
|
||||||
|
```
|
||||||
|
|
||||||
|
It creates a synthetic 3840×2160 scene with overlapping translucent panes and
|
||||||
|
tests unchanged frames, cursor movement and full-background changes. Each case
|
||||||
|
warms up for 60 frames and reports three runs of 60 frames. With `readback=true`,
|
||||||
|
timing includes a full-frame GPU readback and a copy into CPU memory. With
|
||||||
|
`readback=false`, a one-pixel readback waits for GPU completion while avoiding
|
||||||
|
the full-frame transfer cost. These are elapsed times through the renderer,
|
||||||
|
rather than GPU timestamp measurements.
|
||||||
|
|
||||||
|
Compare old and new release binaries in alternating runs on the same GPU, and
|
||||||
|
record other GPU workloads. The test captures no desktop content and excludes
|
||||||
|
PipeWire delivery, encoding and network transport, so it cannot establish an
|
||||||
|
end-to-end OBS or call-frame latency improvement.
|
||||||
|
|
||||||
|
## Window capture conversion benchmark
|
||||||
|
|
||||||
|
The one-shot window capture path converts premultiplied readback pixels to straight
|
||||||
|
ARGB. Streaming performs this conversion in its GPU transfer shader. Opaque pixels need only a channel swap, while partial
|
||||||
|
alpha uses a 64 KiB table compiled into read-only storage. The table preserves
|
||||||
|
the original rounded and saturated conversion without divisions in the frame
|
||||||
|
loop or runtime allocation for lookup data.
|
||||||
|
|
||||||
|
```sh
|
||||||
|
cargo test --release --locked -p ferese profile_window_conversion -- --ignored --nocapture
|
||||||
|
```
|
||||||
|
|
||||||
|
The test measures 1080p and 4K frames with opaque content, translucent edges,
|
||||||
|
uniform translucency and varying alpha. Each case warms up for three frames,
|
||||||
|
then reports three runs of three frames. Input colours vary across the frame,
|
||||||
|
and the benchmark restores the premultiplied input before each conversion,
|
||||||
|
outside the timed interval. Compare release builds with matching CPU affinity
|
||||||
|
and record competing CPU workloads.
|
||||||
|
|
||||||
|
These timings isolate pixel conversion. GPU readback, copying into client
|
||||||
|
buffers, PipeWire delivery and encoding require separate measurements; use
|
||||||
|
live thread CPU samples to identify how much conversion contributes to a
|
||||||
|
running sharing session.
|
||||||
|
|
||||||
|
|
||||||
|
## Native stream validation
|
||||||
|
|
||||||
|
Streaming uses `ext-image-copy-capture-v1` with persistent capture sessions and a
|
||||||
|
four-buffer PipeWire pool. The compositor writes into those buffers directly,
|
||||||
|
using DMA-BUF when allocation and format negotiation permit it, or anonymous
|
||||||
|
shared memory otherwise. A pending capture waits for source damage, and each
|
||||||
|
completed buffer is delivered once. DMA-BUF frames become available only after
|
||||||
|
the exported GPU fence signals; cancellation, privacy changes and source removal
|
||||||
|
also invalidate pending delivery.
|
||||||
|
|
||||||
|
The isolated window test captures synthetic clients through a real PipeWire
|
||||||
|
consumer, checks silence while content is static, then requests distinct updates.
|
||||||
|
It starts a callback-driven animation on an inactive workspace, reconnects
|
||||||
|
without changing the source, resizes while paused, verifies the replacement pool
|
||||||
|
and revokes the window's capture permission. A monitor stream
|
||||||
|
then exercises damage waiting and real updates through the filtered output path.
|
||||||
|
Set `FERESE_TEST_CAPTURE_TRANSPORT` to `shm` or `dma` to require that memory type
|
||||||
|
in the native consumer, rather than merely accepting an available transport.
|
||||||
|
DMA-BUF checks require a hardware-backed nested compositor and access to its
|
||||||
|
render node.
|
||||||
|
|
||||||
|
Set `FERESE_TEST_CAPTURE_OBS_BUFFERS=1` with either transport to exercise OBS's
|
||||||
|
memory-type bitmask and mapped-buffer requirements. This catches negotiation
|
||||||
|
failures that a consumer offering just one memory type can miss.
|
||||||
|
|
||||||
|
```sh
|
||||||
|
FERESE_TEST_WINDOW_CAPTURE=1 FERESE_TEST_CAPTURE_TRANSPORT=shm \
|
||||||
|
python3 scripts/tests/test_window_capture_isolated.py
|
||||||
|
FERESE_TEST_WINDOW_CAPTURE=1 FERESE_TEST_CAPTURE_TRANSPORT=dma \
|
||||||
|
python3 scripts/tests/test_window_capture_isolated.py
|
||||||
|
cargo test --locked -p ferese render::capture::stream_gpu_tests -- --ignored
|
||||||
|
```
|
||||||
|
|
||||||
|
The GPU tests compare all 65,536 alpha/channel combinations against the CPU
|
||||||
|
conversion and check all eight output transforms with both texture orientations.
|
||||||
|
Native streaming tests establish buffer transport and pixel correctness; measure
|
||||||
|
CPU, GPU and encoding workloads separately before attributing a change in total
|
||||||
|
streaming performance to capture alone.
|
||||||
|
|
||||||
|
The recorder retains the last owned image and timestamps one final reference when
|
||||||
|
recording stops, so a static desktop keeps its full duration without requesting
|
||||||
|
or encoding duplicate frames throughout the idle interval. The regression test
|
||||||
|
covers an explicit stop, EOS from the capture source and continuous audio while
|
||||||
|
the video stays static:
|
||||||
|
|
||||||
|
```sh
|
||||||
|
cargo test --locked -p xdg-desktop-portal-ferese --bin ferese-record \
|
||||||
|
recorder::tests::encoder_preserves_the_duration_of_a_static_capture -- --ignored
|
||||||
|
```
|
||||||
+18
-5
@@ -30,8 +30,11 @@ Apps requesting multiple sources can select up to eight.
|
|||||||
A shared display includes everything visible on it, including notifications.
|
A shared display includes everything visible on it, including notifications.
|
||||||
Window sharing captures only the selected app's content and subsurfaces, even if
|
Window sharing captures only the selected app's content and subsurfaces, even if
|
||||||
the window is covered or on another workspace. It excludes separate popups and
|
the window is covered or on another workspace. It excludes separate popups and
|
||||||
compositor decorations. The video size follows window resizes; closing the window
|
compositor decorations. Window streams preserve backing pixel density across
|
||||||
ends sharing.
|
the captured surface tree when it exceeds the display scale. For example, a 2×
|
||||||
|
application buffer stays at 2× in the stream on a 1.75× display, avoiding a
|
||||||
|
fractional downsample before the recording app scales it. The video size follows
|
||||||
|
window resizes and backing buffer density changes; closing the window ends sharing.
|
||||||
|
|
||||||
The picker follows your shell colors, font, corners, opacity, and blur.
|
The picker follows your shell colors, font, corners, opacity, and blur.
|
||||||
|
|
||||||
@@ -64,9 +67,19 @@ see the [package lists](installation.md#requirements).
|
|||||||
The installer supplies Ferese's portal backend and selects it for screen sharing.
|
The installer supplies Ferese's portal backend and selects it for screen sharing.
|
||||||
Log into a new Ferese session after installation.
|
Log into a new Ferese session after installation.
|
||||||
|
|
||||||
Streams run at up to 30 fps with hidden or embedded cursors. Frames are copied
|
Streams negotiate their frame rate with the recording app, with a default of
|
||||||
through CPU-accessible buffers. Large streams or several simultaneous streams
|
30 fps and a ceiling of 60 fps. To set a different ceiling, give the portal
|
||||||
can therefore use substantial CPU and memory bandwidth.
|
process `FERESE_CAPTURE_MAX_FPS` with an integer from 1 to 240. This controls
|
||||||
|
capture independently of the display refresh rate; it does not make an idle
|
||||||
|
desktop generate frames.
|
||||||
|
|
||||||
|
On supported GPUs, Ferese renders directly into DMA-BUF buffers shared with
|
||||||
|
PipeWire and waits for GPU completion before handing them to the recording app.
|
||||||
|
If GPU buffers are unavailable, capture uses shared memory without the portal's
|
||||||
|
intermediate frame copies. Both paths wait for source damage, while a resumed
|
||||||
|
consumer receives an initial image even if the desktop has stayed unchanged.
|
||||||
|
For troubleshooting, `FERESE_CAPTURE_FORCE_SHM=1` selects shared memory; large
|
||||||
|
or rapidly changing captures on this path still incur GPU readback costs.
|
||||||
|
|
||||||
## Troubleshooting
|
## Troubleshooting
|
||||||
|
|
||||||
|
|||||||
Reference in new issue
Block a user