Support isolated live window ScreenCast streams

Render selected toplevels through a private portal-only capture protocol, preserve logical snapshot geometry, and renegotiate PipeWire video on window resizing. Bound outstanding snapshots and retry temporary contention; stop on source closure and session locking.

Validation: portal unit tests (31), capture budget unit, private-bus portal contracts, and real nested/PipeWire isolation, workspace, resize, close and paused-lock tests pass. Independent review cleared. Monitor restore integration timed out on both this build and the unchanged previous compositor under the current nested host.
This commit is contained in:
Samuel Onoja committed 2026-09-30 04:02:33 +01:00
1 parent 56a888adb3
commit baca852ee9
26 files changed
+1307 -139

No files matched your search

+18 -6
View File
@@ -47,7 +47,7 @@ standard file, print and application dialogs. Install both backends.
for acknowledgements for up to one second. Power confirmations list blockers
and recheck them before acting; authentication failure cancels the query.
UserSwitch is recorded, but Ferese currently has no user-switch operation.
- **ScreenCast:** monitor sharing with a native picker, PipeWire streams and explicit opt-in
- **ScreenCast:** monitor and isolated window sharing with a native picker, PipeWire streams and explicit opt-in
saved permissions. The frontend stores and revokes saved choices. Invalid
restore data or changed display identity requires fresh consent.
See [screen sharing](screen-sharing.md) for lifetime and recording details.
@@ -73,7 +73,7 @@ Matching D-Bus signatures does not establish complete feature parity.
| Settings | Native, including appearance change signals |
| Screenshot | Native screen/area/window/active-window capture and PickColor |
| Wallpaper | Native desktop, lock-screen and combined targets |
| ScreenCast | Native v4 monitors with logical geometry and opt-in saved permissions; window sources remain missing |
| ScreenCast | Native v4 monitors and isolated windows, logical geometry, live window resizing and opt-in saved display permissions |
| Access, Account, AppChooser, DynamicLauncher, FileChooser, Notification, Print | Delegated to GTK |
| Background | Native application state, per-instance consent and XDG autostart |
| Usb | Native consent; standard frontend handles enumeration and device descriptors |
@@ -88,10 +88,15 @@ The frontend supplies additional APIs such as OpenURI, network monitoring,
document export and permission storage. Adding a similarly named backend service
is neither required nor sufficient for those APIs.
Do not advertise window sources for ScreenCast or input services until the
compositor provides the corresponding operation. Screen crops include occluding
windows and cannot serve as isolated window streams. GNOME's remote-input features require native
compositor/session work, rather than substituting successful empty replies.
Window streams render only the selected toplevel and its subsurfaces, including
on an inactive workspace. Overlapping apps, shell layers, separate popup windows
and compositor decorations are excluded. The private capture protocol is available
only to the native portal executable and checks session opt-in and locking.
Window resizing renegotiates PipeWire video dimensions. Closing the selected
window ends sharing; its lifetime ID cannot select a replacement app. Window
selections require fresh consent and are not saved as display permissions.
InputCapture still needs native zones, barriers and EIS transport. RemoteDesktop
and its Clipboard integration remain deferred.
## Validation
@@ -143,3 +148,10 @@ an isolated bus and nested compositor:
```sh
FERESE_TEST_RESTORE=1 python3 scripts/tests/test_restore_isolated.py
```
Check live window pixels, overlap isolation, inactive workspaces, resizing and
closure using a disposable nested compositor and a real PipeWire consumer:
```sh
FERESE_TEST_WINDOW_STREAM=1 python3 scripts/tests/test_window_stream_isolated.py
```
+17 -9
View File
@@ -1,7 +1,7 @@
# Screen sharing and recording
Ferese includes a ScreenCast portal backend, `xdg-desktop-portal-ferese`.
Applications use the standard desktop portal to request a display, then receive
Applications use the standard desktop portal to request a display or window, then receive
video through PipeWire. A recorder or conferencing application handles encoding,
saving files, or sending video; those tasks do not run inside the compositor.
@@ -25,8 +25,14 @@ may remain at the path shown in the error.
## Start sharing
Use a Wayland screen-sharing or recording source in your application. Ferese
opens a display picker. Select a display and press **Share**. Nothing is selected
by default. Applications that request multiple displays can select multiple available displays.
opens a native source picker. Depending on what the application requests, choose
from displays, windows, or both and press **Share**. Nothing is selected by default.
Applications that request multiple sources can select up to eight. Display and
window choices are grouped separately. Windows show their title and application.
Window streams capture only the selected app's toplevel and subsurfaces, even
when covered or on another workspace. Separate popups and compositor decorations
are excluded. Resizing a shared window updates the video size without ending the
session. Closing that window ends sharing.
The picker uses your Ferese theme colors, font, and shell corner radius. Its
background uses the same compositor material as popovers and authentication
prompts: opaque in solid mode, or blurred and translucent using the shared
@@ -37,14 +43,15 @@ Ferese’s built-in recorder uses the top-bar stop control instead, leaving your
windows accessible after display selection. The command-line recorder keeps the
sharing window when it is not launched by the shell. Press **Stop sharing** or
close the sharing window to revoke an ordinary application’s session. Closing the requesting application,
locking Ferese, disconnecting the selected display, or changing its capture size
also ends sharing. By default, each request asks for consent. If an application requests saved
locking Ferese, disconnecting the selected display, or changing a monitor's capture
size also ends sharing. By default, each request asks for consent. If an application requests saved
permissions, the picker offers an **Allow without asking** switch, off by default.
Permissions can last while the application runs or until revoked, as requested
by the app. The standard portal frontend owns permission storage and revocation.
A changed or missing display, different app, or changed cursor options asks again.
Displays without a reliable EDID identity, including nested previews, always need
fresh consent. Stopping a stream ends that stream; it does not erase a previously
fresh consent. Window selections also require fresh consent and cannot enable saved
display permissions. Stopping a stream ends that stream; it does not erase a previously
saved permission.
Everything visible on a shared display, including notifications, is included.
@@ -76,7 +83,8 @@ configuration is preserved during installation.
## Current capabilities
- Monitor capture, one PipeWire stream per selected display, at up to 30 fps.
- Monitor and isolated window capture, one PipeWire stream per source, up to 30 fps.
- Window streams keep working across workspace switches and renegotiate on resize.
- Hidden or embedded pointer, as requested by the application.
- Logical stream position and size, including fractional output scales.
- Wayland parent-window relationships for the consent picker.
@@ -84,8 +92,8 @@ configuration is preserved during installation.
- Cancellation during selection, an explicit stop control, and process cleanup.
- ScreenCast backend version 4 with explicit opt-in saved display permissions.
Window-only sharing, region selection, and audio capture in Ferese’s recorder
are not implemented yet. Other recording applications can handle audio separately.
Region selection and audio capture in Ferese’s recorder are not implemented yet.
The top-bar recorder currently requests monitors. Other recording applications can handle audio separately.
Video currently uses CPU copies through the compositor's shared-memory screencopy path, not DMA-BUF zero-copy. High-resolution
or multiple-display recording can therefore use significant CPU and memory
bandwidth. A paused stream checks capture availability once per second so locking