libei/README.md

libei
=====

**libei** is a library for Emulated Input, primarily aimed at the Wayland
stack. It provides two parts:
- 🥚 EI for the client side (`libei`)
- 🍦 EIS for the server side (`libeis`)

The communication between the two is an implementation detail, neither
client nor server need to care about the details. Let's call it the BRidge
for EI, or 🥣 brei.

For the purpose of this document, **libei** refers to the project,
`libei`/`libeis` to the two libraries provided.

In the Wayland stack, the EIS server component is part of the
compositor, the EI client component is part of the Wayland client.


```
    +--------------------+             +------------------+
    | Wayland compositor |---wayland---| Wayland client B |
    +--------------------+\            +------------------+
    | libinput | libeis  | \_wayland______
    +----------+---------+                \
        |          |           +-------+------------------+
 /dev/input/       +---brei----| libei | Wayland client A |
                               +-------+------------------+
```

The use-cases **libei** attempts to solve are:
- on-demand input device emulation, e.g. `xdotool` or more generically the
  XTEST extension
- input forwarding, e.g. `synergy`

**libei** provides three benefits:
- separation
- distinction
- control

**libei** provides **separation** of emulated input from normal input.
Emulated input is a distinct channel for the compositor and can thus be
handled accordingly. For example, the server may show warning sign in the
task bar while emulated input is active.

The second benefit is **distinction**. Each **libei** client has its own
input device set, the server is always aware of which client is requesting
input at any time. It is possible for the server to treat input from
different emulated input devices differently.

The server is in **control** of emulated input - it can filter input or
discard at will. For example, if the current focus window is a password
prompt, the server can simply discard any emulated input. If the screen is
locked, the server can cancel all emulated input devices.

For the use-case of fowarding input (e.g. `synergy`) **libei** provides
capability monitoring. As with input emulation same benefits apply -
input can only be forwarded if the compositor explicitly does so.

High-level summary
------------------

A pseudo-code implementation for server and client are available in
the [`examples/`](https://gitlab.freedesktop.org/whot/libei/-/tree/master/examples)
directory.

The server starts a `libeis` context (which can be integrated with flatpak
portals) and uses the `libeis` file descriptor to monitor for
client requests.

A client starts a `libei` context and connects to the server - either
directly, via DBus or via a portal. The server (or the portal) approves or
denies the client. After successful authentications the client can request
the creation of a device with capabilities `pointer`, `keyboard` or `touch`.

The client triggers input events on this device, the server receives those
as events through `libeis` and can forwards them as if they were libinput
events. The server has control of the client stream. If the stream is
paused, events from the client are discarded. If the stream is resumed, the
server will receive the events (but may discard them anyway depending on
local state).

The above caters for the `xdotool` use-case.

The client may request to monitor a capability. When the server deems the
client to be in-focus, it forwards events from real devices to the client.
The decision of what constitutes logical focus and what events to forward
are up to the server.

For a `synergy` use-case, the setup requires:
- `synergy-client` on host A monitoring the mouse and keyboard capabilities
- `synergy-server` on host B requesting a mouse/keyboard capability device
- when `synergy-client` receives events via `libei` from compositor A it
  forwards those to the remote `synergy-server` which sends them via `libei`
  to the compositor B.

The compositor may choose to implement a hotkey to start/stop the events or
it may implement the screen edges to be the hot key.


Open questions
--------------

### Flatpak integration

Where flatpak portals are in use, `libei` will communicate with
the portal and `libeis` with the portal implementation (e.g.
`xdg-desktop-portal-gdk`). The portal is reponsible for
allowing the client to connect and restrictions on the devices a client may
create. `libeis` will run in a private namespace of the compositor.

The portal **may** control suspending/resuming devices (in addition to the
server). The UI for this is not yet sorted.

### Authentication

Sandboxing is addressed via flatpak portals but a further level is likely
desirable, esp. outside flatpak. The simplest solution is the client
announcing the name so the UI can be adjusted accordingly. API wise-maybe an
opaque key/value system so the exact auth can be left to the implementation.

### Triggers

For `synergy` we need capability monitoring started by triggers, e.g. the
client requests a pointer capability monitoring when the real pointer hits
the screen edge. Or in response to a keyboard shortcut.

### Keyboard layouts

The emulated input may require a specific keyboard layout, for example
for softtokens (usually: constant layout "us") or for the `synergy` case
where the remote keyboard should have the same keymap as the local one, even
where the remote host is configured otherwise.

libei provides keymap negotation: the client can pick a keymap, the server
can accept it, refuse it, or override it with its own. In the latter two
cases it is up to the client to handle the result.

Modifier state handling, group handling, etc. is still a private
implementation so even where the server supports individual keymaps. So it
remains to be seen if this approach is sufficient.