# Preemption

Largely based on an explanation of Erlang/Elixir/BEAM’s concurrency model in The Soul of Erlang and Elixir

Prompt/example: a browser establishes a websocket connection with a server, and gives the server various tasks to perform via the WS connection.

## Erlang

• The BEAM VM’s primary abstraction around a unit of execution is a Process
• Processes are lightweight and share no state; all communication is done via message-passing.
• Processses are multiplexed onto n OS threads (where n > 0)
• Processes are pre-empted by the VM, so CPU-bound processes can’t monopolize execution, even on a single thread.
• This structure allows consumers to flood the system with too many CPU-bound threads, which needs application-level logic (as far as I know) to avoid.

## Rust / Tokio

• The most basic abstraction here is a Future, which represents a unit of execution.
• A Future can be pending or ready, and can be polled.
• You can await a future, which:
• Returns immediately if the future is ready
• “Pauses” execution and returns control to the scheduler if the future is not ready.
• Tokio’s abstraction around futures are “tasks”
• Tasks cannot be pre-empted arbitrarily, but Tokio can swap them out in three ways:
• At await points: if the future/task is not ready, the scheduler regains control.
• Every task is given a budget, and every call to an async function decreases the budget.
• When the budget hits zero, all futures called by the current task return “pending” even if they are ready, to force the task to yield control.
• Is this budget tracking limited to async functions in tokio’s stdlib? If not, how is this tracked?
• The task can “cooperate” with the scheduler and yield control voluntarily at strategic points during execution.
• Three approaches to architect the browser + WS prompt:
1. The server’s WS hander could create a new task for each incoming computation, await it, and send down the response.
• This effectively means that there’s no concurrency to be had within the context of a single connection..
2. The server’s WS handler could directly call an async fn for each incoming computation and await it.
3. A more robust approach is to have the WS handler create a new task for incoming connection but not await it.
• Instead, use mpsc (or a similar method) to let the connection handler know when the task is done, so it can send down a response.
• It can serve other incoming requests while tasks are executing.
• Use spawn_blocking if the task is CPU-bound, which uses a different, larger (512 by default) thread pool.