Skip to main content

Module pump

Module pump 

Source
Expand description

WorkerPump<T> — safe main/worker thread coordination for FFI bodies.

§Overview

WorkerPump manages the common pattern of running a CPU-bound worker on a background thread while the main R thread drives a “pump” loop — processing progress events, rendering output, or any other operation that must happen on R’s main thread (e.g. calls into R’s C API).

It wraps std::thread::scope so the worker is always joined before run returns, and exposes a builder API for common knobs (channel capacity, log-drain cadence).

§R exits in the pump

Call WorkerPump::run from an #[miniextendr] body, on R’s main thread, and have the pump call R only through the APIs that turn an R exit into a Rust unwind: RCall::eval_with_handlers / RCall::try_eval_with_handlers (and Quoted::eval / Quosure::eval_tidy, which use them).

When R leaves such a call early (an error, a tryCatch() exit, an interrupt while R runs), it unwinds as a Rust panic carrying the R exit. The unwind runs through run: the receiver drops, so the worker’s next send returns Err, and std::thread::scope joins the worker before the unwind leaves it. The #[miniextendr] boundary then continues the R exit, so the condition reaches the function’s caller.

R must never longjmp over run instead: a bare Rf_eval / Rf_error in the pump jumps past run’s frames without running their destructors, so the worker, which borrows from them, is never joined.

The worker should stop when send returns Err: one that keeps computing delays the R exit until it returns.

§Error type

WorkerPump::run uses Result<R, Box<dyn Error + Send + Sync>> so it composes naturally with both anyhow::Result (via ?) and std::io::Error without requiring a hard dependency on any error-handling crate.

§Example

ⓘ
use miniextendr_api::pump::WorkerPump;
use std::sync::mpsc::SyncSender;

#[miniextendr]
fn compress_files(paths: Vec<String>) -> i64 {
    WorkerPump::new()
        .run(
            // worker: runs off-main-thread
            |tx: SyncSender<u64>| -> Result<i64, Box<dyn std::error::Error + Send + Sync>> {
                let mut total = 0i64;
                for path in &paths {
                    let bytes = compress_one(path)?;
                    tx.send(bytes).ok();
                    total += bytes as i64;
                }
                Ok(total)
            },
            // pump: runs on main thread, may call R API
            |bytes| render_progress(bytes),
        )
        .expect("compression failed")
}

Re-exports§

use std::error::Error; 🔒
use std::marker::PhantomData; 🔒
use std::sync::mpsc; 🔒
use std::sync::mpsc::SyncSender; 🔒
use std::thread; 🔒

Structs§

WorkerPump
Runs a worker thread in parallel with a main-thread pump loop.

Type Aliases§

WorkerError
Boxed, thread-safe error type used by WorkerPump::run.