Module externalptr

Expand description

ExternalPtr<T> — a Box-like owned pointer that wraps R’s EXTPTRSXP.

This provides ownership semantics similar to Box<T>, with the key difference that cleanup is deferred to R’s garbage collector via finalizers.

§Submodules

Module	Contents
`altrep_helpers`	ALTREP data1/data2 slot access helpers + the `RSidecar` and `Computed` marker types
`sidecar`	`Sidecar<T>` fields: typed values kept in the `prot` list

§Core Types

ExternalPtr<T> — owned pointer wrapping EXTPTRSXP
TypedExternal — display and diagnostic metadata for stored types
ExternalSlice<T> — helper for slice data in external pointers
ErasedExternalPtr — type-erased ExternalPtr<()> alias
IntoExternalPtr — conversion trait for wrapping values

PartialEq/PartialOrd compare the pointee values (like Box<T>), borrowing both. Use ptr_eq when you care about pointer identity.

§Protection Strategies in miniextendr

miniextendr provides three complementary protection mechanisms for different scenarios:

Strategy	Module	Lifetime	Release Order	Use Case
PROTECT stack	`gc_protect`	Within `.Call`	LIFO (stack)	Temporary allocations
VECSXP pool	`protect_pool`	Until released or the pool drops	Any order	Values a struct owns; many values; cross-call roots
R ownership	`ExternalPtr`	Until R GCs	R decides	Rust data owned by R

§When to Use ExternalPtr

Use ExternalPtr (this module) when:

You want R to own a Rust value
The Rust value should be dropped when R garbage collects the pointer
You’re exposing Rust structs to R code

Use gc_protect instead when:

You’re allocating temporary R objects during computation
Protection is short-lived (within a single .Call)

Use ProtectPool instead when:

You need R objects (not Rust values) kept alive by a struct, or across .Calls
You need arbitrary-order release of protections

§How ExternalPtr Protection Works

┌─────────────────────────────────────────────────────────────────┐
│  ExternalPtr<MyStruct>::new(value)                              │
│  ├── Rf_protect() during construction (temporary)               │
│  ├── R_MakeExternalPtr() creates EXTPTRSXP                      │
│  ├── R_RegisterCFinalizerEx() registers cleanup callback        │
│  ├── pool.insert() roots it for the Rust handle's lifetime       │
│  └── Rf_unprotect() after construction complete                 │
│                                                                 │
│  Held in Rust (even across other R allocations, e.g. in a Vec)  │
│  └── stays alive — the pool's GC-traced VECSXP slot roots it     │
│                                                                 │
│  Return to R → R now also references the EXTPTRSXP              │
│  └── Rust handle drops → pool.release(key) drops the root,      │
│      but R's own reference keeps it live                        │
│                                                                 │
│  R GC runs (no refs left) → finalizer (release_any) frees value │
└─────────────────────────────────────────────────────────────────┘

Owning handles (new / from_raw / Clone) root their EXTPTRSXP in a process-wide ProtectPool so they survive R allocations while held in Rust; borrowed views (wrap_sexp / from_sexp / same_object) take no root — the object is kept alive by whatever R-side reference handed it to them. The pool (O(1) any-order release) is used rather than R_PreserveObject because a Vec<ExternalPtr> releases its roots front-to-back, the O(n²) worst case for R_ReleaseObject’s precious-list scan — see analysis/gc-protection-benchmarks-results.md.

When the end goal is an R list() of external pointers (rather than a Vec<ExternalPtr> you keep working with in Rust), prefer ExternalPtr::collect_into_r_list — it builds each EXTPTRSXP straight into the protected result list, so the list roots every element and the pool is never touched at all.

§Type Identification

Type safety is enforced via Any::downcast (Rust’s TypeId). R symbols in the tag and prot slots are retained for display and error messages but are never authoritative for downcast safety — the Any vtable is.

Internally, data is stored as Box<Box<dyn Any>> — a thin pointer (fits in R’s R_ExternalPtrAddr) pointing to a fat pointer (carries the Any vtable for runtime downcasting). The Any value is a RefCell<T>, whose borrow flag is what generated method wrappers, field accessors and trait shims check before they make a &T / &mut T (see “Reentrant calls” below). The outer Box keeps the heap address stable so ExternalPtr::cached_ptr can be cached once at construction.

The tag slot holds a symbol (type name, for display). The prot slot holds a VECSXP (list) of 2 + T::R_SLOT_COUNT elements:

Index 0: SYMSXP (interned type ID symbol, for error messages)
Index 1: User-protected SEXP slot (for preventing GC of R objects)
Index 2..: one value per Sidecar<T> field of T

Before reading the address as Rust storage, checked conversions and erased handle accessors require this metadata shape (including a non-ALTREP list). Trait-ABI pointers carry a reserved tag and a different allocation layout; they are rejected here. These checks reject ordinary foreign pointers, not foreign addresses deliberately forged with miniextendr metadata. Symbol names never establish the concrete Rust type; Any::downcast still does.

§`TYPE_NAME_CSTR` vs `TYPE_ID_CSTR`

TypedExternal exposes two associated constants with distinct roles — mixing them up does not break type safety (Any::downcast is the real gate) but produces noisy diagnostics.

Constant	Role	Visible to R as	Authoritative?
`TYPE_NAME_CSTR`	Display tag	`class()` / `print()`	No
`TYPE_ID_CSTR`	Error-message identifier on downcast failure	Stored in `prot[0]`	No (cosmetic; downcast uses `TypeId`)

#[derive(ExternalPtr)] fills both with sensible defaults; only override manually when implementing TypedExternal by hand.

§Pointer provenance for `cached_ptr`

ExternalPtr caches the address of the value’s RefCell<T> at construction so borrow / borrow_mut avoid an FFI call on every access. A RefCell hands out &mut T from a shared reference to the cell, so a pointer cached from downcast_ref::<RefCell<T>>() is as good for writes as one from Box::into_raw.

§Reentrant calls

R reaches a Rust value only through a generated wrapper, on R’s main thread, and the wrapper borrows the value’s RefCell for the whole call: shared for &self methods, &T arguments, ExternalPtrRef receivers and arguments, and field getters; exclusive for &mut self and self methods, &mut T arguments, ExternalPtrMut receivers and arguments, and field setters. The body gets plain references, which Rust’s borrow checker covers from there. A second borrow during the call, from the same object passed twice with a &mut T, or from R code the body calls back into (an R function argument, RCall) reaching the object again, raises an R error of class miniextendr_borrow_conflict instead of making a second reference. A borrow taken through a trait view lives in the producer’s shim, so views in other packages see it too.

An ExternalPtr<T> handle, as an argument, a self: ExternalPtr<Self> receiver, a Vec element or a value Rust keeps, takes no borrow until it is read: borrow and borrow_mut check the RefCell and raise the same error. A handle has no Deref, so every read through it is one of these. into_inner and the other consuming functions refuse a value something still borrows, with the same error.

The borrow is released when the wrapper returns, panics, or unwinds an R error raised through RCall::eval_with_handlers (or any API that turns an R error into a Rust unwind). A bare Rf_eval that raises an R error jumps over the wrapper’s frames, so the object stays borrowed for the rest of the session.

§See also

crate::altrep — when the alternative (an ALTREP class) makes more sense than ExternalPtr.

§ExternalPtr is Not an R Native Type

Unlike R’s native atomic types (integer, double, character, etc.), external pointers cannot be coerced to vectors or used in R’s vectorized operations. This is an R limitation, not a miniextendr limitation:

> matrix(new("externalptr"), 1, 1)
Error in `as.vector()`:
! cannot coerce type 'externalptr' to vector of type 'any'

If you need your Rust type to participate in R’s vector/matrix operations, consider implementing IntoList (via #[derive(IntoList)]) to convert your struct to a named R list, or use ALTREP to expose Rust iterators as lazy R vectors.

Re-exports§

pub use borrowed::ExternalPtrMut;
pub use borrowed::ExternalPtrRef;
pub use borrowed::Saved;
pub use borrowed::SidecarOwner;
pub use borrowed::SidecarOwnerMut;
pub use borrowed::SidecarRead;
pub use altrep_helpers::*;
pub use sidecar::*;

Modules§

altrep_helpers 🔒: ALTREP helpers for ExternalPtr — data1/data2 slot access.
borrowed 🔒: Borrowed handles over ordinary or erased external-pointer storage.
sidecar 🔒: Typed sidecar fields: Sidecar<T> keeps a Rust-typed value in the external pointer’s prot list.

Structs§

AbortIfUnwinding 🔒: Guard that aborts the process if a panic starts while it is alive.
ConsumedSlot: Marker left in an EXTPTRSXP slot while a consuming (self by value) method runs, and left behind for good if that method panics.
ExternalPtr: An owned pointer stored in R’s external pointer SEXP.
ExternalSlice: A slice stored as a standalone struct, suitable for wrapping in ExternalPtr.
PoolRoot 🔒: One R object rooted in the main-thread pool, released when dropped.
ReceiverOwner 👻: A generated method’s receiver: the EXTPTRSXP resolve_receiver finds in the .Call argument, rooted for the call when a class handle carried it. R code the method calls back into can drop the handle’s reference, and the finalizer would then free the value under the method’s borrow. A bare pointer is the .Call argument itself, which R keeps alive.

Enums§

NullAddress 🔒: What a pointer without an address is, as far as T is concerned (classify_null_address).
TypeMismatchError: Error returned when type checking fails in try_from_sexp_with_error.

Constants§

EXTPTR_ROOTS 🔒
NO_HANDLE_CLASS: The class of the error for a receiver of a type’s R accessors that carries no external pointer: neither a pointer nor an object holding one in .ptr, such as an object of the class an earlier release of a package wrote to disk (#1901). The type’s restored and field-error classes go in front of it.
PROT_TYPE_ID_INDEX 🔒: Index of the type SYMSXP contained in the prot (a VECSXP list)
PROT_USER_INDEX 🔒: Index of user-protected objects contained in the prot (a VECSXP list)
PROT_VEC_LEN 🔒: Length of the prot list (VECSXP) before the Sidecar<T> entries, which start at this index
REBUILDING 🔒: The pointers whose rebuild hook is running on this thread: a hook that reaches its own object again finds it refused, not rebuilt twice.
RESTORED_CLASS: The condition class every error about a pointer restored from a saved session carries, after its specific class.
RESTORED_MISSING_SLOT_CLASS: The class of the error a rebuild hook gets when it reads a Sidecar field the save has no slot for: a field added after the version that saved the object (#2187). The condition carries field, the field’s name. The try_<field> getters return None instead.
RESTORED_NO_VALUE_CLASS: The class of the error for a pointer this version of the crate saved: it came back from readRDS() / unserialize() without its Rust value.
RESTORED_OTHER_VERSION_CLASS: The class of the error for a pointer another version of the crate saved; the condition carries saved_version and current_version.

Traits§

ConsumingFallible: Bound for the receiver of a fallible consuming method (self -> Result<Self, E> / self -> Option<Self>).
IntoExternalPtr: Marker trait for types that should be converted to R as ExternalPtr.
TypedExternal: Trait for types that can be stored in an ExternalPtr.

Functions§

__mx_borrow 👻: ExternalPtr::borrow for a generated wrapper’s argument.
__mx_borrow_mut 👻: ExternalPtr::borrow_mut for a generated wrapper’s argument.
__mx_exclusive 👻: The &mut T a generated wrapper hands a &mut T parameter.
__mx_shared 👻: The &T a generated wrapper hands a &T parameter.
assert_sidecar_clone_on_main_thread 🔒: ExternalPtr<T> is Send, and copying Sidecar slots is R API.
borrow_conflict 🔒
borrow_value 👻: Borrow a stored value shared for a generated wrapper: the R error of class miniextendr_borrow_conflict while a running call holds it mutably.
borrow_value_mut 👻: Borrow a stored value exclusively for a generated wrapper: the R error of class miniextendr_borrow_conflict while a running call holds it.
classify_null_address 🔒 ^⚠: Reads prot[0] of a pointer without an address and says what it is for T. Symbol names live for the session, so the borrowed parts are 'static.
clear_moved_out 🔒 ^⚠: Empties the pointer a value was moved out of (into_raw / into_inner): the address, so the finalizer is a no-op, and the tag and protection list too, so the pointer reads as empty (the plain null external pointer messages) rather than as one restored from a saved session, which keeps its prot (#1872). R_ClearExternalPtr only nulls the address.
clone_for_consuming: Clone the stored value for a fallible consuming step (see ConsumingFallible). Free function so codegen can name the bound.
drop_catching_panic 👻: Run a destructor closure, aborting the process if the closure panics.
env_binding 🔒 ^⚠: Look up a variable bound directly in a single environment frame (no search of enclosing frames) with REnv::get.
handle_downcast_failed: Panic with the right message when a handle’s stored value is not a T: “consumed” if a previous self-by-value step failed; the classed RESTORED_NO_VALUE_CLASS / RESTORED_OTHER_VERSION_CLASS error for a pointer built for T that came back from a saved session without its value; type mismatch otherwise. Used by generated method preludes instead of a bare expect.
handle_downcast_failed_in: handle_downcast_failed for a trait-impl method receiver: the same consumed / restored errors, and a mismatch message naming the trait method (Trait::method()).
install_rebuilt 🔒 ^⚠: Puts the rebuilt value in sexp, a restored pointer for a T, as construction puts a new value in a new pointer: the finalizer first (R does not save it), then the address.
is_type_erased 🔒
list_element 🔒 ^⚠: The element of a generic vector bound to name: the first match in names(x), like x[[name]]. None when x has no character names or no element by that name.
prot_vec_len 🔒: Length of T’s prot list: the type ID, the user slot and one entry per Sidecar<T> field.
rebuild_failed_message 🔒: The message of a refused rebuild: the restored refusal’s, then the cause’s.
rebuild_restored 🔒 ^⚠: Rebuilds the value of sexp, a pointer without an address, when T has a rebuild hook (#[externalptr(rebuild = path)]) and this version of its crate saved it, or another version did and T opts in with any_version (#2187) and the save carries slot names. Installs the value in sexp. Returns whether sexp holds a value now; false leaves the caller to refuse the pointer as before (another version saved it, another type, no hook, or the hook is already running for sexp).
receiver_class 🔒: The first R class of receiver, which is what the user sees and the field methods dispatched on (#1920); None for an object without a class attribute, such as an R6 object’s .ptr, whose messages name the Rust type instead.
receiver_not_a_handle 🔒
refuse_consumed_or_restored 🔒: The errors a receiver prelude raises before a type mismatch: “consumed” for a slot a failed self-by-value step left, the classed restored-object errors for a pointer built for T without an address.
refuse_restored 🔒 ^⚠: Raises the error for a pointer without an address when it was built for T (by this version: RESTORED_NO_VALUE_CLASS; by another: RESTORED_OTHER_VERSION_CLASS), with the package’s classes in front and its message when the type declares them (#[externalptr(restored(...))]), and returns the other cases for the caller to word as it did before. The default message names receiver’s R class, the object the user called on (the pointer itself, or the list, environment or S4 object carrying it), else the Rust type. The condition is raised through the panic transport (panic_any), like error!.
relayout_saved_slots 🔒 ^⚠: Replaces the saved protection list of sexp with one in T’s layout: see install_rebuilt. value’s pending values for the fields the save lacks are written; the rest stay pending, for the caller to drop.
release_any 🔒: Non-generic C finalizer called by R’s garbage collector.
resolve_receiver: Resolve an instance-method receiver to the bare EXTPTRSXP it carries.
restored_classes_of 🔒: The condition classes of a restored refusal of a T: the package’s classes (TypedExternal::RESTORED_ERROR_CLASS, in order), then the specific class, then RESTORED_CLASS.
restored_message_of 🔒: The message of a restored refusal of a T: the package’s (TypedExternal::RESTORED_ERROR_MESSAGE) when it has one, else miniextendr’s (default).
restored_no_value_message 🔒: The message of the RESTORED_NO_VALUE_CLASS error.
restored_other_version_message 🔒: The message of the RESTORED_OTHER_VERSION_CLASS error.
rewrite_saved_slots_for_tests 👻 ^⚠: Lays the Sidecar slots of a restored pointer’s save out as another version of its type might have saved them, for tests of reading a save by field name (#2187). Slot i of the new list holds the slot named layout[i] in the current save, or the string "unknown" for a name the save has no slot for. The list carries the names layout unless named is false, which gives a save made before slots were named.
rewrite_stored_version_for_tests^⚠: Test support: make a restored pointer read as a save by another version of its crate, so a package can test its handling of miniextendr_restored_other_version (#1872) without building two versions.
root_owned 🔒: Root an owning handle’s EXTPTRSXP in the main-thread pool.
sendable_any_ptr_into_ptr 🔒: Get the raw pointer, consuming the sendable wrapper.
sendable_any_ptr_new 🔒 ^⚠: Create a new sendable pointer from a raw *mut Box<dyn Any>.
split_type_id 🔒: Splits a type ID of the derive’s form, crate@version::module::Type, into (crate, version, module::Type); a hand-written ID without a version is (None, None, id).
stored_type_id_symbol 🔒 ^⚠: The type-ID symbol an ExternalPtr-built pointer keeps in prot[0], or None when its metadata is not the shape ordinary ExternalPtr builds. This also excludes trait-ABI storage, which has a different Rust layout.
symbol_name 🔒: Get the type name from a stored symbol SEXP.
type_id_str 🔒: T’s type ID as text (TYPE_ID_CSTR without its NUL).
type_id_symbol 🔒 ^⚠: Get the namespaced type ID symbol used in diagnostics.
type_id_symbol_unchecked 🔒 ^⚠: Unchecked version of type_id_symbol.
type_symbol 🔒 ^⚠: Get the interned R symbol for a type’s name.
type_symbol_unchecked 🔒 ^⚠: Unchecked version of type_symbol - no thread safety checks.
unroot_owned 🔒: Release an owning handle’s pool root. Stale keys are a safe no-op.
unwrap_class_handle 🔒 ^⚠: Attempt to unwrap a class-wrapped handle down to the bare EXTPTRSXP it carries, so ExternalPtr::<T> argument conversion accepts the ergonomic class handle (e.g. Foo$new(...)) in addition to the raw pointer returned by a low-level constructor (audit finding A9 — audit/2026-07-03-api-sense-conversions-dataframe-errors.md #5).
wrap_receiver: The typed handle of an instance-method receiver, for self: ExternalPtr<Self> receivers: resolve_receiver then ExternalPtr::wrap_sexp, failing like the erased receivers do (handle_downcast_failed: consumed, restored from a saved session, or a type mismatch). Runs on R’s main thread (the generated prelude).

Type Aliases§

ErasedExternalPtr: Type-erased ExternalPtr for cases where the concrete T is not needed.
SendableAnyPtr 🔒: Own the allocation during the handoff to R’s main thread.