PangeaOS Code Guide: Walking Through the Rust & Assembly Code

How to Read This Code Guide

This page shows six key pieces of code taken directly from the PangeaOS v1.2.4b repository. You do not need to be an assembly language expert to follow along: before each code block, we explain in plain language what problem the code is solving, what would go wrong without it, and how the code works step by step.

---

1. Deciding How to Handle Errors & Calculating Hardware Key Masks (fault-core)

What Problem Does This Solve?

When something goes wrong inside an operating system, the kernel has to make an instant decision: Should I just stop the one program that made a mistake, or is the core operating system itself broken? In the code below:

• FaultClass lists the seven types of errors in PangeaOS: four local ones (L0Request, L1Domain, L2Resource, L3Subsystem) and three system-wide ones (F1CoreIntegrity, F2HardwareFatal, F3ProtectionUncertain).
• action_for is the decision function. Notice that GlobalFailStop (halting the computer) is never chosen for L0, L1, L2, or L3.
• pkru_for_domain_const calculates the exact 32-bit keycard number for the CPU's PKRU register when Service Domain domain_id (from 1 to 12) is running. It starts with all 16 keys locked (0xFFFF_FFFF), unlocks only that domain's own key (K_d = 0b00), sets the message queue key (K14) to Read-Only (0b10), and leaves the kernel (K0) and all other domains completely locked (0b11).

#![no_std]
#![forbid(unsafe_code)]

#[derive(Clone, Copy, Debug, Eq, PartialEq)]
pub enum FaultClass {
    /// L0: A service passed invalid arguments or an expired token.
    L0Request,
    /// L1: A service crashed (e.g., invalid instruction, page fault, or panic).
    L1Domain,
    /// L2: A service used up its 1 ms time slice, memory quota, or message queue.
    L2Resource,
    /// L3: A hardware device or service failed repeatedly and must be isolated.
    L3Subsystem,
    /// F1: The core kernel (Domain 0) itself faulted or a security canary was altered.
    F1CoreIntegrity,
    /// F2: Physical CPU or RAM hardware reported an uncorrectable error (#MC / #DF).
    F2HardwareFatal,
    /// F3: A CPU core or device failed to confirm an address-cache flush in time.
    F3ProtectionUncertain,
}

#[derive(Clone, Copy, Debug, Eq, PartialEq)]
pub enum RecoveryAction {
    RejectRequest,
    PreemptDomain,
    QuarantineDomain,
    Backpressure,
    EnforceResourcePolicy,
    IsolateDevice,
    RestrictedMode,
    PinResourcesAndQuarantine,
    GlobalFailStop,
}

/// Maps each fault class to its required recovery action at compile time.
pub const fn action_for(class: FaultClass) -> RecoveryAction {
    match class {
        FaultClass::L0Request => RecoveryAction::RejectRequest,
        FaultClass::L1Domain => RecoveryAction::QuarantineDomain,
        FaultClass::L2Resource => RecoveryAction::EnforceResourcePolicy,
        FaultClass::L3Subsystem => RecoveryAction::IsolateDevice,
        FaultClass::F1CoreIntegrity | FaultClass::F2HardwareFatal => {
            RecoveryAction::GlobalFailStop
        }
        FaultClass::F3ProtectionUncertain => {
            RecoveryAction::PinResourcesAndQuarantine
        }
    }
}

pub const PKRU_RW: u32 = 0b00; // Read and Write allowed
pub const PKRU_RO: u32 = 0b10; // Read-Only (Write disabled)
pub const PKRU_NA: u32 = 0b11; // No Access (Read and Write both disabled)

/// Computes the 32-bit hardware PKRU keycard mask for Service Domain `domain_id` (1..=12).
pub const fn pkru_for_domain_const(domain_id: u16) -> u32 {
    assert!(domain_id >= 1 && domain_id <= 12, "Domain ID must be between 1 and 12");
    let mut mask: u32 = 0xFFFF_FFFF; // Start with all 16 keys locked (0b11)
    // Unlock only this domain's key (2 bits per key)
    mask &= !(0b11 << (domain_id as u32 * 2));
    // Allow Read-Only access to K14 (IPC message queue counters)
    mask = (mask & !(0b11 << (14 * 2))) | (PKRU_RO << (14 * 2));
    mask
}

/// Default Kernel PKRU mask (0xCBFF_FFFC):
/// K0 (Kernel) = RW, K1..K12 (Domains) = NA, K13 (Page Tables) = RO, K14 (IPC) = RW, K15 = NA.
pub const PKRU_KERNEL_DEFAULT: u32 = 0xCBFF_FFFC;

---

2. Stopping "Bait-and-Switch" Message Attacks (pangea-ring)

What Problem Does This Solve?

When one program sends messages to another through a shared queue in memory, it updates a tail counter to show how many messages are waiting. If the receiving program reads tail from shared memory to check that it is within bounds, and then reads tail a second time when copying the message, a malicious sender on another CPU core could change tail in the microsecond between those two reads. That trick is called a Double-Fetch attack.

The checked_pending function below prevents this by taking head_snapshot and tail_snapshot—values that were read from memory just once into private CPU registers. It subtracts head from tail using unsigned wrap-around arithmetic (wrapping_sub). If the result is larger than the queue's capacity, it returns an error (Err(RingError::CapacityExceeded)) without crashing.

#[derive(Clone, Copy, Debug, Eq, PartialEq)]
pub enum RingError {
    InvalidCapacity,
    CapacityExceeded,
    InvalidDescriptorLength,
    StaleGeneration,
}

/// Checks how many messages are waiting in a queue using single-read register snapshots.
pub const fn checked_pending(
    head_snapshot: u32,
    tail_snapshot: u32,
    capacity: u32,
) -> Result< u32, RingError > {
    if capacity == 0 {
        return Err(RingError::InvalidCapacity);
    }
    let pending = tail_snapshot.wrapping_sub(head_snapshot);
    if pending > capacity {
        return Err(RingError::CapacityExceeded);
    }
    Ok(pending)
}

---

3. Expiring Old Permission Tickets Automatically (cap-core)

What Problem Does This Solve?

Suppose Service Domain 2 is given a ticket (CapabilityHandle) allowing it to read a shared buffer. Five seconds later, Domain 2 crashes and is restarted from a clean image. What if a leftover copy of that old ticket is still sitting in a message queue? To make sure old tickets cannot be reused after a crash, every ticket has a generation number (like an edition number on a hotel keycard). When Domain 2 crashes, the kernel increments the generation number in the master CapabilityEntry table. When validate_capability checks the old ticket, it sees that entry.generation != handle.generation and immediately rejects it with Err(CapabilityError::StaleGeneration).

#[derive(Clone, Copy, Debug, Eq, PartialEq)]
pub struct CapabilityHandle {
    pub slot: u32,
    pub generation: u32,
}

#[derive(Clone, Copy, Debug, Eq, PartialEq)]
pub enum CapabilityError {
    OutOfRange,
    StaleGeneration,
    WrongOwner,
    RightsDenied,
    Revoked,
}

pub fn validate_capability(
    handle: CapabilityHandle,
    current_domain: u16,
    requested: Rights,
    table: &[CapabilityEntry],
) -> Result< (), CapabilityError > {
    let entry = table.get(handle.slot as usize).ok_or(CapabilityError::OutOfRange)?;
    if !entry.live { return Err(CapabilityError::Revoked); }
    if entry.generation != handle.generation { return Err(CapabilityError::StaleGeneration); }
    if entry.owner_domain != current_domain { return Err(CapabilityError::WrongOwner); }
    if !entry.rights.contains(requested) { return Err(CapabilityError::RightsDenied); }
    Ok(())
}

---

4. Wiring the Emergency Rescue Stacks to the CPU (pangea-irq v1.2.4b)

What Problem Does This Solve?

For the processor hardware to automatically switch to a clean emergency rescue stack (IST 1) when a service crashes, the operating system must give the processor a 104-byte table called the 64-Bit Task State Segment (TSS).

There is a famous pitfall when writing this table in C or Rust: the very first field in the hardware table (reserved0) is 4 bytes long, which means all the 8-byte stack pointers that follow it (rsp0..2 and ist1..7) sit at addresses that are not multiples of 8. Normally, the Rust compiler automatically inserts 4 invisible padding bytes after reserved0 to align the 8-byte fields—which shifts the entire table by 4 bytes and causes the CPU to crash on the first interrupt!

In the code below, #[repr(C, packed)] forbids the compiler from inserting any hidden padding, and the const _: () block forces the compiler to prove at build time that TaskStateSegment64 is exactly 104 bytes and ist1 sits at exact byte offset 0x24 (36).

use core::mem::{offset_of, size_of};

#[repr(C, packed)]
#[derive(Clone, Copy, Debug)]
pub struct TaskStateSegment64 {
    pub reserved0: u32,  // Offset +0x00 (4 bytes)
    pub rsp0: u64,       // Offset +0x04 (8 bytes, unaligned)
    pub rsp1: u64,       // Offset +0x0C
    pub rsp2: u64,       // Offset +0x14
    pub reserved1: u64,  // Offset +0x1C
    pub ist1: u64,       // Offset +0x24 (Emergency Stack 1: General Exceptions & IRQs)
    pub ist2: u64,       // Offset +0x2C (Emergency Stack 2: Double Faults)
    pub ist3: u64,       // Offset +0x34 (Emergency Stack 3: NMI Watchdog)
    pub ist4: u64,       // Offset +0x3C (Emergency Stack 4: Hardware Machine Checks)
    pub ist5: u64,       // Offset +0x44
    pub ist6: u64,       // Offset +0x4C
    pub ist7: u64,       // Offset +0x54
    pub reserved2: u64,  // Offset +0x5C
    pub reserved3: u16,  // Offset +0x64
    pub iomap_base: u16, // Offset +0x66 (Set to 104)
}

// Compile-time proof that our struct matches the x86-64 hardware layout down to the byte:
const _: () = {
    assert!(size_of::< TaskStateSegment64 >() == 104);
    assert!(offset_of!(TaskStateSegment64, ist1) == 0x24);
    assert!(offset_of!(TaskStateSegment64, ist2) == 0x2C);
    assert!(offset_of!(TaskStateSegment64, ist3) == 0x34);
    assert!(offset_of!(TaskStateSegment64, ist4) == 0x3C);
};

---

5. Running a Service and Surviving a Live Crash (gate-asm v1.2.4b)

What Problem Does This Solve?

When the kernel scheduler wants to run a function inside Service Domain 5, it calls invoke_domain_live. Here is what happens behind the scenes:

1. The assembly gate (gate_enter_domain_live) saves a bookmark of the scheduler's current stack pointer and registers into the kernel's protected GS memory (GS:[0x28..0x60]).
2. It switches the stack pointer to Domain 5's stack, locks the hardware keycard (PKRU) so only Key 5 is writable, and calls Domain 5's function.
3. If Domain 5 crashes (for example, by executing the invalid ud2 instruction in our live test), the processor jumps to IST 1, unlocks the kernel's master keycard (0xCBFF_FFFC), records that Domain 5 faulted, and jumps straight back to the saved scheduler bookmark—returning 2 (ABORT_REASON_DOMAIN_EXCEPTION) from invoke_domain_live!

/// Runs `domain_fn` inside isolated `domain_id` with its own stack and hardware PKRU keycard.
/// Returns `0` if the function finishes normally, or `2` if it crashed and was safely recovered!
pub unsafe fn invoke_domain_live(
    domain_id: u16,
    generation: u32,
    domain_fn: extern "sysv64" fn(),
    domain_stack_top: u64,
    domain_pkru: u32,
) -> u64 {
    let result_code: u64;
    unsafe {
        core::arch::asm!(
            "call gate_enter_domain_live",
            in("di") domain_id,
            in("esi") generation,
            in("rdx") domain_fn as usize as u64,
            in("rcx") domain_stack_top,
            in("r8d") domain_pkru,
            lateout("rax") result_code,
            lateout("rdi") _, lateout("rsi") _, lateout("rdx") _,
            lateout("rcx") _, lateout("r8") _,  lateout("r9") _,
            lateout("r10") _, lateout("r11") _,
        );
    }
    result_code
}

---

6. Inspecting Motherboard Hardware Tables Safely (hal-core v1.2.4b)

What Problem Does This Solve?

At boot, the computer's motherboard provides configuration tables (called ACPI tables) that tell the operating system where the interrupt controllers and DMA hardware are located. Many operating systems include a complex interpreter for AML bytecode inside ACPI—which has historically been a frequent source of bugs and security holes.

PangeaOS never runs AML bytecode. Instead, verify_acpi_sdt_slice checks three simple rules on static data tables:

1. The table must be at least 36 bytes (the standard header size) and no larger than 64 Kilobytes.
2. Its 4-letter name must be on our strict allowlist (APIC, DMAR, IVRS, or MCFG). AML tables like DSDT are rejected immediately.
3. Adding up every byte in the table (modulo 256) must equal 0, proving the table has not been corrupted in memory.

pub const MAX_ACPI_TABLE_BYTES: usize = 65_536;
pub const ACPI_SDT_HEADER_SIZE: usize = 36;

pub fn verify_acpi_sdt_slice(table_bytes: &[u8]) -> Result< VerifiedAcpiHeader, AcpiVerifyError > {
    if table_bytes.len() < ACPI_SDT_HEADER_SIZE {
        return Err(AcpiVerifyError::BufferTooSmall(table_bytes.len()));
    }
    if table_bytes.len() > MAX_ACPI_TABLE_BYTES {
        return Err(AcpiVerifyError::TableTooLarge(table_bytes.len()));
    }
    let sig: [u8; 4] = [table_bytes[0], table_bytes[1], table_bytes[2], table_bytes[3]];
    let kind = match &sig {
        b"APIC" => AllowedAcpiTable::MadtApic,
        b"DMAR" => AllowedAcpiTable::DmarVtd,
        b"IVRS" => AllowedAcpiTable::IvrsAmdVi,
        b"MCFG" => AllowedAcpiTable::McfgPcie,
        _ => return Err(AcpiVerifyError::ForbiddenOrUnrecognizedSignature(sig)),
    };
    let mut sum: u8 = 0;
    for &b in table_bytes {
        sum = sum.wrapping_add(b);
    }
    if sum != 0 {
        return Err(AcpiVerifyError::InvalidChecksum(sum));
    }
    Ok(VerifiedAcpiHeader {
        kind,
        length: u32::from_le_bytes([table_bytes[4], table_bytes[5], table_bytes[6], table_bytes[7]]),
        revision: table_bytes[8],
        oem_revision: u32::from_le_bytes([table_bytes[24], table_bytes[25], table_bytes[26], table_bytes[27]]),
    })
}