# ESP32 Performance Optimizations for rlsw This document describes the ESP32-specific optimizations applied to the `rlsw` software renderer. All changes are guarded by platform-detection macros and have zero impact on non-ESP targets. ## Results summary Measured on ESP32-S3 @ 240 MHz, 240x240 R5G6B5 display, rendering a textured 3D model with depth testing: | Metric | Before | After | Improvement | |--------|--------|-------|-------------| | Frame time | ~65 ms | ~47 ms | **-28%** | | FPS | ~15 | ~21 | **+40%** | | `swap` phase | 15 ms | 0.03 ms | -99.8% | | `model` phase | 44 ms | 39 ms | -11% | --- ## 1. Async SPI Double-Buffer **Files:** `rlsw.h`, `raylib_esp32_port.h`, `raylib_esp32_port.c`, `rcore_esp32.c` ### Problem `SwapScreenBuffer()` called `raylib_port_flush_rgb565_vflip()` which blocked for ~15 ms doing chunked byte-swap + DMA transfer to the display. During this time the render core sat idle. ### Solution Double-buffer the color framebuffer and run the display transfer on a dedicated FreeRTOS task pinned to core 0, overlapping it with rendering on core 1. **rlsw.h changes:** - `sw_default_framebuffer_t` now holds `sw_texture_t color[2]` + `int colorIndex` - `sw_default_framebuffer_alloc/free` manage both buffers - New public function `swSwapColorBuffer()` toggles the render target and returns a pointer to the just-rendered buffer **Port layer changes:** - `raylib_port_flush_async(src, w, h)` — stores buffer pointer, notifies flush task, returns immediately - `raylib_port_wait_flush()` — blocks until the previous transfer completes - Flush task: persistent FreeRTOS task on core 0 (4 KB stack), sleeps via `ulTaskNotifyTake`, runs vflip + byte-swap + DMA, signals completion **SwapScreenBuffer flow:** ``` wait_flush() // ensure previous transfer finished rendered = swSwapColorBuffer() // swap render target, get finished buffer flush_async(rendered) // kick off transfer on core 0 ``` **Memory cost:** ~115 KB PSRAM (second color buffer) + 4 KB task stack. Since render time (~42 ms) > transfer time (~15 ms), the wait never blocks in steady state. Effective swap cost drops to ~30 µs. --- ## 2. Fast Reciprocal (`recip0.s`) **File:** `rlsw.h` **Guard:** `#if defined(__XTENSA__)` ### Problem Float division on Xtensa uses a software `__divsf3` call (~20+ cycles). The rasterizer performs multiple `1.0f/x` operations per triangle setup, per 16-pixel affine block, and per vertex transform. ### Solution `sw_rcp(float x)` uses the Xtensa `recip0.s` instruction with a Newton-Raphson refinement sequence, achieving 1-ULP accuracy in ~7 instructions (all in FPU registers, no function call overhead): ```c #if defined(__XTENSA__) __attribute__((always_inline)) static inline float sw_rcp(float x) { float result, temp; __asm__( "recip0.s %0, %2\n" "const.s %1, 1\n" "msub.s %1, %2, %0\n" "madd.s %0, %0, %1\n" "const.s %1, 1\n" "msub.s %1, %2, %0\n" "maddn.s %0, %0, %1\n" : "=&f"(result), "=&f"(temp) : "f"(x) ); return result; } #else static inline float sw_rcp(float x) { return 1.0f / x; } #endif ``` Applied to all hot-path divisions: - **Span function:** `dxRcp`, `blockLenRcp`, `wRcpA`, `wRcpB` (per span + per 16px block) - **Triangle setup:** `h02Rcp`, `h01Rcp`, `h12Rcp` (per triangle) - **Vertex transform:** `1/w` for clip-to-NDC conversion (per vertex) - **Quad rasterizer:** `wRcp`, `hRcp` - **Line rasterizer:** `stepRcp` On non-Xtensa targets (RISC-V ESP32-C3/C6, desktop) the function transparently falls back to `1.0f / x`. --- ## 3. Direct R5G6B5 Pixel Read/Write **File:** `rlsw.h` **Guard:** `#if defined(ESP_PLATFORM)` ### Problem The default R5G6B5 write path goes through three stages: ``` float[4] → sw_color_to_color8 → uint8[4] → sw_pixel_write_color8_R5G6B5 → uint16 ``` This involves 4 float→int conversions (including a wasted alpha channel), expand/compress LUT operations, and an intermediate byte array. ### Solution **Write** (`sw_pixel_write_color_R5G6B5`): converts float RGB directly to packed R5G6B5 in one step — 3 multiplies + 3 float→int + bit packing: ```c uint32_t r = (uint32_t)(color[0] * 31.0f + 0.5f); uint32_t g = (uint32_t)(color[1] * 63.0f + 0.5f); uint32_t b = (uint32_t)(color[2] * 31.0f + 0.5f); ((uint16_t *)pixels)[offset] = (uint16_t)((r << 11) | (g << 5) | b); ``` **Read** (`sw_pixel_read_color_R5G6B5`): unpacks R5G6B5 directly to float RGB, skipping the expand-to-uint8 intermediate: ```c uint16_t pixel = ((const uint16_t *)pixels)[offset]; color[0] = (float)((pixel >> 11) & 0x1F) * (1.0f/31.0f); color[1] = (float)((pixel >> 5) & 0x3F) * (1.0f/63.0f); color[2] = (float)( pixel & 0x1F) * (1.0f/31.0f); color[3] = 1.0f; ``` --- ## 4. Optimized Texture Sampling **File:** `rlsw.h` **Guard:** `#if defined(ESP_PLATFORM)` ### Problem `sw_texture_sample` computes LOD (level-of-detail) every pixel — 6 float ops (4 multiplies + 2 adds + compare) — even when `minFilter == magFilter`, making the LOD decision meaningless. Additionally, the texel read goes through a function pointer (`tex->readColor`), preventing inlining. ### Solution **Skip LOD when filters match** (the common case): ```c if (tex->minFilter == tex->magFilter) { if (tex->minFilter == SW_NEAREST) sw_texture_sample_nearest_inline(color, tex, u, v); else sw_texture_sample_linear(color, tex, u, v); return; } ``` **Inline nearest sampling** for common texture formats. A new `sw_texture_sample_nearest_inline` function inlines the texel read for R8G8B8A8 and R5G6B5 textures, avoiding the function pointer indirection and allowing the compiler to schedule memory loads: ```c switch (tex->format) { case SW_PIXELFORMAT_COLOR_R8G8B8A8: { /* direct byte read */ } case SW_PIXELFORMAT_COLOR_R5G6B5: { /* direct uint16 unpack */ } default: tex->readColor(color, tex->pixels, offset); } ``` --- ## 5. Framebuffer Clear Optimization **File:** `rlsw.h` **Guard:** `#if SW_COLOR_IS_R5G6B5` / `#if SW_DEPTH_IS_D16` ### Problem The default clear loop fills the framebuffer byte-by-byte: ```c for (int i = 0; i < size; i++) for (int b = 0; b < PIXEL_SIZE; b++) dst[i * PIXEL_SIZE + b] = pixel[b]; ``` For a 240x240 R5G6B5 buffer this means 115,200 individual byte stores. ### Solution For R5G6B5 and D16 formats (both 2 bytes per pixel): 1. **memset fast path**: when both bytes of the packed pixel are equal (e.g. black = `0x0000`, depth 1.0 = `0xFFFF`), use `memset` — a single call that the runtime can optimize with DMA or wide stores. 2. **32-bit word fill**: pack two pixels into a `uint32_t` and fill with aligned 32-bit writes, halving the number of store operations. --- ## 6. Compile-Time Format Detection **File:** `rlsw.h` Two token-pasting macros detect the framebuffer format at compile time, enabling `#if` guards with zero runtime cost: ```c #define SW_DEPTH_IS_D16 /* 1 when SW_FRAMEBUFFER_DEPTH_TYPE == D16 */ #define SW_COLOR_IS_R5G6B5 /* 1 when SW_FRAMEBUFFER_COLOR_TYPE == R5G6B5 */ ``` These gate the fixed-point depth test (from a prior optimization), the direct pixel read/write paths, and the clear optimizations. --- ## 7. Fixed-Point Depth Test **File:** `rlsw.h` **Guard:** `#if SW_DEPTH_IS_D16` The inner-loop depth test converts `float z` to `uint16_t` every pixel via `(uint16_t)(z * 65535.0f)` — an expensive float→int conversion. When the depth format is D16, the depth value and step are pre-converted to `int32_t` before the pixel loop: ```c int32_t z_fp = (int32_t)(z * 65535.0f); int32_t dZdx_fp = (int32_t)(dZdx * 65535.0f); ``` The inner loop then uses integer compare and add: ```c uint16_t z16 = (uint16_t)z_fp; if (z16 > depth) goto discard; ((uint16_t *)dPtr)[0] = z16; // ... z_fp += dZdx_fp; ``` --- ## 8. ESP-DSP Matrix Acceleration **File:** `rlsw.h` **Guard:** `#ifdef SW_HAS_ESP_DSP` When the ESP-DSP library is available, optimized matrix operations replace the scalar implementations: - `sw_matrix_mul_rst` → `dspm_mult_4x4x4_f32` (4×4 matrix multiply) - Vertex MVP transform → `dspm_mult_4x4x1_f32` (4×4 × 4×1 multiply) A row-major transposed MVP copy (`matMVP_rm`) is maintained for ESP-DSP compatibility since `rlsw` uses column-major layout internally. --- ## 9. SRAM-Aware Framebuffer Allocation **File:** `rlsw.h`, `raylib_esp32_config.h` **Guard:** `#ifdef CONFIG_IDF_TARGET_ESP32S3` Custom allocator hooks (`SW_FB_REALLOC`, `SW_FB_FREE`) allow framebuffer memory to be placed in specific memory regions. On ESP32-S3: - `_sw_fb_alloc_prefer_internal`: tries `MALLOC_CAP_INTERNAL` first, falls back to `MALLOC_CAP_SPIRAM` - The depth buffer is allocated first (it benefits most from fast SRAM since it is accessed for every pixel attempt, including occluded ones) --- ## 10. Fixed-Point Color Interpolation **File:** `rlsw.h` **Guard:** `#if SW_COLOR_IS_R5G6B5 && !defined(SW_ENABLE_TEXTURE) && !defined(SW_ENABLE_BLEND)` ### Problem In the triangle span inner loop, vertex colors were interpolated as `float[4]`, then converted per pixel via `color[i] * 31.0f + 0.5f` → `(uint32_t)` → pack to R5G6B5. That is 3 float multiplies + 3 float→int conversions + 4 float adds (stepping) per pixel. ### Solution At each affine block boundary, convert `srcColor` and `dSrcColordx` to 16.16 fixed-point pre-scaled to R5G6B5 channel ranges (×31, ×63, ×31): ```c int32_t r_fp = (int32_t)(srcColor[0] * (31.0f * 65536.0f)); int32_t dr_fp = (int32_t)(dSrcColordx[0] * (31.0f * 65536.0f)); ``` The inner pixel loop becomes pure integer: `>> 16`, clamp, shift-OR pack, and `r_fp += dr_fp` stepping. No floats touched per pixel. Applies to non-textured, non-blend template variants (BASE, DEPTH). Textured paths still use float for the `srcColor × texColor` multiply. --- ## 11. POT Texture Bit-Masking **File:** `rlsw.h` **Guard:** `#if defined(ESP_PLATFORM)` (in both `sw_texture_sample_nearest` and `sw_texture_sample_nearest_inline`) ### Problem For `SW_REPEAT` wrap mode, `sw_fract(u)` calls `floorf` to get the fractional part. While `floorf` maps to `floor.s` on Xtensa FPU, the full wrap sequence (`sw_fract` + multiply + cast + clamp) is 6 instructions per coordinate. ### Solution For power-of-two textures (`width & wMinus1 == 0`), replace the entire wrap chain with a single bitmask: ```c x = (int)(u * tex->width) & tex->wMinus1; ``` This handles both positive and negative UVs through two's complement wrapping (off by at most 1 texel for rare negative UVs — imperceptible at 240×240). Non-POT textures fall through to the original `sw_fract` path. Saves ~2 instructions per texel per coordinate on the nearest-filter hot path.