Files
lilygo-t-deck/lib/raylib-esp32/docs/ESP32_OPTIMIZATIONS.md
T
Dejvino 0f22c6afca Add raylib-esp32 lib and FloorPlan3D demo (3D floor plan via rlsw)
Vendor raylib-esp32 v1.0.3 (63db94b) under lib/ — raylib 6.x core with
the rlsw software renderer plus the ESP32-S3 port layer (SPI display
drivers, async double-buffered flush). raylib/examples, projects, tools
and logo are trimmed (~80 MB of content nothing here compiles); see
lib/raylib-esp32/VENDOR.md for provenance.

New floorplan3d-demo (emulator-template base) renders the NetDeck demo
map (nd_demomap.h) as a 3D floor plan — bulkhead walls, blast door,
cover crates, airlock pad, terminal — through the real on-device 3D
stack on the desktop:

- rcore_emu.c: desktop PLATFORM_CUSTOM layer mirroring rcore_esp32.c,
  flushing the RGB565 sw-renderer buffer into emuGfx::fb instead of SPI
- raylib_emu_config.h: same module/limit defines as the device build
- HUD via 3x5 bitmap font into fb: rlsw's 2D textured-quad path
  (DrawText/DrawRectangle) misplaces/flips quads and samples wrong
  atlas texels in this build; the 3D path is pixel-accurate

Verified: dual build (make netdeck) green, demo smoke test clean at
~59 FPS, 600-frame stability run OK.
2026-11-16 21:46:31 +01:00

10 KiB
Raw Blame History

ESP32 Performance Optimizations for rlsw

This document describes the ESP32-specific optimizations applied to the rlsw software renderer. All changes are guarded by platform-detection macros and have zero impact on non-ESP targets.

Results summary

Measured on ESP32-S3 @ 240 MHz, 240x240 R5G6B5 display, rendering a textured 3D model with depth testing:

Metric Before After Improvement
Frame time ~65 ms ~47 ms -28%
FPS ~15 ~21 +40%
swap phase 15 ms 0.03 ms -99.8%
model phase 44 ms 39 ms -11%

1. Async SPI Double-Buffer

Files: rlsw.h, raylib_esp32_port.h, raylib_esp32_port.c, rcore_esp32.c

Problem

SwapScreenBuffer() called raylib_port_flush_rgb565_vflip() which blocked for ~15 ms doing chunked byte-swap + DMA transfer to the display. During this time the render core sat idle.

Solution

Double-buffer the color framebuffer and run the display transfer on a dedicated FreeRTOS task pinned to core 0, overlapping it with rendering on core 1.

rlsw.h changes:

  • sw_default_framebuffer_t now holds sw_texture_t color[2] + int colorIndex
  • sw_default_framebuffer_alloc/free manage both buffers
  • New public function swSwapColorBuffer() toggles the render target and returns a pointer to the just-rendered buffer

Port layer changes:

  • raylib_port_flush_async(src, w, h) — stores buffer pointer, notifies flush task, returns immediately
  • raylib_port_wait_flush() — blocks until the previous transfer completes
  • Flush task: persistent FreeRTOS task on core 0 (4 KB stack), sleeps via ulTaskNotifyTake, runs vflip + byte-swap + DMA, signals completion

SwapScreenBuffer flow:

wait_flush()          // ensure previous transfer finished
rendered = swSwapColorBuffer()  // swap render target, get finished buffer
flush_async(rendered) // kick off transfer on core 0

Memory cost: ~115 KB PSRAM (second color buffer) + 4 KB task stack.

Since render time (~42 ms) > transfer time (~15 ms), the wait never blocks in steady state. Effective swap cost drops to ~30 µs.


2. Fast Reciprocal (recip0.s)

File: rlsw.h
Guard: #if defined(__XTENSA__)

Problem

Float division on Xtensa uses a software __divsf3 call (~20+ cycles). The rasterizer performs multiple 1.0f/x operations per triangle setup, per 16-pixel affine block, and per vertex transform.

Solution

sw_rcp(float x) uses the Xtensa recip0.s instruction with a Newton-Raphson refinement sequence, achieving 1-ULP accuracy in ~7 instructions (all in FPU registers, no function call overhead):

#if defined(__XTENSA__)
__attribute__((always_inline))
static inline float sw_rcp(float x)
{
    float result, temp;
    __asm__(
        "recip0.s %0, %2\n"
        "const.s  %1, 1\n"
        "msub.s   %1, %2, %0\n"
        "madd.s   %0, %0, %1\n"
        "const.s  %1, 1\n"
        "msub.s   %1, %2, %0\n"
        "maddn.s  %0, %0, %1\n"
        : "=&f"(result), "=&f"(temp) : "f"(x)
    );
    return result;
}
#else
static inline float sw_rcp(float x) { return 1.0f / x; }
#endif

Applied to all hot-path divisions:

  • Span function: dxRcp, blockLenRcp, wRcpA, wRcpB (per span + per 16px block)
  • Triangle setup: h02Rcp, h01Rcp, h12Rcp (per triangle)
  • Vertex transform: 1/w for clip-to-NDC conversion (per vertex)
  • Quad rasterizer: wRcp, hRcp
  • Line rasterizer: stepRcp

On non-Xtensa targets (RISC-V ESP32-C3/C6, desktop) the function transparently falls back to 1.0f / x.


3. Direct R5G6B5 Pixel Read/Write

File: rlsw.h
Guard: #if defined(ESP_PLATFORM)

Problem

The default R5G6B5 write path goes through three stages:

float[4] → sw_color_to_color8 → uint8[4] → sw_pixel_write_color8_R5G6B5 → uint16

This involves 4 float→int conversions (including a wasted alpha channel), expand/compress LUT operations, and an intermediate byte array.

Solution

Write (sw_pixel_write_color_R5G6B5): converts float RGB directly to packed R5G6B5 in one step — 3 multiplies + 3 float→int + bit packing:

uint32_t r = (uint32_t)(color[0] * 31.0f + 0.5f);
uint32_t g = (uint32_t)(color[1] * 63.0f + 0.5f);
uint32_t b = (uint32_t)(color[2] * 31.0f + 0.5f);
((uint16_t *)pixels)[offset] = (uint16_t)((r << 11) | (g << 5) | b);

Read (sw_pixel_read_color_R5G6B5): unpacks R5G6B5 directly to float RGB, skipping the expand-to-uint8 intermediate:

uint16_t pixel = ((const uint16_t *)pixels)[offset];
color[0] = (float)((pixel >> 11) & 0x1F) * (1.0f/31.0f);
color[1] = (float)((pixel >>  5) & 0x3F) * (1.0f/63.0f);
color[2] = (float)( pixel        & 0x1F) * (1.0f/31.0f);
color[3] = 1.0f;

4. Optimized Texture Sampling

File: rlsw.h
Guard: #if defined(ESP_PLATFORM)

Problem

sw_texture_sample computes LOD (level-of-detail) every pixel — 6 float ops (4 multiplies + 2 adds + compare) — even when minFilter == magFilter, making the LOD decision meaningless. Additionally, the texel read goes through a function pointer (tex->readColor), preventing inlining.

Solution

Skip LOD when filters match (the common case):

if (tex->minFilter == tex->magFilter) {
    if (tex->minFilter == SW_NEAREST)
        sw_texture_sample_nearest_inline(color, tex, u, v);
    else
        sw_texture_sample_linear(color, tex, u, v);
    return;
}

Inline nearest sampling for common texture formats. A new sw_texture_sample_nearest_inline function inlines the texel read for R8G8B8A8 and R5G6B5 textures, avoiding the function pointer indirection and allowing the compiler to schedule memory loads:

switch (tex->format) {
    case SW_PIXELFORMAT_COLOR_R8G8B8A8: { /* direct byte read */ }
    case SW_PIXELFORMAT_COLOR_R5G6B5:  { /* direct uint16 unpack */ }
    default: tex->readColor(color, tex->pixels, offset);
}

5. Framebuffer Clear Optimization

File: rlsw.h
Guard: #if SW_COLOR_IS_R5G6B5 / #if SW_DEPTH_IS_D16

Problem

The default clear loop fills the framebuffer byte-by-byte:

for (int i = 0; i < size; i++)
    for (int b = 0; b < PIXEL_SIZE; b++)
        dst[i * PIXEL_SIZE + b] = pixel[b];

For a 240x240 R5G6B5 buffer this means 115,200 individual byte stores.

Solution

For R5G6B5 and D16 formats (both 2 bytes per pixel):

  1. memset fast path: when both bytes of the packed pixel are equal (e.g. black = 0x0000, depth 1.0 = 0xFFFF), use memset — a single call that the runtime can optimize with DMA or wide stores.
  2. 32-bit word fill: pack two pixels into a uint32_t and fill with aligned 32-bit writes, halving the number of store operations.

6. Compile-Time Format Detection

File: rlsw.h

Two token-pasting macros detect the framebuffer format at compile time, enabling #if guards with zero runtime cost:

#define SW_DEPTH_IS_D16    /* 1 when SW_FRAMEBUFFER_DEPTH_TYPE == D16 */
#define SW_COLOR_IS_R5G6B5 /* 1 when SW_FRAMEBUFFER_COLOR_TYPE == R5G6B5 */

These gate the fixed-point depth test (from a prior optimization), the direct pixel read/write paths, and the clear optimizations.


7. Fixed-Point Depth Test

File: rlsw.h
Guard: #if SW_DEPTH_IS_D16

The inner-loop depth test converts float z to uint16_t every pixel via (uint16_t)(z * 65535.0f) — an expensive float→int conversion. When the depth format is D16, the depth value and step are pre-converted to int32_t before the pixel loop:

int32_t z_fp   = (int32_t)(z    * 65535.0f);
int32_t dZdx_fp = (int32_t)(dZdx * 65535.0f);

The inner loop then uses integer compare and add:

uint16_t z16 = (uint16_t)z_fp;
if (z16 > depth) goto discard;
((uint16_t *)dPtr)[0] = z16;
// ...
z_fp += dZdx_fp;

8. ESP-DSP Matrix Acceleration

File: rlsw.h
Guard: #ifdef SW_HAS_ESP_DSP

When the ESP-DSP library is available, optimized matrix operations replace the scalar implementations:

  • sw_matrix_mul_rst → dspm_mult_4x4x4_f32 (4×4 matrix multiply)
  • Vertex MVP transform → dspm_mult_4x4x1_f32 (4×4 × 4×1 multiply)

A row-major transposed MVP copy (matMVP_rm) is maintained for ESP-DSP compatibility since rlsw uses column-major layout internally.


9. SRAM-Aware Framebuffer Allocation

File: rlsw.h, raylib_esp32_config.h
Guard: #ifdef CONFIG_IDF_TARGET_ESP32S3

Custom allocator hooks (SW_FB_REALLOC, SW_FB_FREE) allow framebuffer memory to be placed in specific memory regions. On ESP32-S3:

  • _sw_fb_alloc_prefer_internal: tries MALLOC_CAP_INTERNAL first, falls back to MALLOC_CAP_SPIRAM
  • The depth buffer is allocated first (it benefits most from fast SRAM since it is accessed for every pixel attempt, including occluded ones)

10. Fixed-Point Color Interpolation

File: rlsw.h
Guard: #if SW_COLOR_IS_R5G6B5 && !defined(SW_ENABLE_TEXTURE) && !defined(SW_ENABLE_BLEND)

Problem

In the triangle span inner loop, vertex colors were interpolated as float[4], then converted per pixel via color[i] * 31.0f + 0.5f → (uint32_t) → pack to R5G6B5. That is 3 float multiplies + 3 float→int conversions + 4 float adds (stepping) per pixel.

Solution

At each affine block boundary, convert srcColor and dSrcColordx to 16.16 fixed-point pre-scaled to R5G6B5 channel ranges (×31, ×63, ×31):

int32_t r_fp = (int32_t)(srcColor[0] * (31.0f * 65536.0f));
int32_t dr_fp = (int32_t)(dSrcColordx[0] * (31.0f * 65536.0f));

The inner pixel loop becomes pure integer: >> 16, clamp, shift-OR pack, and r_fp += dr_fp stepping. No floats touched per pixel.

Applies to non-textured, non-blend template variants (BASE, DEPTH). Textured paths still use float for the srcColor × texColor multiply.


11. POT Texture Bit-Masking

File: rlsw.h
Guard: #if defined(ESP_PLATFORM) (in both sw_texture_sample_nearest and sw_texture_sample_nearest_inline)

Problem

For SW_REPEAT wrap mode, sw_fract(u) calls floorf to get the fractional part. While floorf maps to floor.s on Xtensa FPU, the full wrap sequence (sw_fract + multiply + cast + clamp) is 6 instructions per coordinate.

Solution

For power-of-two textures (width & wMinus1 == 0), replace the entire wrap chain with a single bitmask:

x = (int)(u * tex->width) & tex->wMinus1;

This handles both positive and negative UVs through two's complement wrapping (off by at most 1 texel for rare negative UVs — imperceptible at 240×240). Non-POT textures fall through to the original sw_fract path.

Saves ~2 instructions per texel per coordinate on the nearest-filter hot path.