fluxer-desktop-patched/fluxer_desktop/native/webrtc-sender/vendor/tract-linalg-0.23.1/MULTITHREAD_BENCHMARKS.md

Multithreaded MMM benchmarks

Validation data for the multithread-mm rayon path with this PR's chunked_dispatch_rayon + THREADING_PANEL_THRESHOLD + RayonGlobal changes.

Setup

• Base: tract main (commit 41b7b02), with the merged WASM kernel kit (PRs #2164 + #2173).
• Vanilla baseline: same commit, MMM dispatch unchanged (1D into_par_iter over single panel axis).
• Patched: this PR applied — chunked 2D dispatch, threshold, RayonGlobal.
• Both binaries built identically: same compiler, same kernel kit, same +atomics +bulk-memory +mutable-globals +simd128 target features for WASM.
• Driver: Playwright headless, real browser engines. Median of 60 iterations after 3-iter warmup.
• Output verification: FNV-1a hash of result tensor. All 60 cells produce identical hash (20ea4579c427f925 for DFN3, shape-deterministic for synthetic) — bit-equal output preserved.

Synthetic dense matmul (the parallelism-bound case)

Shape	Engine	Vanilla (1 thread)	Patched, 4 threads	Speedup
1024×1024×1024 (transformer FFN)	Chromium	55.4 ms	16.6 ms	3.34×
	WebKit	79.8 ms	25.6 ms	3.12×
	Firefox	1107 ms	325 ms	3.40×
512×768×768 (BERT FFN)	Chromium	15.7 ms	5.1 ms	3.07×
	WebKit	22.6 ms	6.8 ms	3.30×
	Firefox	311 ms	91 ms	3.42×
256×256×128	Chromium	0.47 ms	0.22 ms	2.19×
	WebKit	0.66 ms	0.20 ms	3.30×
	Firefox	9.0 ms	2.6 ms	3.41×
64×256×64 (DFN-like small)	Chromium	0.07 ms	0.07 ms	1.0× (within noise)
	WebKit	0.08 ms	0.04 ms	2.00×
	Firefox	1.18 ms	0.38 ms	3.11×
32×32×32 (tiny)	All	sub-ms	sub-ms	1.0× (threshold gates)

The threshold correctly gates the smallest shape; threading kicks in once panel count clears the gate, and scales near-linearly with thread count on all three engines.

Real model (DeepFilterNet 3, full streaming inference)

5-frame chunks at 48 kHz (50 ms of audio per iteration).

Engine	Vanilla mono	Patched, 4 threads	RTF (vanilla → patched)	Speedup
Chromium	3.31 ms	3.17 ms	0.066 → 0.063	1.04× (within noise)
WebKit	4.32 ms	4.00 ms	0.086 → 0.080	1.08×
Firefox	34.20 ms	33.34 ms	0.684 → 0.667	1.03×

DFN3 is Amdahl-bound: only ~25% of runtime is in MMMs that clear the threshold; the rest is FFT, complex multiplication, and small RNN-internal matmuls that the threshold deliberately keeps single-threaded. The threshold's role here is to prevent regression, not deliver speedup. This is correct behavior — DFN3-class workloads should not pay rayon overhead on tiny ops.

Native (macOS aarch64, generic kernels)

Spot-check on the rayon path before/after. Tract's existing rayon path already worked well on native; the change is mostly a refactor.

Shape	Vanilla 1D	Patched 2D	Change
256×256, 4 threads	2.13 ms	2.12 ms	net-neutral
512×512, 4 threads	9.93 ms	9.80 ms	+1%
64×256, 4 threads	615 µs	625 µs	−2%

Within noise on common shapes. The 2D dispatch shows a latent benefit on shapes 1D parallelism handles poorly (e.g. m=8 n=2048, where 1D over m can only feed 2 threads); not yet measured directly on native but the dispatch math is the same on both targets.

Determinism

Across all measured cells (synthetic + DFN3, 3 engines, 1/2/3/4 threads):

• WASM: 60 cells, all produce identical hash per (shape, mode) pair.
• Native: existing tract proptests (3524 lib tests) pass with this PR's multithread-mm enabled.

Tuning the threshold

The default THREADING_PANEL_THRESHOLD is 64 panels (m_panels × n_panels). Adjust at runtime via:

use tract_linalg::multithread::set_threading_panel_threshold;

set_threading_panel_threshold(0);    // thread every size, no gate
set_threading_panel_threshold(256);  // gate harder — transformer-only
set_threading_panel_threshold(64);   // default

Useful when profiling or specialising the build for a known workload class:

Workload class	Suggested threshold
Streaming RNN / mobile vision (many small MMMs)	64 (default) or higher
Mid-size dense (BERT-class)	32–64
Large dense only (transformer FFN, LLM)	16 or lower

The constant lives in linalg/src/multithread.rs; readers go through current_threading_panel_threshold() (AtomicUsize::Relaxed, no lock on the dispatch hot path).

Reproduction

The harness uses Vonage's libDF fork (branch dfn3-wasm-opt-tract-022-kernel-kit) migrated to tract main, with a wasm-bindgen-rayon-based threading bootstrap. Build:

RUSTFLAGS="-C target-feature=+atomics,+bulk-memory,+mutable-globals,+simd128" \
wasm-pack build --target web --release \
    --no-default-features --features wasm-mt -- \
    -Z build-std=std,panic_abort

JS-side:

import init, { initThreadPool, df_set_thread_count } from './pkg/df.js';
await init();
await initThreadPool(navigator.hardwareConcurrency);  // wasm-bindgen-rayon
df_set_thread_count(4);  // sets Executor::RayonGlobal in tract-linalg

Without Executor::RayonGlobal (this PR), df_set_thread_count would need to construct an Arc<rayon::ThreadPool> — which fails on wasm32-unknown-unknown because rayon::ThreadPoolBuilder::new().build() internally calls std::thread::spawn (unsupported there). That's the crux of why this enabling change is needed in tract-linalg itself: any browser threading via wasm-bindgen-rayon would otherwise silently fall back to single-threaded.