Toma Okugawa日本語

Research

Proposal of Fast Single-Scale Retinex and Evaluation of Real-Time Human Pose Estimation Performance under Low-Light Conditions

A luminance-only Retinex with a separable Gaussian, about 5.7× faster than 2-D SSR, that lets MediaPipe Hands find landmarks in near-dark video

Hand-landmark and pose libraries such as MediaPipe and OpenPose assume a reasonably lit scene. Point a webcam at a hand in a dark room and detection fails. Retinex-based enhancement can correct the lighting naturally, but classic Single-Scale Retinex (SSR) estimates illumination with a 2-D Gaussian blur that is too slow for live video, and prior work had not quantitatively validated SSR as a preprocessing step for skeleton detection. This sole-author preprint, published on Jxiv on November 12, 2025, was my first attempt at both problems: a stripped-down SSR fast enough to run in front of a detector, and a test of whether the detector benefits.

Method

FSSR works only on the luminance channel. I extract Y = 0.299R + 0.587G + 0.114B, model it as the product of illumination and reflectance following Retinex theory, and move to log space so the product becomes a sum. Illumination is estimated with a separable Gaussian filter, a 1-D pass along x followed by a 1-D pass along y, which cuts the per-pixel cost from O(N²) to O(2N) for a kernel of radius N. Subtracting log illumination from log luminance gives the reflectance, and exponentiating it gives the enhanced luminance.

Color is restored by multiplying the original RGB image by the ratio of enhanced to original luminance, so chroma is left untouched. A gamma of 0.8 is applied to the output, and the Gaussian uses σ = 15.0.

Block diagram of the FSSR pipeline: RGB is converted to luminance Y = 0.299R + 0.587G + 0.114B; a Retinex block writes log Y as log illumination plus log reflectance and estimates the illumination with a separable Gaussian filter (a 1-D kernel along x, then along y); the enhanced luminance exp(log R) is mapped back to RGB by multiplying the original image by the ratio Y_enh / Y_orig.

Evaluation

I ran the pipeline on live 1920×1080 webcam video with MediaPipe Hands (detection and tracking confidence both 0.5), logging the filter time and the total time for every frame. Over 955 frames and 120.14 seconds of continuous operation, the separable-Gaussian FSSR took about 4.72 ms per frame against about 26.95 ms for a full 2-D Gaussian convolution, a speedup of roughly 5.7×. The whole pipeline, MediaPipe included, averaged about 8.43 fps. The mean squared error between luminance before and after correction, MSE(L), was 472.73, which the paper interprets as the dark-region tonal range being effectively expanded.

The detection result is qualitative. In the paper’s single figure, the unprocessed frame shows a barely visible hand with no landmarks, while the FSSR frame yields all 21 MediaPipe hand landmarks, fingertips and joints included. The paper compares only against the full 2-D Gaussian and reports no pose-accuracy metrics; comparisons with other enhancement methods came in the later papers.

What it led to

The question behind this paper, whether a pose estimator can find joints after enhancement, was the first sign that my two research threads, making cameras see in the dark and reading human motion, were really one. FSSR still estimated illumination with a Gaussian. XCR, published on Jxiv the same month, replaced it with a separable kernel derived from complex exponential and cosine functions, and SD-Retinex (June 2026) made that kernel stable. The paper lists further FPS improvement, multi-person detection, and fusing non-RGB sensor data as future work.

Publications