Benchmark


Overview

For each of the 50 images of our benchmark we provide locations of 20 bounding boxes that are to be denoised individually – thus yielding 1000 bounding boxes in total. For each bounding box we compute PSNR and SSIM values. The final PSNR and SSIM values listed in the tables belows are computed by averaging the per-bounding-box values.

Application of denoising algorithms should be done in one of the following three ways:

  1. Denoising on raw data directly.
  2. Denoising on raw data after a variance stabilizing transformation (VST) was applied. The final denoised image is obtained by inverting the VST.
  3. Denoising on sRGB data directly.

For option 1 and 2 we provide evaluation against RAW and sRGB ground truth, respectively, while for option 3 we evaluate against the sRGB ground truth only.


Results for denoising on raw data without VST

NameUses VSTDenoised onPSNR on rawSSIM on rawPSNR on sRGBSSIM on sRGB
BM3D0raw46.650.985237.780.9384
WNNM0raw46.320.984437.560.9357
EPLL0raw46.320.982537.160.931
FoE0raw45.790.982335.990.9133
KSVD0raw45.550.982536.60.9219
TNRD0raw44.980.979435.570.9007
NCSR0raw42.860.862930.650.7693
MLP0raw42.710.964633.630.8752

Results for denoising on raw data with VST

NameUses VSTDenoised onPSNR on rawSSIM on rawPSNR on sRGBSSIM on sRGB
BM3D1raw47.160.985337.8737.8651
NCSR1raw47.080.979937.7937.7924
WNNM1raw47.070.983537.737.6955
KSVD1raw46.880.984337.6337.6315
EPLL1raw46.870.984637.4637.458
MLP1raw45.720.980436.7236.7249
TNRD1raw45.710.97836.0936.0899
FoE1raw44.130.97635.9335.9257

Results for denoising on sRGB data

NameUses VSTDenoised onPSNR on sRGBSSIM on sRGB
KSVD0srgb36.490.9261
WNNM0srgb34.670.8894
FoE0srgb34.620.897
BM3D0srgb34.510.8668
MLP0srgb34.230.8622
NCSR0srgb34.050.8534
TNRD0srgb33.650.8601
EPLL0srgb33.520.8577