Benchmark


Overview

For each of the 50 images of our benchmark we provide locations of 20 bounding boxes that are to be denoised individually – thus yielding 1000 bounding boxes in total. For each bounding box we compute PSNR and SSIM values. The final PSNR and SSIM values listed in the tables belows are computed by averaging the per-bounding-box values.

Application of denoising algorithms should be done in one of the following three ways:

  1. Denoising on raw data directly.
  2. Denoising on raw data after a variance stabilizing transformation (VST) was applied. The final denoised image is obtained by inverting the VST.
  3. Denoising on sRGB data directly.

For option 1 and 2 we provide evaluation against RAW and sRGB ground truth, respectively, while for option 3 we evaluate against the sRGB ground truth only.

Anonymous results are shown in italic font.


Results for denoising on raw data without VST

NameUses VSTDenoised onPSNR on rawSSIM on rawPSNR on sRGBSSIM on sRGB
BM3D0raw46.640.972437.780.9308
EPLL0raw46.310.967937.160.9291
WNNM0raw46.30.970737.560.9313
FoE0raw45.780.966635.990.9042
KSVD0raw45.540.967636.590.9162
TNRD0raw44.970.962435.570.8913
NCSR0raw42.850.852730.650.7025
MLP0raw42.70.939533.630.8829

Results for denoising on raw data with VST

NameUses VSTDenoised onPSNR on rawSSIM on rawPSNR on sRGBSSIM on sRGB
BM3D1raw47.150.973737.860.9296
NCSR1raw47.070.968837.790.9233
WNNM1raw47.050.972237.690.926
KSVD1raw46.870.972337.630.9287
EPLL1raw46.860.97337.460.9245
MLP1raw45.710.962936.720.9122
TNRD1raw45.70.960936.090.8883
FoE1raw44.120.955435.920.911

Results for denoising on sRGB data

NameUses VSTDenoised onPSNR on sRGBSSIM on sRGB
TWSC0srgb37.9390.9403
KSVD0srgb36.490.8978
WNNM0srgb34.670.8646
FoE0srgb34.620.8845
BM3D0srgb34.510.8507
MLP0srgb34.230.8331
NCSR0srgb34.050.8351
TNRD0srgb33.650.8306
EPLL0srgb33.510.8244