Benchmark


Overview

For each of the 50 images of our benchmark we provide locations of 20 bounding boxes that are to be denoised individually – thus yielding 1000 bounding boxes in total. For each bounding box we compute PSNR and SSIM values. The final PSNR and SSIM values listed in the tables belows are computed by averaging the per-bounding-box values.

Application of denoising algorithms should be done in one of the following three ways:

  1. Denoising on raw data directly.
  2. Denoising on raw data after a variance stabilizing transformation (VST) was applied. The final denoised image is obtained by inverting the VST.
  3. Denoising on sRGB data directly.

For option 1 and 2 we provide evaluation against RAW and sRGB ground truth, respectively, while for option 3 we evaluate against the sRGB ground truth only.

Anonymous results are shown in italic font.


Results for denoising on raw data without VST

Name Uses VST Denoised on PSNR on raw SSIM on raw PSNR on sRGB SSIM on sRGB RPB Details
UPI (Raw) 0 raw 48.8905 0.9824 40.1728 0.9623 0.022
UPI (sRGB) 0 raw 48.8824 0.9821 40.3545 0.9641 0.022
RU_raw 0 raw 48.7322 0.9812 39.7821 0.9552 0
PRIDNet_raw 0 raw 48.4839 0.9806 39.4681 0.9522 0
N3Net 0 raw 47.5643 0.9767 38.3158 0.9384 1
DSSNet 0 raw 47.3348 0.9784 37.8575 0.9427 0
BM3D 0 raw 46.64 0.9724 37.78 0.9308

NLH_raw 0 raw 46.5998 0.9676 37.8305 0.9348 0

EPLL 0 raw 46.31 0.9679 37.16 0.9291

WNNM 0 raw 46.3 0.9707 37.56 0.9313

FoE 0 raw 45.78 0.9666 35.99 0.9042

KSVD 0 raw 45.54 0.9676 36.59 0.9162

TNRD 0 raw 44.97 0.9624 35.57 0.8913

advect_run05 0 raw 44.4837 0.9526 35.6892 0.8963 0.04

NCSR 0 raw 42.85 0.8527 30.65 0.7025

MLP 0 raw 42.7 0.9395 33.63 0.8829

Results for denoising on raw data with VST

Name Uses VST Denoised on PSNR on raw SSIM on raw PSNR on sRGB SSIM on sRGB RPB Details
BM3D 1 raw 47.15 0.9737 37.86 0.9296

NCSR 1 raw 47.07 0.9688 37.79 0.9233

WNNM 1 raw 47.05 0.9722 37.69 0.926

KSVD 1 raw 46.87 0.9723 37.63 0.9287

EPLL 1 raw 46.86 0.973 37.46 0.9245

MLP 1 raw 45.71 0.9629 36.72 0.9122

TNRD 1 raw 45.7 0.9609 36.09 0.8883

FoE 1 raw 44.12 0.9554 35.92 0.911

Results for denoising on sRGB Data

Name Uses VST Denoised on PSNR on sRGB SSIM on sRGB RPB Details
mwresnet 0 srgb 39.8447 0.958 0
Path-Restore-Ext 0 srgb 39.7227 0.9591 0.42

RU_sRGB 0 srgb 39.5064 0.9528 0

DRDN 0 srgb 39.4342 0.9531 0

PRIDNet_sRGB 0 srgb 39.4247 0.9528 0.05

VDN 0 srgb 39.383 0.9518 0.004

IERD+ 0 srgb 39.3032 0.9531 0.1

SmartDSP 0 srgb 39.302 0.9513 0.01

RIDNet 0 srgb 39.2555 0.9528 0.1

MLDN 0 srgb 39.2311 0.9516 0.48

CIMM 0 srgb 39.1983 0.9524 0.1

ATDNet 0 srgb 39.1901 0.9526 0.01

DRSR_v1.4 0 srgb 39.0941 0.9509 0.257

Path-Restore 0 srgb 39.0047 0.9542 0.15

WDnCNN+ 0 srgb 38.8695 0.9501 0.4

NLH 0 srgb 38.8071 0.952 5.3

HT-MWResnet 0 srgb 38.6735 0.9473 0
NLH_LE 0 srgb 38.6468 0.9508 0

NTGAN+ 0 srgb 38.5768 0.9474 0.4
PD 0 srgb 38.4005 0.9452 0

CBDNet 0 srgb 38.0564 0.9421 0.4

DSSNet 0 srgb 38.0391 0.9358 0
TWSC 0 srgb 37.9643 0.9416 0

DnCNN+ 0 srgb 37.9018 0.943 0.05

FFDNet+ 0 srgb 37.6107 0.9415 0

MCWNNM 0 srgb 37.379 0.9294 0

LBD 0 srgb 37.3233 0.9431 0
ULBD 0 srgb 37.2179 0.9341 0.01

Nonblind_MCD 0 srgb 37.1976 0.9361 0

MCD 0 srgb 37.12 0.9353 0
128-DnCNN tensorflow 0 srgb 37.0343 0.9324 0.04
SCD 0 srgb 36.9453 0.9299 0
KSVD 0 srgb 36.49 0.8978

DRNet 0 srgb 35.8881 0.9351 0

GCBD 0 srgb 35.5796 0.9217
NC 0 srgb 35.434 0.8841 18.5

NI 0 srgb 35.1125 0.8778 1.2

DenoiseNet 0 srgb 35.0802 0.868 0.33

WNNM 0 srgb 34.67 0.8646

FoE 0 srgb 34.62 0.8845

advect_run_sRGB_02 0 srgb 34.5311 0.8763 1

BM3D 0 srgb 34.51 0.8507

UNET based Image denoising 0 srgb 34.3599 0.9221 0.005
MLP 0 srgb 34.23 0.8331

NCSR 0 srgb 34.05 0.8351

TNRD 0 srgb 33.65 0.8306

EPLL 0 srgb 33.51 0.8244

DnCNN 0 srgb 32.4296 0.79
Original Noisy 0 srgb 29.836 0.7018 0