Computing Image Statistics and Applying Both Transforms

Computing Image Statistics and Applying Both Transforms#

Twitter Handle LinkedIn Profile GitHub Profile Tag Tag

This page implements the two transforms defined in Image Normalization vs Standardization: Why Scale Matters. The goal is concrete: compute per-channel mean and standard deviation over an image tensor, apply standardization ((168)) and min-max normalization ((167)) to a batch, and watch the before/after statistics change. It closes with the train-only statistics rule that prevents the data leakage warned about in the concept page.

By the end you will have a reusable calc_mean_std, the matching standardize and normalize_minmax transforms, and a split-then-standardize template you can drop into a data pipeline. Stats are computed in NumPy (the analysis tool the original author used); the transforms run on torch.Tensor, which is what a real training loop hands them.

Prerequisites

Read Image Normalization vs Standardization: Why Scale Matters first — this page implements its definitions (Definition 80, Definition 81) and assumes the variance-budget motivation. Basic familiarity with torch.Tensor indexing is enough.

A synthetic RGB dataset on the \([0, 255]\) scale#

Rather than download CIFAR-10, we synthesize an RGB tensor with the same fingerprint natural images carry — red dominant, blue suppressed — so the per-channel means are visibly different and the effect of standardization is unambiguous.

1torch.manual_seed(1930)
2
3N, C, H, W = 2048, 3, 32, 32  # 2048 images, 3 channels, 32x32 px each
4channel_means = torch.tensor([125.0, 123.0, 114.0]).view(1, C, 1, 1)
5images = (channel_means + 25.0 * torch.randn(N, C, H, W)).clamp(0, 255)
6
7print("dataset shape:", tuple(images.shape))
8print("per-channel mean (raw [0,255]):", images.mean(dim=(0, 2, 3)).tolist())
9print("per-channel std  (raw [0,255]):", images.std(dim=(0, 2, 3)).tolist())

dataset shape: (2048, 3, 32, 32)
per-channel mean (raw [0,255]): [124.99977111816406, 123.00863647460938, 114.0224380493164]
per-channel std  (raw [0,255]): [25.000476837158203, 24.999204635620117, 25.006010055541992]

The means sit near \((125, 123, 114)\) — the reddish bias natural photographs share — and the raw scale is in the hundreds. Feeding this straight into a network hands the first layer the \(\operatorname{Var}(x)\approx 600\) input that (166) will faithfully propagate forward.

Computing per-channel mean and std#

The function below mirrors the author’s calcMeanStd: it rescales pixels to \([0, 1]\) first, then reduces over every axis except the channel axis. Reducing over \((N, H, W)\) is exactly equivalent to flattening all pixels of one channel into a single array and calling .mean() — the pedagogical point of the original — but without materializing the flattened copy.

 1def calc_mean_std(images: np.ndarray) -> dict[str, tuple[float, ...]]:
 2    """Per-channel mean and std after rescaling pixels to [0, 1].
 3
 4    Args:
 5        images: array of shape ``(N, C, H, W)`` on the ``[0, 255]`` scale.
 6
 7    Returns:
 8        ``{"mean": (per-channel means), "std": (per-channel stds)}``, one
 9        entry per channel, in $[0, 1]$ units.
10
11    Raises:
12        ValueError: if ``images`` is not 4-dimensional.
13    """
14    if images.ndim != 4:
15        raise ValueError(f"expected (N, C, H, W); got shape {images.shape}")
16    images01 = images.astype(np.float64) / 255.0
17    means = images01.mean(axis=(0, 2, 3))
18    stds = images01.std(axis=(0, 2, 3))      # population std, matching the original
19    return {
20        "mean": tuple(float(m) for m in means),
21        "std": tuple(float(s) for s in stds),
22    }

1stats = calc_mean_std(images.numpy())
2print("calc_mean_std mean:", [round(m, 4) for m in stats["mean"]])
3print("calc_mean_std std :", [round(s, 4) for s in stats["std"]])
4
5# Idiomatic one-liner: reduce over every axis except the channel axis.
6images01 = images.float() / 255.0
7print("vectorized   mean :", images01.mean(dim=(0, 2, 3)).tolist())
8print("vectorized   std  :", images01.std(dim=(0, 2, 3), correction=0).tolist())

calc_mean_std mean: [0.4902, 0.4824, 0.4471]
calc_mean_std std : [0.098, 0.098, 0.0981]
vectorized   mean : [0.49019479751586914, 0.48238617181777954, 0.44714629650115967]
vectorized   std  : [0.09804105758666992, 0.09803607314825058, 0.09806276112794876]

Both forms agree. The per-channel means — roughly \((0.49, 0.48, 0.45)\) — are in the same ballpark as the canonical CIFAR-10 constants \((0.491, 0.482, 0.447)\), which is no accident: it is the shared fingerprint of natural RGB images.

Applying the two transforms#

The transforms operate on a torch.Tensor mini-batch. Both broadcast a per-channel parameter over the \((N, C, H, W)\) layout by reshaping it to \((1, C, 1, 1)\).

 1def standardize(
 2    batch: torch.Tensor, mean: torch.Tensor, std: torch.Tensor
 3) -> torch.Tensor:
 4    """Per-channel z-scoring, ``x' = (x - mean) / std``.
 5
 6    Args:
 7        batch: shape ``(N, C, H, W)``, already on the ``[0, 1]`` scale.
 8        mean, std: shape ``(C,)`` per-channel statistics.
 9    """
10    shape = (1, -1, 1, 1)  # broadcast over (N, C, H, W)
11    return (batch - mean.view(shape)) / std.view(shape)
12
13
14def normalize_minmax(batch: torch.Tensor) -> torch.Tensor:
15    """Per-channel min-max rescaling to ``[0, 1]``.
16
17    Args:
18        batch: shape ``(N, C, H, W)`` on any common scale.
19    """
20    x_min = batch.amin(dim=(0, 2, 3), keepdim=True)
21    x_max = batch.amax(dim=(0, 2, 3), keepdim=True)
22    return (batch - x_min) / (x_max - x_min)

Standardization: zero mean, unit variance#

1batch01 = images[:256].float() / 255.0        # a mini-batch on [0, 1]
2mean_c = torch.tensor(stats["mean"])
3std_c = torch.tensor(stats["std"])
4
5standardized = standardize(batch01, mean_c, std_c)
6print("before standardize — mean:", batch01.mean(dim=(0, 2, 3)).tolist())
7print("after  standardize — mean:", standardized.mean(dim=(0, 2, 3)).tolist())
8print("after  standardize — std :", standardized.std(dim=(0, 2, 3), correction=0).tolist())

before standardize — mean: [0.4902641475200653, 0.4823571741580963, 0.44720590114593506]
after  standardize — mean: [0.0007039904012344778, -0.00030110112857073545, 0.0006058933213353157]
after  standardize — std : [1.0015027523040771, 0.9990074634552002, 0.9990968108177185]

After standardization each channel’s mean is \(\approx 0\) and each std is \(\approx 1\) — the \(O(1)\) scale the variance budget (166) assumes. That is the whole point: the initializer can now preserve a unit scale because a unit scale is what it was handed.

Min-max normalization: range pinned, shape preserved#

1normalized = normalize_minmax(batch01)
2print("before normalize — min:", batch01.amin(dim=(0, 2, 3)).tolist())
3print("before normalize — max:", batch01.amax(dim=(0, 2, 3)).tolist())
4print("after  normalize — min:", normalized.amin(dim=(0, 2, 3)).tolist())
5print("after  normalize — max:", normalized.amax(dim=(0, 2, 3)).tolist())
6print("after  normalize — mean:", normalized.mean(dim=(0, 2, 3)).tolist())

before normalize — min: [0.07546952366828918, 0.06748534739017487, 0.0]
before normalize — max: [0.9469466805458069, 0.9191795587539673, 0.904196560382843]
after  normalize — min: [0.0, 0.0, 0.0]
after  normalize — max: [1.0, 1.0, 1.0]
after  normalize — mean: [0.47596725821495056, 0.48711374402046204, 0.4945891797542572]

Min-max nails every channel to exactly \([0, 1]\), but note what it leaves untouched: the per-channel means are still offset (the red channel is still brighter than blue). Standardization removed that offset; min-max did not. That is the operational difference between Definition 80 and Definition 81 — pin the range, or pin the moments.

Split before you standardize#

The data-leakage rule from Image Normalization vs Standardization: Why Scale Matters in code: split first, compute statistics on the training split alone, then apply those same constants to both splits.

 1perm = torch.randperm(N)
 2train_idx, val_idx = perm[: 4 * N // 5], perm[4 * N // 5 :]
 3train_raw, val_raw = images[train_idx], images[val_idx]
 4
 5train_stats = calc_mean_std(train_raw.numpy())   # train-only statistics
 6mean_t = torch.tensor(train_stats["mean"])
 7std_t = torch.tensor(train_stats["std"])
 8
 9train_std = standardize(train_raw.float() / 255.0, mean_t, std_t)
10val_std = standardize(val_raw.float() / 255.0, mean_t, std_t)   # same constants
11
12print("train — standardized mean:", train_std.mean(dim=(0, 2, 3)).tolist())
13print("val   — standardized mean:", val_std.mean(dim=(0, 2, 3)).tolist())
14print("(val is NOT zero-mean — that is correct: it borrows train's fingerprint.)")

train — standardized mean: [3.587752317457671e-08, -5.771811828481077e-08, -1.051928251172285e-07]
val   — standardized mean: [-0.002717789029702544, -8.494147914461792e-05, -0.0006845386233180761]
(val is NOT zero-mean — that is correct: it borrows train's fingerprint.)

The training split’s standardized mean is \(\approx 0\) by construction, while the validation split’s is only close to \(0\). That residual offset is not a bug — it is the absence of leakage. The validation set is being judged on the training set’s yardstick, which is precisely what held-out evaluation requires.

Summary#

If this page had to be one sentence: calc_mean_std turns an image tensor into the three-mean-three-std fingerprint, standardize and normalize_minmax apply the two rival transforms, and the split-then-standardize pattern keeps that fingerprint honest by fitting it on the training split alone. Drop the three functions into a data pipeline, compute the constants once on your training set, and your first layer receives the \(O(1)\)-scale input that weight initialization needs to do its job.

For the conceptual half — why input scale drives gradient scale, when to prefer min-max over z-scoring, and the per-image/per-channel/dataset-wide taxonomy — read Image Normalization vs Standardization: Why Scale Matters.

Further reading