Weights ≠ Function · 3 of a series · live lab
Do two networks trained from different seeds learn the same thing?
Train two real networks in your browser on a polygon. Watch both draw the same edges, then see why averaging them breaks until you account for mirror-image neurons. Smallest case: one hidden layer, sigmoid units.
Lines: each hidden unit’s decision line; the small tick marks the side where the unit switches on. Matching colours link units across networks after the best alignment; a dashed line is a unit that had to be flipped (its tick points the other way). Charts: orange dashed = no alignment, white = permutations only, cyan = permutations and sign flips. Left chart: error (%). Right chart: loss. Same two networks in both.
How many neurons does this shape need?
Run CAPACITY SCAN: trains 5 networks at each of three widths (about half a minute). Each dot is one network’s final training error; the vertical line is your tolerance.
Technical statement
Networks follow f(x)=Σ sᵢσ(wᵢᵀx+bᵢ)+c with a sigmoid output and cross-entropy, Glorot-uniform initialisation, zero biases, full-batch Adam (lr 0.01, 3,000 steps). Alignment searches all 2hh! permutation and sign-flip combinations ((w,b,s,c) ↦ (−w,−b,−s,c+s)), chosen by midpoint loss on 400 points; barriers use 25 interpolation weights on all 1,000 points: B = max over α of [L(θα) − ((1−α)L(θA)+αL(θB))].
Evidence label: live mini-replication. This runs the preprint’s pipeline on simplified polygons with 1,000 points and one seed pair at a time. It is not the preprint’s data: the paper uses 100–120 seeds per task, irregular polygons, 2,000 points and a 2% inclusion rule, and reports minimal width 3 or 4 at 2% error and k at 1%. A network that misses 2% is not included in the paper’s results, and narrow networks often miss it. Scope: one hidden layer, sigmoid; ReLU has no sign flip. Source: Lakhanpal (2026), preprint v2, Sec. 3–5.