Network Diagrams

MSE-assisted WDL head training§

It would be very nice if our networks could give outcome distributions instead of the pure expected-value outputs they currently provide, so that we could access properties like the distribution’s entropy.

Directly training on outcome labels sacrifices the extremely informative $v$ target available in normal training, and so methods for training outcome-estimating networks on intermediate scores have seen some attention. At time of writing, the most effective schemeInvented by Yoshie, author of PlentyChess. is to add an auxiliary MSE loss between the standard blended NNUE target and the expected value computed from the WDL probability vector.

In practice these objectives are blended with a high weighting toward the value objective.

$$p_i = \frac{e^{x_i}}{\sum_j e^{x_j}}, \qquad s = p_\text{win} + \tfrac12\, p_\text{draw}$$

$$t = \lambda z + (1-\lambda)\,\sigma\!\left(\tfrac{v}{400}\right)$$

$$\mathcal{L} = \underbrace{(s - t)^2}_{\text{MSE}} \;-\; \alpha \underbrace{\sum_{i} \mathrm{onehot}_3(z)_i \log p_i}_{\text{CE}}$$

Where $\alpha = 0.1$. As a result of the fact that we’re only ever taking cross-entropy against one-hot vectors, we could write the loss as simply $-\log p_{z}$, the negative log-probability of the empirical outcome.

I trained a net in this fashion, called goibniu, which performed admirably against the standard network recipe:

  LLR +2.95 (−2.94LO +2.94HI BOUNDS for +0.00LO +3.00HI ELO)
  ELO +4.13 ± 2.82 (+1.31LO +6.94HI)
 CONF 25 KNODE (1 THREAD 16 MB CACHE)
GAMES 24004 (7111W29.6% 10067D41.9% 6826L28.4%)
PENTA 538+2 2981+1 5186+0 2822−1 475−2

  LLR −3.01 (−2.94LO +2.94HI BOUNDS for +0.00LO +3.00HI ELO)
  ELO −6.00 ± 3.69 (−9.69LO −2.31HI)
 CONF 8+0.08 SEC (1 THREAD 16 MB CACHE)
GAMES 9384 (2210W23.6% 4802D51.2% 2372L25.3%)
PENTA 42+2 1031+1 2401+0 1159−1 59−2

  LLR −2.99 (−2.94LO +2.94HI BOUNDS for +0.00LO +3.00HI ELO)
  ELO −4.78 ± 3.28 (−8.07LO −1.50HI)
 CONF 40+0.4 SEC (1 THREAD 128 MB CACHE)
GAMES 10970 (2550W23.2% 5719D52.1% 2701L24.6%)
PENTA 9+2 1271+1 2791+0 1388−1 26−2

  ELO +2.54 ± 7.33 (−4.79LO +9.87HI)
 CONF 120+1.0 SEC (1 THREAD 128 MB CACHE)
GAMES 2052 (483W23.5% 1101D53.7% 468L22.8%)
PENTA 2+2 245+1 545+0 234−1 0−2

This was back in January, and I suspect that a slightly more subtle form of this recipe has a shot of working quite well.

Block-sparse L1§

A feature common to deep NNUE architectures is the L1 bottleneck. As noted in Writing a top-10 chess engine in Rust, the first layer (“L0”) of an NNUE can afford to have a very large output dimension, due to a combination of both guaranteed input sparsity and incremental computation, while the second layer (“L1”) has no such affordances. This means that the output of the second layer must be very narrow if network inference is going to be tractable.

The downside of this is that layers after L1 now have a very impoverished representation space in which to do their work.

A possible solution to this is to use structured matrixes, which have sparsity enforced at training time. One can design an L1 that generates output vectors of twice the typical dimension using equal flops, by forcing each half of the output vector to be “blind” to half of the input vector.

When I trained a network like this, it got fairly poor loss, and so I didn’t explore much further, but this is still worth considering as an avenue of exploration.

✦✦✦