Synopsis

Teaching a Neural Network the Hard Way

Physics 14, s25
A neural network can be made to produce more reliable predictions of nonlinear systems if it is created with conservation laws built in.
R. Gauthier-Butterfield/UC Irvine

A baby learns by observation how objects move. But without accounting for conservation of momentum, its developed understanding is just an educated guess. Similarly, an artificial neural network (ANN) learns from empirical data how a particular system works, but without explicitly considering the conservation laws that govern that system, it risks making unreliable predictions. To address this limitation, Tom Beucler at the University of California, Irvine, and colleagues have devised a way to hardwire an ANN with such laws. They demonstrated the technique using an atmospheric model for the climate, but they say that their method can be applied to models of any physical system [1].

An ANN is an algorithm-based tool for turning a set of inputs into a set of outputs. The development or “training” process for an ANN involves incrementally reconfiguring its underlying algorithm with data from observations until the outputs accurately match reality. During this process, a conventional ANN can be discouraged from making physically impossible predictions by imposing “soft constraints” on its outputs—rewarding outcomes that better conform to physical laws.

Beucler and colleagues used an ANN to simulate the climatic effects of atmospheric convection. Such models are reliable only if they respect strict conservation of mass and energy. Instead of imposing these laws using soft constraints, the team embedded them in the architecture of the ANN as “hard constraints.” Likened to an infant’s learning process, the method is equivalent to making physics-defying predictions literally unthinkable.

The addition of these hard constraints makes it more difficult for the ANN to reach its optimal output. The researchers say that this price is worth paying though, as conservation laws are critical to climate models.

–Marric Stephens

Marric Stephens is a Corresponding Editor for Physics based in Bristol, UK.

References

  1. T. Beucler et al., “Enforcing analytic constraints in neural networks emulating physical systems,” Phys. Rev. Lett. 126, 098302 (2021).

Subject Areas

Biological PhysicsComplex SystemsNonlinear Dynamics

Related Articles

Shock Waves from Ions Damage DNA
Biological Physics

Shock Waves from Ions Damage DNA

Simulations show that the mechanical force of shock waves propagating through cells may be a key component of ion radiation damage to DNA.     Read More »

1D Random-Field Ising Model Experimentally Realized
Complex Systems

1D Random-Field Ising Model Experimentally Realized

Researchers accurately replicate the predictions of the 1D random-field Ising model in an artificial spin ice. Read More »

How CRISPR/Cas9 Finds Genetic Targets
Biological Physics

How CRISPR/Cas9 Finds Genetic Targets

A model of facilitated diffusion and the theory of Anderson localization help explain how the Cas9 protein explores DNA in search of its targets. Read More »

More Articles