Improved Training for Energy-Saving Neural Nets
Physical neural networks (PNNs) are analog computing systems based on electronic, optical, or even biological hardware rather than computer chips. PNNs can potentially perform as well as conventional AI systems. But they consume less energy while being more resistant to the effects of noisy environments. Now Satoshi Sunada of Kanazawa University in Japan and his colleagues have developed a training protocol that addresses some of the challenges for PNN training [1]. In tests, the researchers found that an optoelectronic circuit trained using their protocol performed as well as conventional neural networks. They say their protocol should allow a wider variety of physical systems to serve as PNN computing platforms.
Sunada and his colleagues focused on a general type of PNN that is not a traditional network. Instead, it could be any complex physical process that takes an input signal and produces an output signal that depends on both the input and a control signal, with all three signals varying in time. The training process allows the control signal to be optimized to provide the most accurate outputs given a range of inputs.
Previous protocols have either required a detailed model of the computing system—which often isn’t possible for PNNs—or they have been limited to step-by-step (discrete-time) calculations rather than those involving variables that evolve continuously in time. The new protocol avoids these issues by combining elements from two different theoretical frameworks. One is a mathematical approach to optimizing an externally controlled system’s performance called optimal control theory. The other is a training approach called direct feedback alignment. Tests using an optoelectronic circuit indicated rapid convergence to accurate outputs and good performance in the presence of environmental noise.
–David Ehrenstein
David Ehrenstein is a Senior Editor for Physics Magazine.
References
- S. Sunada et al., “Blending optimal control and biologically plausible learning for noise-robust physical neural networks,” Phys. Rev. Lett. 134, 017301 (2025).