Reinforcement learning in densely recurrent biological networks.

Abstract

Training highly recurrent networks is a technical challenge: gradient-based methods suffer from exploding or vanishing gradients, while purely evolutionary searches converge slowly. We introduce a hybrid, gradient-free optimization framework that implements reinforcement learning by coupling global evolutionary exploration with local direct search exploitation. The method, termed ENOMAD (Evolutionary Nonlinear Optimization with Mesh Adaptive Direct search), is benchmarked on a suite of food-foraging tasks instantiated in the fully mapped neural connectome of the nematode <i>Caenorhabditis elegans</i>. Crucially, ENOMAD leverages biologically derived weight priors, letting it refine-rather than rebuild-the organism's native circuitry. The resulting network significantly over-performs the untrained connectome, in what can be interpreted as an example of transfer learning. Our findings demonstrate that integrating evolutionary search with nonlinear optimization provides an efficient, biologically grounded strategy for specializing natural recurrent networks toward a specified set of tasks.