Controlling Robot Dynamics With Spiking Neurons

NIPS 2013 Demonstration

Samir Menon, Sam Fok, Alex Neckar, Kevin Chavez, Chris Aholt, Scott Bertics, and Kwabena Boahen

Our lab recently developed Neurogrid, a neuromorphic chipset that emulates a million spiking neurons in real time. In this demo we will demonstrate Neurogrid controlling Neuroarm, a three degree-of-freedom force controlled robot arm.

There are three novel aspects to our demonstration:

  1. Real-time physical interaction between a human and a neuromorphic, compliant, force-controlled robotic system
  2. Non-linear dynamic control of the robot achieved by implementing operational space control on multiple spiking neuron populations.
  3. The application of the Neural Engineering Framework to real-world neuromorphic hardware and robotic systems, with physical noise and neural heterogeneity due to hardware mismatch.

Real-time Neural Control: A. Neurogrid controlling a simulated three degree-of-freedom robot (sped up x6). B,C. A real-time
spiking neuron network in Nengo implementing gravity compensation (B) and end-effector force control (C) for Neuroarm.

A fundamental problem while implementing robot control is to factorize the effect of the robot's non-linear dynamics from the trajectories that it must execute at the end-effector. Operational space control provides a general solution to this motor coordination problem and uses force control to combine the robot's forward dynamic model with trajectory control forces. The controller directly controls forces at the robot's generalized coordinates (joints).

For this demo, we controlled forces on Neuroarm by programming its force Jacobian, which maps hand forces into motor torques, on to a spiking neural network using the Neural Engineering Framework (NEF).

Neurogrid, a million neuron emulator
Neurogrid: emulates a million neurons across sixteen neurocores. The neurocores (blue squares) are connected in a tree
and have programmable connectivity through an FPGA (red). Neurogrid can communicate with a computer using USB (green).

A challenge to implementing computations onto spiking neural substrate is to operate with noisy, heterogeneous neurons on Neurogrid's neuromorphic hardware. NEF provides a principled method for mapping mathematical functions and dynamical systems onto a heterogeneous spiking neural substrate in a neurally plausible manner. NEF casts system inputs into the high dimensional space of neuron responses. First, each neuron is assigned an encoding vector, or preferred directions, and the system input is converted into input to each neuron by taking the inner product. After casting inputs into the high dimensional neural space, NEF finds the least-squares optimal decoding weights across each of the neural populations to approximate the desired function as the linear combination of neuron responses.


Credit to the Neurogrid team. Special thanks to Travis DeWolf for assisting in the demonstration setup.


[1] Boahen, K., "Neurogrid: Emulating a Million Neurons in the Cortex", 28th Annual International Conference of the IEEE Engineering in Medicine and Biology Society, pp. 6702,6702, Sept 2006

[2] Choudhary, S Sloan, S Fok, A Neckar, E Trautmann, P Gao, T Stewart, C Eliasmith, and K Boahen, "Silicon Neurons that Compute", International Conference on Artificial Neural Networks, LNCS vol 7552, pp 121-28, Springer, Heidelberg, 2012

[3] Sentis, L.; Jaeheung Park; Khatib, O., "Compliant Control of Multicontact and Center-of-Mass Behaviors in Humanoid Robots,", IEEE Transactions on Robotics, vol.26, no.3, pp.483,501, June 2010

[4] Eliasmith, Chris and Stewart, Terrence C. and Choo, Xuan and Bekolay, Trevor and DeWolf, Travis and Tang, Yichuan and Rasmussen, Daniel, “A Large-Scale Model of the Functioning Brain”,, Science, vol.338, no.6111, pp.1202,1205, November 2012

    © Samir Menon. CCA 3.0 license.
    Valid HTML and CSS.
    Last updated on Jan 1st, 2014