back to main page

Brain is a massively parallel computing machine

English Русский

There are many people who do not know that neurons in the brain process the information in parallel.
Below is the explanation why the brain is a massively parallel computing machine.

Human eye contains retina, which contains millions of neural cells (called “cones” and “rods”) and all these cells process visual information in parallel. Rods and cones are connected to other neural cells called “bipolar cells” and all these all these bipolar cells process visual information in parallel. Bipolar cells are connected to other neural cells called “ganglion cells” and all these all these ganglion cells process visual information in parallel. The axons of ganglion cells form optic nerve which contains between 770,000 and 1.7 million nerve fibers and all these million of fibers send visual information in parallel. And so on and so on.
Physical structure of human retina. <…> The entire retina contains about 7 million cones and 75 to 150 million rods. <…> In the outer the rods and cones connect to the vertically running bipolar cells, and the horizontally oriented horizontal cells connect to ganglion cells.
Each human optic nerve contains between 770,000 and 1.7 million nerve fibers, which are axons of the retinal ganglion cells of one retina.

From neurobiological point of view the eye is actually part of the brain.
The retina is actually part of the brain that is isolated to serve as a transducer for the conversion of patterns of light into neuronal signals. The lens of the eye focuses light on the photoreceptive cells of the retina, which detect the photons of light and respond by producing neural impulses. These signals are processed in a hierarchical fashion by different parts of the brain, from the retina upstream to central ganglia in the brain.

Hierarchical processing does not contradict to the concept of parallel processing. Millions of neural cells in retina (cons and rods) receive and process visual information simultaneously in parallel, then processed information is passed to the next layer of neural cells (bipolar cells) where millions of bipolar cells process visual information simultaneously in parallel, then processed information is passed to the next layer of neural cells and so on. All these layers also process information simultaneously in parallel, the only difference is that different layers process information with different time delay, however all layers process information simultaneously in parallel and also every layer has millions of neural cells which process information simultaneously in parallel. The brain works as massively parallel computing machine.
Neural networks are also similar to biological neural networks in performing functions collectively and in parallel by the units, rather than there being a clear delineation of subtasks to which various units are assigned.

We will repeat once again – from neurobiological point of view the eye is actually part of the brain. The neural cells in retina do process visual information, and this processing can be pretty complex, as for example the retina of the frog contains “fly detectors”, which can detect the moving fly/insect. The eye of the frog acts as parallel computing machine which is able to detect moving flies/insects without involving any other parts of the brain in the computations.
Horace B. Barlow was one of the first investigators to use the concept of the feature detector to relate the receptive field of a neuron to a specific animal behavior. In 1953, H.B. Barlow’s electrophysiological recordings from excised retina of the frog provided the first evidence for the presence of an inhibitory surround in the receptive field of a frog’s retinal ganglion cell. In reference to “on-off” ganglion cells – which respond to both the transition from light to dark and the transition from dark to light – and also had very restricted receptive fields of visual angle (about the size of a fly at the distance that the frog could strike), Barlow stated, “It is difficult to avoid the conclusion that the ‘on-off’ units are matched to the stimulus and act as fly detectors”.

There are many people who think that the brain can not run the software. These people think that only electronic computers can run the software.

The misunderstanding here is due to the lack of knowledge in the fields of information theory and computer science. The term “software” has a much broader meaning than many people are used to. Usually when people hear the word “software” they associate that word with digital computers which run digital software. However there is a completely separate class of computing machines which is called “analog computers”. Analog computers process information in completely different way than digital computers and also what is important – the software which runs on analog computers has completely different form from digital computers. Instead of digital digits analog computers process information in another form – they process analog signals. As for example, analog computer can process information in the form of pneumatic or hydraulic streams instead of electric signals.
An analog computer is a form of computer that uses the continuously changeable aspects of physical phenomena such as electrical, mechanical, or hydraulic quantities to model the problem being solved. In contrast, digital computers represent varying quantities symbolically, as their numerical values change. As an analog computer does not use discrete values, but rather continuous values, processes cannot be reliably repeated with exact equivalence, as they can with Turing machines. Analog computers do not suffer from the quantization noise inherent in digital computers, but are limited instead by analog noise.
Analog computers were widely used in scientific and industrial applications where digital computers of the time lacked sufficient performance. Analog computers can have a very wide range of complexity. Slide rules and nomographs are the simplest, while naval gunfire control computers and large hybrid digital/analog computers were among the most complicated. Systems for process control and protective relays used analog computation to perform control and protective functions.
The advent of digital computing and its success made analog computers largely obsolete in 1950s and 1960s, though they remain in use in some specific applications, like the flight computer in aircraft, and for teaching control systems in universities.
Analog computers often have a complicated framework, but they have, at their core, a set of key components which perform the calculations, which the operator manipulates through the computer's framework.
Key hydraulic components might include pipes, valves and containers.
Key mechanical components might include rotating shafts for carrying data within the computer, miter gear differentials, disc/ball/roller integrators, cams (2-D and 3-D), mechanical resolvers and multipliers, and torque servos.
Key electrical/electronic components might include:
   ● Precision resistors and capacitors
   ● operational amplifiers
   ● Multipliers
   ● potentiometers
   ● fixed-function generators

The core mathematical operations used in an electric analog computer are:
   ● addition
   ● integration with respect to time
   ● inversion
   ● multiplication
   ● exponentiation
   ● logarithm
   ● division

What is important here to understand is – in order to process information you need to have two things: 1) a hardware (computing machine) which runs 2) software (algorithms which process that information).
Human brain does process information, which means that by definition human brain contains software which runs on the neural-hardware (neurons of the brain). Human brain is the computing machine which process information in both forms (analog and digital) simultaneously.

Below is an example of the software that runs on an analog computer.
The V-2 (German: Vergeltungswaffe 2, "Retribution Weapon 2"), technical name Aggregat-4 (A4), was the world's first long-range guided ballistic missile. <…>
Some later V-2s used "guide beams", radio signals transmitted from the ground, to keep the missile on course, but the first models used a simple analog computer that adjusted the azimuth for the rocket

Below are some more examples.
 Impressions of Analog Computers
Large scale russian special purpose analog computer ZI-S used to solve questions in the field of hydraulics (cf. V. B. Ushakov, "Soviet Trends in Computers for Control of Manufacturing Processes", in "Instruments and Automation", Nov. 1958, p. 1812).
Analog computers at a NASA simulation facility during the development of the control stick for project Mercury.
Analog computer used at NASA for lunar landing simulations.
Analog Museum

Below is a textbook “Handbook of analog computation”, please read the Chapter 3, which explains the analog programming of analog computers.
Chapter 3. Elementary analog programming
Book title: Handbook of analog computation: Including application of digital control logic.
Year: 1967.
Author: Maxwell C Gilliland