I have just discovered the paper “Superior pattern processing is the essence of the evolved human brain”
by Mark P. Mattson (Frontiers of Neuroscience, 2014; 8: 265 – online) and I am extremely excited
because of the way it ties in with my own research. The abstract reads as
follows:
Humans have long pondered the nature of their
mind/brain and, particularly why its capacities for reasoning, communication
and abstract thought are far superior to other species, including closely
related anthropoids. This article considers superior pattern processing (SPP)
as the fundamental basis of most, if not all, unique features of the human
brain including intelligence, language, imagination, invention, and the belief
in imaginary entities such as ghosts and gods. SPP involves the
electrochemical, neuronal network-based, encoding, integration, and transfer to
other individuals of perceived or mentally-fabricated patterns. During human
evolution, pattern processing capabilities became increasingly sophisticated as
the result of expansion of the cerebral cortex, particularly the prefrontal
cortex and regions involved in processing of images. Specific patterns, real or
imagined, are reinforced by emotional experiences, indoctrination and even
psychedelic drugs. Impaired or dysregulated SPP is fundamental to cognitive and
psychiatric disorders. A broader understanding of SPP mechanisms,
and their roles in normal and abnormal function of the human brain, may enable
the development of interventions that reduce irrational decisions and
destructive behaviors.
I have emphasised the final sentence and note the following
paragraph from the body of the text.
While some principles by which the brain uses
pattern recognition and encoding to represent the past and the future have been
established, a clear understanding of the underlying molecular and cellular
mechanisms is lacking. How the encoded patterns are recalled and
processed to generate enduring memories of the different patterns and their
association with other encoded patterns (e.g., associations of the image of an
object with the sound, smell, or feel of that object) is also not well
understood. Nevertheless, the human brain is capable of using
stored information to generate novel images, sounds, and other patterns in the
processes of imagination and invention.
The reason for my excitement is that CODIL (Context
Dependent In formation Language) clearly involves “Superior Pattern Processing”
and the ideas discussed on this blog suggest how it can be related to the
neural network of the brain. A second blog post will look in detail at what the
model tells us about how the human brain evolved from an animal brain.
Significant information on CODIL is available on this blog
(see An
introduction to the publications on CODIL) and its relevance to superior
pattern processing is perhaps best introduced in term of a simple model of the
human short term memory, and then by a summary of the history of the project
and what was achieved before the project closed in 1988. The implication of the
model on the evolution of intelligence will be discussed in a separate blog
post.
A simple model of
human short-term memory.
Imagine that the neural network in your brain is like a sea
consisting of interlinked neurons with waves of activity passing through the
network.
Your look out of the window and see a rabbit. The concept
“rabbit” is dropped into the sea and a series of ripples spread out across the
surface. Another concept “hunger” is already causing ripples in the sea of
neurons and the two sets of ripples collide and generate a new concept “rabbit
pie”. This may lead to further ripples relation to how you are going to catch
the rabbit, and how to prepare and cook rabbit pie.
What has happened is that somewhere in the sea of neurons
there is at least one neuron that that links the concepts “rabbit, “hunger” and
“rabbit pie.” This suggest a way in which modelling mental activities can be divided into two
separate but interlinked tasks. One is modelling how the network stores information and how the
activity waves spread and is not concerned with the meaning in terms of
external concepts. The other models relates to the way the named concepts interact to provide
a high level processing facility which could be deemed to be intelligent.
The Origins of the
CODIL Research
As CODIL started by looking at very different problems to
those looked at by Mark P. Mattson it is appropriate to take a historical look
at the research, with the benefit of hindsight.
In the 1960s computers were just beginning to be used on a
significant scale for commercial data processing. Perhaps the biggest
commercial user in the UK was the oil marketing giant, Shell Mex & BP. They
had a vast sales accounting system in which information relating to good
delivered and payments received were punched into paper tape and feed to a
large LEO III computer. This was processed used information held on magnetic
tapes and used to produce invoices, sales statistics, etc. Towards the end of
the decade they started to plan a move to the next generation of computers
which would have direct access storage (about as much as today could be stored
on one CD!) and a small number of very simple (by today’s standards) computer
terminals.
In the early 1960’s I was working as an information
scientist providing management with research and development information in an
international veterinary company. No computers were used but the ideas of Vannevar Bush in “As we may think” interested me, and as my employer seemed
disinterested I moved to Shell Mex & BP in late 1965 to become a systems
analyst. I started by training as a programmer and became very interested in
how programming errors arose, and how they could be avoided. After a year I
moved to systems and was asked to look at the existing sales contracts program
(by far the most complex on the site) and the associated clerical procedures,
in order to prepare for a move to a more advanced computer.
I quickly decided that many of the difficulties with the
existing system were due to the fact that the sales staff did not have control,
most errors occurred in the systems/programming chain between the sales
managers and the computer, and once an application was running the sales staff
had no real insight as to what the computer was actually doing. I thought that
terminals would provide an opportunity to put the sales staff in control and
what was needed was a program which would work symbiotically with the sales
staff, so that they could tell it what they wanted in marketing terms – and the computer
would reply, use the same language, to tell them what is was doing on their behalf.
What I had done was to transfer my first-hand experience of
how humans process complex information into the design of an “intelligent”
computer program and was unaware that (in the 1960’s context) I had suggested
anything unusual. The idea was rejected because “sales staff can’t program
computers” and I might well have abandoned the idea at this point and come up
with the conventional solution my boss was looking for. However at this point I
was approached by the computer manufacturer who made the LEO computers, and
knew some of my work, to join a market research team looking at what customers
would expect from the next generation of computers.
Proposal for a
Human Friendly Information Procesor
This job meant talking to other large computer users and I quickly
discovered that many had applications which were hard to specify in advance and
which would benefit from an “intelligent” human interface. I also had contacts
with the research teams interested in the design of new computer architectures
and peripherals as I needed to be aware of what future computers might be able
to do at a reasonable cost. While it was
not my job to design hardware, within a few months I had suggested that by
adding some special registers to an IBM360 style computer and modifying its
instruction set, you could have a “white box” computer which operated at the
hardware level using the user’s own language and where it would be possible
for the human user to “look” inside and see what it was doing - a complete contrast to the normal "black box" computer where the inner working are incompatible with the way people think.
The working of this user-friendly computer processor can be
illustrated in terms of the model of human short term memory described above.
The computer memory (the sea of neurons) contains statements (patterns) such
as:
Hunger; Rabbit;
Rabbit pie.
The special registers, called The Facts, describe the
current context and contain:
Hunger & Rabbit
The Facts (the ripples on the sea of
neurons) act as a window and only match one statement in the computer memory so
Rabbit Pie
is added to The Facts (becomes a new ripple). As The Facts
have now changed the process is repeated. Of course the approach was not
designed for such simple tasks and in a real application The Facts might start
with a statement of concepts such as:
Customer=Mill Garage;
Product=Petrol; Quantity=1000
and the automatic pattern matching process would be repeated until a
Price appears in the Facts.
Working CODIL
Systems
The original note was expanded into a formal project
proposal to demonstrate that the idea actually worked. It was then approved by
the UK computer pioneers David Caminer and John Pinkerton (almost certainly in
consultation with Professor Maurice Wilkes at Cambridge) and I was rapidly
moved into research, with a generous budget, to produce a demonstration
simulation program. Two years later the simulation (although crude by later
attempts) was working better than the original proposal had predicted.
Unfortunately the company had merged with other to form ICL and the new
management streamlined the research to support the design and building of a
conventional computer series (the ICL 2900). The CODIL project was closed as
irrelevant to their plans and I became redundant.
ICL gave me permission to continue the work in a suitable
university environment and I naïvely thought all I needed was an income and
access to a computer. I had no idea how difficult it would be to get support
and funding for an unconventional project which, in many ways, questioned the
foundations of a significant technology and had very wide multi-disciplinary
implications. What I needed was a good manager, in an organisation with a strong
research pedigree which understood unconventional research. I also needed to
have plenty of opportunity to exchange ideas with people of many different
disciplines who didn’t take it for granted that technology was wonderful. In
retrospect it was a mistake to accept a senior post in a brand new university dedicated
to teaching technology, in a department
with no significant experience of research of any kind, and with such poor
facilities that for the first 8 years I had to simulate using a terminal with
punch card input to a batch processing computer system.
Once at Brunel University I rewrote the interpreter with a number of
improvements assuming that most of the applications would be in the data processing
area. I improved the way in which the software handles recursion and added
facilities to experiment with learning – for instance by allowing the software
to give a higher priority to more frequently used patterns. Later a number of
approximate matching facilities were added to allow the fuzzy matching of
patterns. However in terms of superior pattern processing the most significant
application was TANTALIZE.
At the time (mid 1970s) Artificial Intelligence research concentrated
in two main areas – board games such as chess (CODIL is best for tasks where
the rules and data structures are not precisely defined in advance) and solving
logical puzzles. I had used some simple logical puzzles as test data – to check
that the interpreter was working correctly but I soon discovered that almost
all the examples in the Artificial Intelligence literature could be represented
as patterns and when these patterns were presented to the CODIL interpreter it
would find the correct answer, nearly always in a very reasonable time. I found
it was possible to use CODIL to produce a very powerful heuristic problem
solver. TANTALIZE
demonstrated an unplanned aspect of CODIL which made it very much more
powerful. In CODIL all information is held as patterns and (with a few special
exceptions concerned with driving the display and managing the keyboard input)
there is no formal distinction between program or data. TANTALIZE was a
heuristic problem solver written in CODIL in which the patterns were designed
to be used as if they were a program. It asked the user questions about the
logical problem to be solved, and generated a series of patterns representing
the problem. In some cases it then used the learning facility
to give priority to the most helpful patterns – and then used the patterns to
generate the solution. Often in the minimum number of steps. At the time the magazine
New
Scientist published a weekly brain teaser called Tantalizer and on one
occasion TANTALIZE solved the puzzle over 15 consecutive weeks.
Of course you are wondering where the publication describing
this are – and at the time a number of papers were written and duly submitted
for publication. After several papers came winging back with rejection notes
saying something like “Too theoretical to ever work” when the paper included
details of actual results obtained I got so depressed I stopped trying.
Around 1980 a new mainframe computer was installed which
provided descent terminal facilities and I switched to using CODIL as an
education tool with classes of up to 125 students using CODIL driven teaching
packages. Personal computers were just coming in and the interpreter was
completely redesigned to run on a BBC computer. The BBC version, MicroCODIL,
was trial released in 1985 as a teaching package which could demonstrate different
way in which information could be processed. The package received very favourable
reviews in many different magazines, and a detailed paper, CODIL:
the architecture of an information language, was prepared for the
Computer Journal which was not published until 1990.
The Project
Collapses.
While the reaction to MicroCODIL and the acceptance of the
Computer Journal paper could have put me in a better position to get research
funding, and possibly move to a better venue, I was exhausted and urgently needed
a break (I had no opportunity to take a sabbatical in the 17 years I was at
Brunel University). I was also suffering post-traumatic stress disorder follow
the distressing illness and eventual death of my daughter Lucy. Then in 1987 a new head of
department decided that I was prime material for Maggie Thatcher’s early
retirement package as I did not have a supporting research grant. He also
thought that by writing software on a “toy” computer I was bringing the department
into disrepute. In the circumstance the easiest thing seemed to be to take
early retirement in 1988, arrange a year’s sabbatical in Australia, and
continue with the CODIL research when I returned to England. However on my
return I decided I would find it more rewarding to do voluntary work to support
the mentally ill, and switch my research interests away from CODIL and into
local History.
The link between
CODIL and brain research.
A few years ago I decided than something needed to be done
with the bulk pile of papers which represent the relics of the CODIL project. I
decided that before they were transferred to a skip I should check online to
see if they had any relevance to modern research. I quickly found that there
was nothing equivalent and that there was a massive black hole in brain
research with no acceptable models linking research at the neuron level with
high level human intelligence, and no reasonable evolutionary pathway to relate
use to animals. Everyone seemed to be digging deeper and narrower specialist
holes, and making exciting discoveries on the way, but none seems anywhere near
finding the “Philosopher’s Stone” that made human brains fundamentally different
to animal brains. There was so much information that it is almost impossible to
step back and see the wood for the trees.
Could CODIL somehow bridge the gap? It was designed as a
sequential digital processing language and brains are based on a neural net
where components work in parallel. I came up with a crude neural net model (From the
Neuron to Human Intelligence: Part 1: The “Ideal Brain” Model) and the
linkage is via the simple model of short term memory described above. The
neural model considers the network to ccontain linked groups of neurons which I
have called memodes and each memode can be considered to represent a concept.
These concepts can be as simple as a single nerve cell telling the brain “my
toe hurts” via recognising a “peacock” to abstract concepts such as “evolution.”
At the neural level all neurons, and
memodes (however complex the concept they represent) are equivalent. CODIL
gives a name to each concept (memode) and is a language for describing how the
concepts interact.
The fact that the CODIL/model describe above fit well with
the superior pattern processing model of Mark Mattson, would suggest that renewed
research around the CODIL model could be very profitable. One area which I will
address shortly, in an additional blog post related to the evolution of
intelligence where the model I have been developing on this blog significantly
extend the ideas put forward by Mattson. These differences are a natural
consequence of having an actual processing model and relate to some of the
weaknesses in the human mind (which stem from the fact that our neural net did
not evolve to do mathematically correct set theory) and the interaction of tool
making, language, learning/teaching and culture.
No comments:
Post a Comment