Parallel Machines: A Rebuttal

Crazy Eddie

August 9, 2008

Abstract
Because Turing Machines are abstractions existing without real time, the conclusions of Andrew

Boucher are false and misleading. This article explains why, discusses the limitations of the Turing

Machine concept, and discusses whether this matters in the question of AI.

Introduction

1
In Parallel Machines , Andrew Boucher argues that two Turing Machines running in parallel exibit compu-
tational activities that cannot be mimiced by a Universal Turing Machine. This, he contends, is due to the
time constraints introduced by parallel processing. To explain this, Boucher uses two illustration devices:
two Turing Machines communicating in an unspecied manner; and a shared printer on a typical computer
network.

The problem with Boucher's illustrations is that the rst violates the denition of Turing Machine and
therefore of computing device, the second does not actually result in what he claims. These issues speak to
the heart of Boucher's argument and are not just the consequence of poor choice of illustration on his part.

To show this I'll have to go over some denitions. Readers already familiar with what a Turing Machine
is, computability theory, and how a computer network operates may wish to skip some sections though they
do provide the foundation for my rebuttal of Boucher's argument.

Turing Machines and Computability

A Turing Machine is an abstract concept of a computing device. It is able, through manipulation of a tape,
to perform any computation that is possible to perform. It is necessary at this point to note that this does
not mean it is able to calculate anything at all. There are answers and problems that a Turing Machine
cannot deal with. These problems are uncomputable.
I am not going to dive into the formal denition of a Turing Machine for that would raise the content of
this article above the level of much of my audience. Those with the background to be able to understand
the denition can nd it just about anywhere on the Internet, including wikipedia. I am going to focus on
layman denitions. They are what was used in the original and are adiquate for my needs.

A Turing Machine is composed of four parts: an innite in length tape, in both directions, that is divided
into discrete cells in which can be read and written the symbols of the machine's language; a head that
reads or writes the content of the cell it is located on; a state table that species what happens when a
particular symbol is found on the head in the state the machine is in; and a register that contains the current
state. Even this denition may be superuously complex for the task at hand and you can think of a Turing
Machine as a set of states, actions, and decisions based on input and given that it can store its state and the
data it is working with.

1 http://www.andrewboucher.com/papers/parallel.htm
1
An example at this point would be good. Consider a problem of simple boolean addition. There's more than
one type of boolean addition and I'm going to pick mod 2 so that 1+1=0, 0+0=0, and 1+0 or 0+1 = 1. I
wish to compute the addition of an arbitrary length of binary digits. In order to develop a Turing machine
capable of this I need two states in addition to the standard start and end, some simple I/O operators, and
a three symbol language that contains a symbol for 0, one for 1, and a third for the string terminal. Here
is that machine drawn in a basic state diagram.

The above machine starts in state S0. If it reads a 0 from the tape it stays in state S0 and moves to the
right on the tape. If it reads a 1 it switches to S1 and moves to the right. If it reads the END character it
writes a 0 on the tape and switches to the HALT state. If the machine is in state S1 it will perform similar
actions but write a 1 on the tape before moving to HALT.

It is important that Turing Machines are purely algorithmic. The object is to be able to hand some
instructions and some data to someone and they can perform the steps without any sort of interpretation to
arrive at the calculation. It is a mindless process that works the same every time with the same input. I
can hand it to a person, or feed it into a machine that can read Turing Machine construction language and
the result is the same even if they don't work at the same speed.

Anything that can be calculated by such a process is computable. Quite a lot of things are computable but
there is one important thing that is not, and that is some of the members of the real number set. Integers,
fractions, and numbers that can be computed to some length of digits are all computable numbers, can
be represented in some way and calculated by an algorithmic process. These are the only numbers that a
Turing Machine, and thus any computer, can work with, compute, or accept as input.

It is an important nding by Turing (the man who invented the Turing Machine) that there exists a Turing
Machine that can accept, as input, a description of another Turing Machine, some inputs, and when it is
run it will generate the same output that the described machine would have being run on its own. Such
a machine is called a Universal Turing Machine (UTM) and ANY Turing Machine can be run on such a
machine. Computers are examples of a type of Universal Turing Machine though they do not have innite
storage space and are actually complex constructions that are rather parallel in nature.

However, there are certain things that cannot be computed by this UTM. For instance there is no way to
construct a UTM that can tell if any given Turing Machine will ever stop its calculations. A Turing Machine
can enter an endless loop and though there are ways to construct a UTM that could recognize some Turing
Machines that will do this, there is no way to construct one that will recognize any and all non-halting
Turing Machines that might be fed into it.

It is also important to note at this time, before I move on to Boucher's Machines, that the fact that a UTM
will run any Turing Machine fed into it, there is no guarantee that it can run some other kind of machine. If
something is being done by a machine that can not be done in a Turing Machine then it is not a computation,
the machine is not a Turing Machine, and it is not implementable in a UTM. For example, if a real number
that is not computable is involved in a process that generates an output, this is not a computation and
cannot be done in a Turing Machine.

2
Boucher's Machines

Boucher's Machines are interesting creatures. His description of the machines seems straight forward enough,
and it is, but he left out many details and it is in these details that the problem resides. To see this I will
diagram them both and show that he leaves important information out that when entered into the system
creates a set of machines that generate outputs that cannot be duplicated in a Turing Machine. To start,
here is his exact description:

To see this, model communication between two Turing machines as the ability of one to
read the contents of the square where the other's read-write head happens to be at a particular
moment in time. Now consider a parallel Turing machine consisting of two Turing machines A
and B. A's program tells it to move rightward across its argument until it hits the rst 0, and
once it is there, communicate with B, by reading what B is reading at that instant in time; A
then returns this value as the "answer." B meanwhile simply alternates its head between a square
which contains 0 and a square which contains 1.

To expose the fallacy in Boucher's argument it is necessary to look at the two machines individually before
composing a network out of them. Let's start with the rst. An initial attempt at describing this machine
might look like this:

This looks like a Turing Machine but it has a couple of serious problems. For one, it doesn't halt. We
can tell that because it simply does not have a halting state. This, in turn, means that it also doesn't
actually compute anything. However, it's obviously being used to calculate something for the other Boucher
Machine. What is actually being calculated is based on the time it takes to op the head around in the
particular machine implementing the above Turing Machine. But here in lies a very serious problem! Turing
Machines don't work in real time. Turing Machines are abstract machines working in a universe of innities:
an innite storage space; an innite amount of time in which to work (for computations can take an arbitrary
number of steps); and an innitely short period, or arbitrarily large one, to do state switching and mechanical
motions. In working with Turing Machines we have no idea how long it is going to take this above described
machine to op around and we don't care...or wouldn't if we were actually computing something.

So now it is important to discuss how we can turn this machine into something that is a Turing Machine,
and a Machine that computes something. In Boucher's description of the process going on it becomes more
clear what that machine must be. We are attempting to implement something that will take it's input and
return a 0 or a 1 depending on that input. That input is a variable based on time. What is being calculated
by this machine is actually the equation oor(T / R) mod 2, or in average terms, divide the continuous
variable 'T' by 'R' and return 0 if that number is even, 1 otherwise. What is R? That is the time it takes
to do the head movement from one cell to the other. What is T? T is likewise a variable representing a
time.
3
Now why have I taken you through all of this? Boucher mentions this formula already, or one pretty
similar to in, in his article. Am I not just proving his point? Actually no. Boucher did not take us
through a full description of his machines and therefore did not expose R at this point. What we see here
is that the continuous variable timeif it is indeed continuousis already an issue and for the very argument
he uses against a UTM implementing the parallel construct he illustrates in the above quote. There are
some values for R that any UTM can work with, and for those values of R any UTM can implement this
calculationthe Turing Machine that calculates this computation is actually quite complex and that's why I
have chosen not to draw it. However, for some values of R there is NO Turing Machine that can implement
the calculation. For these machines, R cannot be computed and is therefor not possible to generate or
store in an algorithmic manner. The value cannot even be represented by anything other than the machine
that it is implemented by.

The variable T has dierent problems. The problem with T is that only some values of T can be represented
and therefore only some values of T that can be the input to this function.

Remember, to be a Turing Machine it needs to represent an algorithmic process and that is not what we have
here. Once we introduce the time that it takes the machine to do something into the value it is computing,
it is no longer possible, in some cases, to do the same calculation in a UTM. Furthermore, in order to be a
Turing Machine the inputs to the system must be representable or calculatable and again, that is not what
we have here. We will discuss the implications of this later for as we can see here, this Boucher Machine is
able to calculate something that a Turing Machine cannot.

Now lets look at the other machine. Does it have the same problem? Here is a drawing of that machine:

That seems straight forward enough even with the mystery process in quotes. If, in fact, all that was being
done in Boucher's problem was that the machine was reading to the end of a string and then printing some
value out then this could be implemented by a UTM without any issues. But that isn't what is happening
here. The value that is being printed is actually something that depends on time, this time being the input
to the machine explained above. What this means is that this machine itself is actually calculating a time
in addition to its other, main output. So again, for the same reasons, we are no longer discussing Turing
Machines.

In short, in the case of each of these individual serial machines we can say, in the words of Boucher about
parallel machines, ...it is more accurate to say that it simply does not compute a function at all. On denite
input there is no denite output. It is not an eective procedure. Except that in the case of these machines
denite input is not at all guaranteed.

In the end, this means that Boucher's introduction of time, which he is correct to do, changes the problem
so that it is no longer a discussion about Turing Machines (if time is in fact continuous, if not then none
of these problems would appear). It was this introduction and not the introduction of parallelism that
caused any non-determinism or uncomputability for that was already existant in the machines he networked
together. Because the output to the rst machine is dependent on a process that is not necessarily com-
putable, the use of that machine causes indeterminism for what this machine will do is not something we can
compute or necessarily determine. The second machine has the same problem for it relies on the possibly
uncomputable time it takes to perform its task as input to the process that generates its output, a process
that is not always computable. Neither of these machines can be implemented in a UTM, so the fact that
the two together also can't is not at all surprising.

We can now refute the claims made by Boucher about this particular example network of machines. Under
what conditions are the two machines both describable as Turing Machines? The one with more constraints
is the machine that calculates the oor of the ratio of T/R. In order for this calculation to be possible by
4
a computing machine T, R, and T/R must all be computable numbers. For the other machine, T is its
output and so all operands used to compute it must be computable, namely the time it takes to look at the
tape and move the head (r) and the length of the supplied string (|S|). Once these constraints on these
variables are met, the entire network equation, X = oor(|S|*r/R) mod 2, is computable because both r and
R are. This then can be implemented on a Turing machine.

The fact that time exists is an issue that is dealt with in the implementation of computing machines. There
are many cases when time is important and one of those cases is when you are trying to get many computers
to talk with each other. Another case of time being important is when you want to get something done by
a particular time interval. The fact that any Turing Machine can compute something doesn't mean that it
is useful to use any device that implements a Turing Machine to do so. If it takes one Turing Machine 20
minutes to calculate 2+2 then trying to ask it what the square root of PI is at 100 digits of precision is going
to take quite a long time. It can, to be certain, but such a device would not be practical. The only thing
that Turing Machines and computational theory speak about is whether it is possible that a machine can
calculate a given problem, not whether it will do so within the space and lifetime of the Universe much less
when you want it to. When you introduce time into the problem you nd that not any old Turing Machine
will do.

Computer Networks

Computer networks must deal with a lot more issues than time. Computer networks have to also work in
environments in which they can be interfered with. Place a powerful electromagnetic eld near a computer
cable and interesting things are going to happen. Stray electrons can come carening through and change
a 0 into a 1 and this can happen at any time in the communication. These are also things that Turing
Machines do not have to deal with! How do computer networks even function?

Computer networks work because there are many constraints that the system works with that allow it
to absorb error. Packets sent about are marked with data that allows the machines along the path of
communication to tell if the data is what was sent. The protocols used in talking back and forth often
contain methods for the destination machine to inform the sender that data is missing, and methods to
tell that it's missing to begin with. The hardware itself is built in such a way as to limit interference; for
instance, the cable used in many computer networks is twisted tightly so that it doesn't interfere with itself
(bleedover of data from one cable into the other because of induction).

We haven't introduced time yet! We don't have to introduce parallelism yet either. If two machines worked
in such a way that one did its task, sent data to the other through a network, and then it did its task we
would still have issues to contend with. Within the description of the two systems themselves we could
not tell how many times it would be necessary to send dierent packets of information through the network
cable. We would have no way of knowing how much time, exactly, it is going to take for the process to
complete. As such, the whole thing can't be described as a Turing Machine, even though it is serial, because
we cannot describe all the information that is input to the problem. The outside environment can aect
the process and there's no description of this environment, in detail, within the system. Because of this the
2
process is undetermined within the description , and we're still working in a serial system!

However, we can still say that the whole system implements a Turing Machine. The way we do this is by
abstracting certain details. We know that, short of a total loss of communication, that the redundancy
and error checking that takes place within certain parts of the networking layer that the data sent is the
data recieved. We don't know how long it took, how many times the packets had to be sent, and many
other practical details about the process of getting data from one point to another. What we do is just
assume it will get there, Turing Machines are not practical things and don't have to deal with real world
problems. Once we abstract out the error correction and assume a working system in which all control
paths are clear it is at that point that we can say something implements a Turing Machine.

2 This is the last point I will clarify within the system and will simply call it undetermined from now on. I am not
making a statement about the deterministic vs. non-deterministic nature of the universe, only about the inability to determine
what will happen within the sphere of description while still having to deal with input that lies outside that description.
5
In other words, computer networks, even computers themselves, must operate in the real world in which there
is electric interference, limited storage space, and time constraints and Turing Machines do not. However,
computers implement Turing Machines, except for the space constraints (so they're limited in the size of
problem they can compute), because they internally deal with the errors associated with being real world
devices and implement a higher level abstraction on top of these error correction systems. Sometimes this is
done at the hardware level and the computer hardware itself implements some amount of internal integrity
checking that the rest of the computer doesn't have to deal with. Sometimes it is in the software level and
in those cases the input to the system is just considered unknown and then data that makes no sense is
simply disgarded. When attempting to describe a practical, real world, system as a Turing Machine you
must pick the correct level to do that in.

Boucher's Printer Problem

Would Boucher's printer problem, that you can't tell which computer will get the printer rst, be an under-
terminable problem in a computer network? Yes, it would because of the errors associated with network
communication. Would it be undeterminable in a network of Turing Machines? That's a good question.

First we must decide what it means for Turing Machines to operate in parallel. There is no real time
associated with Turing Machines but we could say that each operation takes one unit of time. Boucher would
object to this right away but we'll get back to him; Turing Machines don't have any other type of time and
so we have to pick something that doesn't purposfully introduce indeterminism for the discussion of parallel
machines to make sense .
3 His objections are to the point though because now we can see that by removing
any type of erroneous input that isn't declared as input, and removing any indeterminism introduced by an
unknown segment of time that stu takes place in, that the entire system is VERY describable and we can
determine exactly what will occur at any given unit of time.

So then, on the one hand it must be admitted that, short of calculating the location and trajectory of every
particle in the universe (if that's even possible), we can't decide who exactly will get to print to the printer
rst, but on the other hand this is not a problem of Parallelism but of the introduction of real world issues
into a theoretical framework. It might make sense then to try and create a problem here that might make
sense of the issues on hand. So instead of trying to decide what and who we'll specify. We want computer
A to print a page before printer B prints its page and we want this to happen after 4:58PM and before
5:02PM, which is 20 minutes after telling them both to set about two parallel processes within a margin of
error of 2 minutes because we're going to start that process at 4:40PM.

We need one more constraint. We want this to happen as described and within the time constraint with a
99.98% certainty. This means that for every 10,000 attempts to do this the system might fail twice. This
is a very stable system requirement. Two failures out of ten thousand would be very dicult to acheive.

How would we do it though? Well, we describe the algorithm in a perfect world. The computations that
must occur are implementable by a Turing Machine; we know this already. Once we know what must be
done we can start placing estemated time constraints on the various operations that must occur, within a
margin of error that we will calculate based on the one we were given. We will have to nd hardware that
can match the requirements established by this procedure. We have to do this for both of these machines
individually, parallelism is not even considered yet and as you can see, not any old Turing Machine will do.

Parallelism will introduce a problem though for we know that the process must occur within a particular 4
minute window. This means that system B must print its document no later than 22 minutes, and system
A no sooner than 18 minutes, after the process is started. There are now more constraints introduced into
the problem because the two systems must do something together within a particular window. System A

3 Furthermore, the use of this unit of time is quite appropriate in discussing Turing Machines for such discussions often
involve the question of complexity, in which time is the order (count) of operations that must take place for an algorithm
(a computation) to complete. Since algorithms usually involve numerous operations, the complexity of the algorithm has a
stronger inuence than the real time of any given operation on the time it takes to complete the computation. Someone
interested in this could look up Big O Notation as a starting point.

6
needs to be completely nished, accounting for any errors that can take place, and allow B to get its job
done, accounting for error, as well.

Not a symple task. It might not even be possible. The constraints may be too particular for the hardware we
have. However, these are real world constraints imposed by needs outside the system. They do not aect the
computations occuring. Those computations can be done in any Turing Machine and it is by absorbing the
errors and constraints while providing an abstract computation machine that computers implement Turing
Machines.

Boucher's Statements on AI

Following the description of his Turing like machines and his explaination that time introduces problems in
computability, Boucher continues to question the goals and methods of Articial Intelligence research. Many
of his assessments are, in fact, correct. We do not know whether the brain implements a logic system or
not. If it does then, using the denition of logic we have, it is an implementation of parallel Turing
machines. If the brain is not a logic system then it is doing things that are not computations and thus the
Turing machine concept is incompatible with the brain in every way.

One way we could falsify the assertion that the brain is a Turing machine would be if we could come up with
anything that is not computable. The problem here is one of classication though. We need to establish
that it is impossible for something we think, or some way in which the brain works to control some aspect of
our anatomy, is impossible to compute. Such proofs are not always possible but some thing can be proven
to be impossible, such as the answer to the halting problem: the halting problem is impossible to solve.

We can exercize our logic muscles and think about this for a bit though and ask ourselves if we really can
come up with anything that cannot be computed. We could just try one example to illustrate the problems
with the assumption that we can: think of a number that a computer could not calculate. I'll start and just
rattle o a bunch of random numbers:

(.01793926749604742982085762098720452458764768).

So, did I succeed? It seems on rst glance that I did. That sequence of numbers is, as far as I can tell,
completely random. I just let my ngers nd some keys and then deleted the erroneous ones. Could I come
up with the same number in the same way? Probably not, but that's not really an important point. The
question we must ask is whether a computer could do something similar, and not just similar but at least as
random as I just did. The question we have to ask is whether that number was actually created randomly,
or just in a way that appears to be random. Computers can be made to output numbers that appear, for
all practical purposes, to be just as random as those numbers appear to be. How do we know that my brain
did it any dierently?

The answer is that we obviously do not know. We can't test it either because once that number was created
the fact that it terminates means it is a rational number and is therefor computable. A computer can
represent it and be made to come up with it upon request; in fact it just did as it rendered the image you are
reading! We are left with a question very similar to the debate between determinism and non-determinism
as a basic construct of the universe; do things happen in an ordered manner or is the universe actually
completely random?

In Dennett's book on free will, Freedom Evolves, the question of determisim is brought up. Dennett shows
quite well that a random universe gives us nothing that a pseudo-random universe could not. This ability
to conceive of options and decide between them, this thing we call free will, is the same thing either way. In
either case, we must conceive of available (and not available) optionswe must calculate possibilitiesand
then decide between them in some way, either calculated or random. What this indicates is that at least the
perceptions we have about ourselves are not outside the realm of being implementable by Turing machines;
everything we are seems as though it can be a calculated engine.

7
It also makes great sense that our brains would be logic engines. Even though we often act in illogical
manners, the reasons we do seem to be entirely predictable and logical from a calculation perspective. The
formation of false beliefs is governed by poor inputs. The holding of false beliefs in the face of their obvious
falsity can be the simple consequence that our brains where programmed for an utterly dierent problem
than it is being forced to adapt to now that we have words and ideas. Like trying to display images in
pure text, our brains are capable of inventing and conversing in symbols but sometimes it's limitations make
themselves plain.

In short, although it is not yet proven that everything we are can be described in a formal logic system it
seems that it is mainly wishful thinking and ego that asserts the other direction. There doesn't seem to be
any reason to believe it is not the case that our selves, souls, whatever you want to call them, are completely
logical in their own right. I have yet to see any convincing evidence that our brains are not logic systems
and there seems to be plenty of reason to believe they are; simply considering the world they had to evolve
in is, I think, a great argument that the brain would HAVE to be rational while sometimes accounting for
your standard bug.

Concluding Remarks

I have argued that Boucher's machines were not Turing machines to begin with because time, a possibly
uncomputable range of values, was a part of the calculations they performed. Since time in Turing machines
is simply the discrete steps with no real-world equivelent, Boucher had stepped outside the range of discussion
in which Turing machines have meaning. It was this, and not the introduction of parallelism in the system
that caused the nal parallel Turing machine to be non-deterministic. Although it is not established if
the brain is a rational machine it seems reasonable to believe it likely to be and Boucher has not provided
a good argument that they are not. His assertion that there could be some sort of logic not implementable
by serial computation devices has not been shown.