Intelligent Game Playing Using Neural Networks

Table of Contents:
1. The Abstract.
2. Brief Overview of Artificial Intelligence.
3. Knowledge versus Intelligence.
4. The Turing test.
5. Neural Networks
6. Interactive Computer Games.
7. Computer Game Genres.
8. Role playing and enhancing the performance by integrating AI.
9. Use of Neural Networks for Search Problem Formulation
10. Games Making Interesting Use of Neural Network Architecture.
11. Creating a Learning Agent.
12. Problems with using Neural Networks in Games.
13. Recent Developments and Ongoing Research.
14. The Conclusion.
15. References and Bibliography

Abstract
Artificial Intelligence is the study of how to make
computers do things which, at the moment, people
do better. Game playing and Theorem proving share
the property that people who do them well are
considered to be displaying intelligence. Artificial
Intelligence opens a new dimension of introducing
intelligence inside machines.
Initially the Computers could perform well at task
such as playing games like Tic-Tac-Toe, where the
numbers of solutions path are less. It was thought
that this process required very less knowledge and
could therefore be programmed easily. But even in
simple games like Chess Playing there are around
10e100 possible moves for every move. So even to
finish the first move would require our whole life
time.
A neuron is a cell in the brain whose principal
function is the collection, processing and
dissemination of electrical signals. The brains
information-processing capacity is thought to
emerge primarily from networks of such neuron.
For this some of the earliest AI work aimed to
create artificial neural network. Researchers in AI
and statistics became interested in the more abstract
properties of neural networks. Such as their ability
to perform distributed computation to tolerate noisy
inputs, and to learn. Neural networks remain one of
the most popular and effective forms of learning
system.
Hence we use the concept of Neural Networks in
game playing, making games more robust by
inducing learning from experience ability. The
BC3K Artificial Intelligence & Logistics, AILOG,
engine, uses a neural net for very basic goal
oriented decision making and route finding when
navigating the vast expanse of the game's galaxy.
In this paper I explain the concept of game playing,
different genres, role of neural networks hence AI
and also the current research work going in the field
of game playing and making games more
intelligent.

Introduction
Brief Overview of Artificial Intelligence:
The danger from computers is not that they will
eventually get as smart as men, but we will
meanwhile agree to meet them halfway
Artificial Intelligence is the study of how to make
computers do things that at the moment humans do
better.
Artificial
Intelligence
deals
with
incorporating Intelligence inside machines so that
their functionality is enhanced and optimized.
It is the science and engineering of making
intelligent machines, especially intelligent computer
programs. It is related to the similar task of using
computers to understand human intelligence, but AI
does not have to confine itself to methods that are
biologically observable.
Artificial Intelligence focuses on tasks
that we do every day also called as common sense
tasks. Tasks can be categorized into three types:1. Mundane Tasks: These are the tasks that
include perception, natural language
understanding, and commonsense reasoning.
2. Expert Tasks: These are the tasks that
require carefully acquired expertise.
Examples of such task include Engineering
design, scientific discovery, and Medical
diagnosis.
3. Formal Tasks: These are the tasks that
involve game playing, logic building, and
mathematics etc.
Expert task require complete knowledge of a certain
thing. These tasks are easy to be achieved in a
machine, whereas the tasks such as perception are
pretty hard to achieve. As a result the problems
where AI is now flourishing most as practical
discipline are primarily the domains that require
only specialized expertise without the assistance of
commonsense knowledge.
Artificial Intelligence problems span a very broad
spectrum. One of the very few hard and fast results
to come out of the first three decades of AI research
is that intelligence requires knowledge.

Knowledge versus Intelligence:

Knowledge is ones grasp, comprehension of a
subject. In AI terms knowledge possesses certain
less desirable properties:
1. Knowledge is voluminous.
2. It is hard to characterize accurately.
3. It is constantly changing.
4. It differs from data by being organized in a
way that corresponds to the ways it will be
used.
Knowledge must also possess certain essential
properties:
1. Knowledge captures generalization.
2. It must be understood by people who
provide it.
3. It can be easily be modified to correct errors
and to reflect changes in the world and in
our world view.
4. It can be used in great many situations even
if it is not totally accurate or complete.
5. It can be used to help overcome its own
sheer bulk by helping to narrow the range of
possibilities that must be considered.

Neural Networks:
A neuron is a cell in the brain whose principal
function is the collection, processing and
dissemination of electrical signals. The brains
information-processing capacity is thought to
emerge primarily from networks of such neuron.
For this some of the earliest AI work aimed to
create artificial neural network. Researchers in AI
and statistics became interested in the more abstract
properties of neural networks. Such as their ability
to perform distributed computation to tolerate noisy
inputs, and to learn. Neural networks remain one of
the most popular and effective forms of learning
system.
Neural networks are composed of nodes or units
connected by directed links. A link from unit j to
unit i serve to propagate the activation aj from j to i.
Each link also has a numeric weight W j,I associated
with it, which determines the strength and sign of
the connection. Each unit I first computes a
weighted sum of its inputs.
Ai= from (j=0 to n) Wj, iaj

Turing Test:
Turing test is the test in which one person plays the
interrogator who is in separate room from the
computer and the other person. The interrogator can
ask questions of either the person or the computer
by typing and receiving typed responses. However
the interrogator knows them only as A and B and
aims to determine who the person is and who is the
machine. The goal of the machine is to fool the
interrogator into believing that is the person. If the
machine succeeds at this, then we will conclude that
the machine can think. The machine is allowed to
do whatever it can to fool the interrogator.
The more serious issue though is the amount of
knowledge that a machine would need to pass the
Turing test.

A Biological Neuron and a Perceptron

A preceptron models a neuron by taking a weighted
sum of its inputs and sending the output 1 if the
sum is greater than some threshold value. The
inputs (x1, x2, x3,..xn) and connection weights (w1,
w2,w,3,wn), are typically real values both
positive and negative. The perceptron itself consists
of the weights, the summation processor, and the

adjustable threshold processor. Learning is a

process of modifying the values of weights and the
threshold. It is convenient to implement the
threshold as just another weight w0. The weight can
be thought of as propensity of the perceptron to fire
irrespective of its inputs. A perceptron computes a
binary function of its input. Several preceptron can
be combined to compute more complex functions.

A perceptron

Games, Genres, Roles

Interactive Computer Games:
Although one of the fundamental goals of AI is to
understand and develop intelligent systems that
have all of the capabilities of humans, there is little
active research directly pursuing that goal. I
propose that AI for interactive computer games is
an emerging application area in which this goal of
human-level AI can successfully be pursued.
Interactive computer games have increasingly
complex and realistic worlds and increasingly
complex and intelligent computer-controlled
characters. In this paper, I further motivate the
proposal of using interactive computer games,
review previous research on AI and games, and
present the different game genres and the roles that
human level AI could play within these genres. I
then describe the research issues and AI techniques

that are relevant to each of these roles. My

conclusion is that interactive
Computer games provide a rich environment for
incremental research on human-level AI.
In late 1997, researchers started to look for another
application area, one where they could use what
they learned from computer generated forces and
pursue further research on human-level intelligence.
They have found it in interactive computer games.
The games I am talking about are not Chess,
Checkers, Bridge, Othello, or Go, which emphasize
only a few human capabilities such as search and
decision making. The types of games I am talking
about use the computer to create virtual worlds and
characters for people to dynamically interact with
games such as Doom, Quake, Tomb Raider,
Starcraft, Myth, Madden Football, Diablo,
Everquest, and Asheron's Call.
Human-level AI can have an impact on these games
by creating enemies, partners, and support
characters that act just like humans. The AI
characters can be part of the continual evolution in
the game industry to more realistic gaming
environments. Increasing realism in the graphical
presentation of the virtual worlds has fueled this
evolution. Human-level AI can expand the types of
experiences people have playing computer games
by introducing synthetic intelligent characters with
their own goals, knowledge, and capabilities.
Human-level AI can also recreate the experience of
playing with and against humans without a network
connection. From the AI researcher perspective, the
increasing realism in computer games makes them
an attractive alternative to both robotics in the real
world and home-grown simulations. By working in
simulation, researchers interested in human-level AI
can concentrate on cognitive capabilities and
finesse many of the pesky issues of using real
sensor and real motor systems - they must still
include some sensor modeling to get realistic
behavior, but they don't have to have a team of
vision researchers on their staff.

Computer Game Genre:

In this section we review the major genres of
computer games to which human-level AI is
relevant. For each of the genres in this section, I
discuss the different roles that human level
AI can play: enemies, partners, support characters,
strategic
opponents,
low-level
units
and
commentators. Other roles are possible, but these
are the most common.

Action Games
Shortly after landing on an alien surface you learn
that hundreds of your men have been reduced to
just a few. Now you must fight your way through
heavily fortified military installations, lower the
citys defenses and shut down the enemys war
machine.
These games vary in the perspective that the human
has of their character, be it first-person where the
human sees what the character would see, or thirdperson, where the player looks over the shoulder of
the character. Popular examples include Doom,
Quake, Descent, Half-Life, and Tomb Raider.

In pure action games, AI is used to control the

enemies, which are invariably alien monsters or
mythical creatures. Realism in graphics has been
the point of competition for these games;
however, the graphics race seems to have run its
course, with better AI becoming the point of
comparison among the two of these.

Role-Playing Games
Immerse yourself in aworld, where nations hang
in the balance of your actions, dark prophecies test
your resolve, and heroic dreams can be fulfilled at
last. Baldurs Gate
In role-playing games, a human can play different
types of characters, such as a warrior, a magician,
or a thief. The player goes on quests, collects and
sells items, fights monsters, and expands the
capabilities of their character (such as strength,
magic, quickness, etc.), all in an extended virtual
world. Example games include Baldur's Gate,
Diablo, and Ultima. Recently, massively
multiplayer role-playing games have been created
where thousands of people play and interact in the
same game world: Ultima Online, Everquest, and
Asheron's Call.
In both types of role-playing games AI is used to
control enemies, like in action games, partners
who travel and adventure with the Players and
also supporting characters, such as shopkeepers.
The massively multiplayer games provide an
additional opportunity to use AI to expand and
enhance the player to player social interactions.

Strategy Games:
Players must successfully construct and rule their
medieval empire while engaging in real-time
tactical warfare over land, sea, and air.
AI is used in two roles: to control the detailed
behavior of individual units that the human
commands, and as a strategic opponent that
must play the same type of game against the
human. The AI needs of the individual units
differs from the enemies and partners of action
and role-playing games because they are not
meant to be autonomous but are meant to be
good soldiers who follow orders.

Team Sports
Welcome to Madden NFL 97, the game that
captures the excitement of a 30 yard touchdown
pass, the strategy of a well executed scoring drive,
and the atmosphere of a crisp autumn afternoon in
the stadium. Madden NFL 97.
AI is used in two roles that are similar to the
roles in strategy games, the first being unit level
control of all the individual players. Usually the
human controls one key player, like the
quarterback, while the computer controls all the
other members of the team. A second role is as
the strategic opponent, which in this case is the
opposing coach. One unique aspect of team sport
games is that they also have a role for a
commentator, who gives the lay by play, and
color commentary of the game.

Role Playing and Enhancing the

Performance by Integrating AI:
In a game AI plays different role like, Tactical
Enemy, Partner, Support Character, Story Directors,
Strategic Opponents, and Units Commentators etc
In early games, the enemies were made more
challenging, not with improved intelligence, but
with bigger guns, tougher hides, and superior
numbers. They also usually cheated by being able

to see through walls or out of the back of their

heads.
More recently, games such as Half Life, Descent 3,
Quake III, and Unreal Tournament have
incorporated path-planning and many tactics that
make these enemies more human-like. Research has
concentrated on building enemies for Quake II that
have the same strengths and weaknesses as human
players. To beat them, you have to out-think them
as much as you have to outshoot them. Soar
Quakebot is essentially a real-time expert system
that has multiple goals and extensive tactics and
knowledge of the game. It is built within the Soar
architecture and has over 800 rules. While
exploring a level, it creates an internal model of its
world and it uses that model in its tactics, to collect
nearby weapons and health, to track down an
enemy, and to set ambushes. It also tries to
anticipate the actions of human players by putting
itself in their shoes (creating an internal model of
their situation garnered from its perception of the
player) and projecting what it would do if it were
the human player. Building human-level enemies
for these games requires solving many general AI
problems and integrating the solutions into coherent
systems. The enemies must be autonomous. They
must interact with complex a dynamic environment,
which requires reactive behavior, integrated
planning, and common sense reasoning. As they
advance, they will also need models of high-level
vision that have the same strengths and weaknesses
as humans. One common complaint among game a
player is that the enemy AI is cheating, which
destroys the game playing experience. For example,
if the human is in a dark room, the AI would be
cheating if it could easily sense, identify, and locate
the human. However, if the human is back-lit by a
bright hall, the AI enemy should be able to easily
sense and locate the human, but possibly not
identify him. This is important for game play so that
the same tactics and behaviors that work well with
humans work well with AI enemies. There are
many other applications of AI to building intelligent
enemies. Because of the extended geography of the
environment, they must navigate, use path

planning, spatial reasoning, and temporal

reasoning. As the games become more complex, the
enemies will need to plan, counter-plan, and adapt
to the strategies and tactics of their enemies, using
plan recognition and opponent modeling
techniques, and learning. Their responses need to be
within the range of humans in terms of reaction
times and realistic movement. One can even
imagine adding basic models of emotions, where
the enemies get mad or frustrated and change
their behavior as a result.

Games Making Interesting Use of

Neural Network Architecture:
Age of Empires
Age of Empires uses limited learning to
augment a pretty straightforward approach to
the AI. The computer knows how to do certain
types of strategies that are common, including
things like frontal assaults. When you play any
of campaign scenarios the first time, the game is
even.

Use of Neural Networks for Search Problem

Formulation:
Neural networks give us the ability to do enormous
amount of calculations. It makes use of
mathematical functions for solving a problem, so its
result is definite. There are basically two different
types of neural network. Feed-forward network
and Recurrent network. The response of the
network to a given input depends on its initial state
which, may depend on previous inputs. When faced
with a problem neural networks use weighted
functions and apply the most optimum solution to
the problem.
Considering the problem of playing Tic-Tac-Toe
game, we would go about solving a problem in the
following manner:
1.
2.
3.
4.
5.
6.
7.
8.
9.

Define the initial state.

Define board, positions of pieces.
Decide whose turn is it.
Define the legal moves.
Determine when the game is over.
Calculate the result.
Usually win, lose, and draw.
Calculate utility or payoff function.
Numeric value for the outcome of a game.

However, as a human, you carry over

information about the scenario when you replay
it the next time. So, we let the CPs do the same
thing. They remember where you attacked them
or they attacked you, etc. We also let the CPs
remember your general playing tendencies so
that they can improve playing against you in the
randomly generated games. This has helped the
quality of the AI out a lot. Well enough, in fact,
that we'll be able to ship the game with an AI
that doesn't cheat. However, we may do a
"Doom-style Nightmare mode" where the AI
overtly cheats by way of getting a resource
boost at the start just to pound on people who
like that kind of thing. The scenario replay
learning feature was actually created out of a
desire to have people who didn't win the first
time get a different play experience when they

replayed. The goal here was to remove the need

to just optimize your strategy well enough so
that you can eventually beat the scenario with
the same thing you tried to do the last five
times. .

Battlecruiser: 3000 Ad (Bc3k):

The BC3K Artificial Intelligence & Logistics,
AILOG, engine, uses a neural net for very basic
goal oriented decision making and route finding
when navigating the vast expanse of the game's
galaxy. In some cases a supervised learning
algorithm is used and in other areas, an
unsupervised one is used. It also employs some
fuzzy logic where a neural net would not
suffice. Some aspects of a SOM model and
various Back propagation variations are also
employed. Virtually every NPC (non-player
character) in the game is driven by a neural
network. This includes each of the 125 crew
members of your ship, which is quite
impressive technically. The computer opponents
also use neural networks to guide negotiations,
trading, and combat
So, AILOG, contrary to popular belief, and
hyped just as much as the game itself,
incorporates several basic AI and NN
techniques in several of the engines. Some
training (for supervised learning algorithms) is
basically done at a low level where ship and
personnel characteristics are involved as well as
route finding in the galaxy, route finding within
the ship, engineering repair assignments,
internal ship combat with marines and
intruders/escaped prisoners. It also involves
threat identification and decision making.

Creating a Learning Agent:

behavior of each agent, as otherwise it will be

simple to undermine any tactics used during the
game by simply picking off the nave agents
one-by-one.
A learning agent is composed of a few
fundamental parts:
A learning element, a performance element, a
curiosity element (or 'problem generator'), and
a performance analyzer (or 'critic').
The learning element is the part of the agent
which modifies the agent's behavior and creates
improvements. The performance element is
responsible for choosing external actions based
on the percepts it has received (percepts being
information that is known by the agent about its
environment). To illustrate this, consider that
one of our agents is in the woods playing
paintball. He is aware of an opposing paintball
nearby. This would be the percept that the agent
responds to, by selecting an action - moving
behind a tree. This choice of action is made by
the performance element.

The

performance analyzer judges the

performance of the agent against some suitable
performance measure (which in this case could
be how close the agent is to being hit by the
enemy or how many enemies have been hit).
The performance must be judged on the same
percepts as those received by the performance
element - the state of affairs 'known' to the
agent. When the analysis of performance has
been made, the agent must decide whether or
not a better performance could be made in the
future, under the same circumstances. This
decision is then passed to the learning element,
which decides on the appropriate alteration to
future behavior, and modifies the performance
element accordingly.

We must consider the factors which will

influence an agents performance in the game.
Terrain is an obvious start point, as this is all
that stands between the two teams, so it must be
used to the agent's advantage. Secondly, there
must be an element of stealth behind the

few of the problems commonly encountered

when constructing a Learning AI:

So far, so good. But then how do we make sure

that the agent advances in its learning, and
doesn't merely confine itself to previously
observed behavior? This is dealt with by the
curiosity element which has knowledge of the
desirable behavior of the agent. To achieve
optimal performance, this element will pose
new challenges to the agent in an attempt to
prevent (bad) habits developing. To understand
the benefits of this, consider a paintball who is
hiding behind a tree. From his past experience,
he knows that he is safe to stay where he is, and
this would result in an adequate performance.
However, the curiosity element kicks in, and
suggests that he makes a break from his cover
and heads to a nearby tree which is closer to the
enemy flag. This may result in the agent
ultimately being shot at, but could also achieve
a more desirable goal. It is then up to the
performance analyzer and the learning element
to consider whether there is a benefit to this
change in strategy. Decisions trees are widely
believed to be a good method of 'reasoning' - as
are belief networks and neural networks

Problems with Using Neural Networks

in Games:
Despite the obvious potential that learning has
to offer the gaming world, it must be used
carefully to avoid certain pitfalls. Here are but a

1. Mimicking Stupidity - When teaching

an AI by copying a human player's strategy,
you may find that the computer is taught
badly. This is more than likely when the
player is unfamiliar with a game. In this
situation, a reset function may be required to
bring the AI player back to its initial state, or
else a minimum level must be imposed on
the computer player to prevent its
performance
dropping
below
a
predetermined standard.
2. Overfitting - This can occur if an AI
agent is taught a certain section of a game,
and then expected to display intelligent
behavior based on its experience. Using a
FPS as an example, an agent which has
learnt from its experience over one level
will encounter problems when attempting a
new level, as it may not have learnt the
correct 'lessons' from its performance. If it
has found that when opening doors, it has
been able to escape the line of fire by diving
behind a wall to its left, it will assume that
this is a generalized tactic. As you can
imagine, this could lead to amusing
behavioral defects if not monitored in the
correct way.
3. Local Optimality - When choosing a
parameter on which the agent is to base its
learning, be sure to choose one which has no
dependency on earlier actions. As an
example, take a snow-boarding game. The
agent learns, through the use of an
optimization algorithm, the best course to
take down the ski slope, using its rotation as
a parameter. This may mean that a nonoptimal solution is reached, in which any
small change cannot improve performance.
Think about the data being stored - a
sequence of rotations clockwise and
anticlockwise. An alteration to a rotation in

the first half of the run may lead to a better

time over the course in the long run, but in
the short-run, could cause a horrific crash
further down the slope, as the rest of the
rotations are now slightly off course!
4. Set Behavior - Once an agent has a
record of its past behavior and the resulting
performance analysis, does it stick to the
behavior which has been successful in the
past, or does it try new methods in an
attempt to improve? This is a problem
which must be addressed or else an agent
may either try to evaluate every possible
behavior, or else stick to one without finding
the optimal solution.

Recent Developments and

Ongoing Research
Recent Developments
1. The Soar game project at the University of
Michigan A.I lab has developed an interface
between Soar and the commercial Computer
games Quake-3 and Descent-3. Techniques
from various research areas have been used
in developing agents in these two games.
2. A game has been developed that can map
the Human Emotions. It can be set to
various options like I am sad, I am very
happy ,I am very angry etc. Artificial
Intelligence has been able to map perception
and sense inside the machine giving a life
like experience while playing games.
3. World champion Garry Kasparov was
defeated by a Chess playing computer
developed by IBM called Deep Blue, it was
clear that computers had finally dominated
the game of Chess.

Ongoing Research:
1. The Artificial Intelligence (AI) program at
the University of Michigan comprises a
multidisciplinary group of researchers
conducting theoretical, experimental, and
applied investigations of intelligent systems.
Current projects include research in rational
decision making, distributed systems of
multiple agents, machine learning, cognitive
modeling, design, collaboration technology,
natural language processing, real-time and
intelligent dynamical control, autonomous
and tele-autonomous robotic systems,
computer vision, and digital libraries.
2. National Research Foundation-USA has
shown that Particle Swamp Optimization
Algorithm (PSO) can successfully train
neural networks to evaluate competitive
board states during a tic-tac-toe game,
without the use of any prior strategic game
knowledge apart from its official rules. It
has also shown how co-evolution can be
used to drive this training process. Judging
by the positive results obtained through the
experimental work, a more challenging
problem like checkers warrants further
investigation. The performance of PSO in
training co-evolving game playing agents in
a much larger problem space will once again
be compared to existing game learning
research. The use of niching algorithms to
construct game playing strategies will also
be thoroughly investigate
3. Joe Bates OZ research group at Carnegie
Mellon University and Barbara Hayes at
Stanford University have been working on
developing believable agents for interactive
fiction and related computer games. Their
research emphasizes Personality.
4. A tournament RoboCup is organized
annually for the development of AI system

10

in both robotic and simulated AI soccer

game

A Look Back a Glance Ahead

The Conclusion
From a researcher's perspective, even if you are
not interested in human-level AI, computer
games offer interesting and challenging
environments for many, more isolated, research
problems in AI.. One attractive aspect of
working in computer games is that there is no
need to attempt a Manhattan Project approach
with a monolithic project that attempts to create
human-level intelligence all at once. Computer
games provide an environment for continual,
steady advancement and a series of increasingly
difficult challenges. Just as computers have
inexorably gotten faster, computer game
environments are becoming more and more
realistic worlds, requiring more and more
complex behavior from their characters. Now is
the time for AI researchers to jump in and ride
the wave of computer games.

References and Bibliography:

1. Stern, A. 1999. AI beyond Computer
Games. In Papers from the AAAI 1999
Spring Symposium on Artificial Intelligence
and Computer Games, Technical Report SS99-02, 77-80. AAAI Press.
2. J. E. Laird, A. Newell, and P. S.
Rosenbloom: "Soar: architecture for general
intelligence: Artificial Intelligence, 33(3), 164.
3. S. Woodcook: "Game AI: The State of the
Industry" Game Developer, 6(8).
4. SOAR: an architecture for
intelligence
http://sitemaker.umich.edu/soar

general

5. Chellapilla K and Fogel D. B. Evolving an

expert
checkers
playing
program
without relying on human expertise. IEEE
Trans. Evolutionary Computation, 2001.
6. Introduction to Learning in Games
http://www.lupinegames.com/articles/introle
arn.htm
7. www.aaai.org.
8. www.gameai.com.

11