Home / Research / Artificial Intelligence / Constructing Intelligent Agents in Games

Constructing Intelligent Agents in Games

Constructing Intelligent Agents in Games
Risto Miikkulainen
Department of Computer Sciences
The University of Texas at Austin

Reference: Link


PDF – Miikkulainen

Real world is complex, open ended, messy

Virtual worlds more tractable than the real world
– Games are controlled, formal, measurable
– They are a safe platform for AI
– They provide realistic, significant challenges
Traditional AI Technolgy
• Much of AI developed in games & for games
– Board games
– Good Old-Fashioned AI (GOFAI): Rules, logic, search…
Video games have become a major industry
– $25B worldwide (2005)
– Sophisticated simulated worlds
– Part of everyday life

Very little AI in games; Still mostly GOFAI
– Scripting, authoring
– A* pathfinding, finite state machine behaviors

• GOFAI does not work well in video games
• They are different from board games:
– Multiple agents
– Embedded: continuous, noisy, large-dimensional
– Real-time, changing environments

A New Approach: “Computational Intelligence”
• Natural Computation: Neural networks, evolution, reinforcement learning
• Powerful in many statistical domains
– E.g. pattern recognition, control, prediction, decision making
– When hard to formulate rules, but plenty of examples
• Can learn and generalize
– Learn a nonlinear function that matches the examples

Current CI Research in Games
• Initial successes with board games
– Checkers, chess, backgammon, go, othello…
• Technique apply to video games as well
– FPS, RTS: Unreal, Neverwinter, Quake…

Neuroevolution (NE)
• Chromosomes (in Genetic Algorithm) are strings of connection weights, Evolved through crossover and mutation

Advanced NE: Complexification
Neuroevolution of Augmenting Topologies(NEAT) (Stanley et al. 2004)
• Optimizing connection weights and network topology
• Incremental construction of intelligent agents

Applying NE to Games
• Can be used to build “mods” to existing games
– Adapting characters, assistants, tools

NERO: A Machine Learning Game
• Produced at the Digital Media Collaboratory at UT Austin, Uses Garage Games TorqueTM game engine – http://nerogame.org

NERO Gameplay
• Teams of agents trained to battle each other
– Player trains agents through excercises
– Agents evolve in real time
• Challenging platform for reinforcement learning
– Real time, open ended, requires discovery

• Player can place items on the field
e.g. static enemies, turrets, walls, rovers, flags
• Sliders specify relative importance of goals
e.g. approach/avoid enemy, cluster/disperse, hit target, avoid fire…
• Networks evolved to control the agents

• Given a problem, NE discovers a solution by exploring
– Sometimes highly original solutions
– Requires lots of exploration
• Designers may want to have more control
– Seeding with initial behaviors
• Players may want to interact with learning
– Giving advice during evolution

Incorporating Rules into NE

Incorporating Knowledge with KB-NEAT

Lessons from NERO
• NE constructs intelligent agents in games
– Discovers effective behaviors
– Adapts in real time
• Can add adaptation to existing games
• Can build machine-learning games
• Requires many evaluations
– Best when parallel evaluations possible
• Best when combined with human guidance
– E.g. examples or rules

• Killer application
– Huge potential economic impact
– Entertainment, training simulators
– Robotics, resource optimization, intelligent assistants

About Mohammad Khazab

%d bloggers like this: