1、Evolving Reactive NPCs for the Real-Time Simulation Game,IEEE Symposium on Computational Intelligence and Games,Outline,Motivation Objective Introduction The game: Build however, the movement of a character is apt to be unrealistic. there is a trend towards fuzzy state machine (FuSM).,Adaptation and

2、 learning: NNs, EAs, and Artificial life,The adaptation and learning in games will be one of the most major issues making games more interesting and realistic. Neural network, and evolutionary algorithms (e.g. genetic algorithm) are promising artificial intelligence techniques for learning in comput

3、er games. NN - is badly trained GE - required too many computations and were too slow to produce useful results.,Co-evolution,By simultaneously evolving two or more species with coupled fitness. Superior strategies for an environment have been discovered by co-evolutionary approaches.,Reactive behav

4、ior,Reactive model performs effectively since it considers the current situation only. Neural networks and behavior-based approaches are recently used for the reactive behavior of NPCs keeping the reality of behaviors.,The game: Build & Build,Build & Build developed in this research is a real-time s

5、trategic simulation game, in which two nations expand their own territory. Each nation has soldiers who individually build towns and fight against the enemies, while a town continually produces soldiers for a given period.,The game: Build & Build,Designing the game environment,The game starts two co

6、mpetitive units in a restricted land with an initial fund. The units are able to take some actions at the normal land but not at the rock land. A unit can build a town when the nation has enough money, while towns produce units using some money.,Designing the game environment(cont.),Designing NPCs,N

7、PC can move by 4 directions as well as build towns, attack units or towns, and merge with other NPCs. The attack actions are automatically executed when an opponent locates beside the NPC.,Designing NPCs (cont.),Designing NPCs (cont.),Basic behavior model (cont.),Two different grid scales are used f

8、or the input of the neural network such as 55 and 1111.,Basic behavior model (cont.),In order to actively seek a dynamic situation, the model selects a random action with a probability (in this paper, a = 0.2) in advance.,five neural networks are used to decide whether the associating action execute

9、s or not.,Co-evolutionary behavior generation,We use the genetic algorithm to generate behavior systems that are accommodated to several environments. Two pair-wise competition patterns are adopted to effectively calculate the fitness of an individual.,Co-evolutionary behavior generation (cont.),The

10、 fitness of an individual is measured by the scores against randomly selected M opponents.,Experiment and Results,Four different battle maps = demonstrate the proposed method in generating strategies adaptive to each environment.,Experiment and Results (cont.),The case with 1111 shows more diverse b

11、ehaviors than that with 55, since it observes information on a more large area. 55 obtains lower winning averages for complex environment, while it performs better when the environment is rather simple.,Experiment and Results (cont.),Fig. 8. Winning rate between 55 behavior and 1111 behavior at each

12、 generation on map type 3.,The 1111 shows the better performance than the 55, since it considers more various input conditions so as to generate diverse actions.,Experiment and Results (cont.),For the plain map, 55 behavior system shows a simple strategy that tries to build a town as much as possibl

13、e. Building a town leads to generate many NPCs so as to slowly encroach on the battle map as showns in Fig. 9.,Discussion,The reactive system shows good performance on simple environments like the plain map, but it does not work well for complex environments. Also, the amount of input information is important for the reactive system when the environment is not simple.,Conclusion,A reactive behavior system was presented for the flexible and reactive behavior of the NPC. Co-evolutionary approaches have shown the potentialities of the automatic generation of excellent strat