Jessica Cooper

University of St Andrews


FISH is a prototype game in which the player may interact with and influence the evolution of a population of virtual fish by means of a variety of physical sensors. The fish are powered by neural networks which evolve through a process of random mutation and natural selection.

  1. Author Keywords

Neural networks, evolutionary algorithms; simulation; sensors; physical computing; Phidgets; Processing; Java.


Upon launching the game a two dimensional world is created and populated at random with fish and food. The fish proceed to swim around and eat the food. At first, they are quite stupid and many will die out, if competition is fierce and their strategies are not useful – but any that are successful enough at finding food to reproduce will pass on their traits to the next generation. In this way, the population becomes gradually smarter as a whole. If no useful traits are generated and all the fish die out, the world is automatically repopulated.

The player may interact with and influence this world via physical Phidget sensors. It is possible to help or hinder the fish in a number of ways, and thus shape their evolution.


There are four different Phidget sensor inputs to the game.

Joystick:  Controls the water current. This allows the player to influence the movement of the fish in the world, encouraging them towards or away from concentrated food areas.

Radio dial: Controls the speed of the game, or pauses it. At a high speed generations pass by in a number of seconds, enabling the player to see more interesting behaviours develop quickly. At lower speeds it is possible to watch the progress of individual fish or shoals as they move through the world.

Sound sensor: Fish don't like loud noises. If the sound level is too high, they will freeze momentarily – and while frozen they cannot find food. The player may use this to adjust the population of an overcrowded world, or to kill all the fish if feeling bored or vindictive.

Light sensor: The fish eat plants which flourish in the dark. By covering the light sensor or exposing it to a light source, the player may alter the rate at which food appears in the world.


The environment is two dimensional, with both fish and food represented symbolically by circles of varying sizes and colours. Information about individual fish is communicated solely visually: as the fish eat more food, they become larger on screen. Parents pass along their colour to their children, allowing the player to get a sense of intergenerational relationships.


FISH is written in Java. Three additional libraries are used: Encog, to implement the neural networks that constitute the brains of the fish; Processing, to create the visual output and animate the world; and the Phidget Kit interface, which allows sensor input to be taken.


Ideas for further work include introducing different types of food, introducing different types of fish (e.g. predator and prey types) extending the world to include other interactive objects (such as hiding places to avoid predators, obstacles etc) or adding further sensor input.


Interacting with the virtual world in a physical way is very satisfying, and whistling at fish until they expire is certainly a novel experience. FISH is an unusual little game which would benefit from further development, but the core concept of watching as different behavioural patterns evolve and occasionally intervening with godlike power is quite engaging. It is also very accessible - there is no text in the game and the controls can be learned experimentally by all ages.