Evolutionary Game Theory

Winter 2017

Psychology 120

Last lecture we talked about finding Nash equilibria for one-shot games.  The point of solving for equilibria is to determine which pure or mixed strategies rational agents should use for different games.  The analytical tractability of game theory comes in large part by abstracting strategies from the agents that use them.  Once we put strategies back into agents and let them play more than once, all bets are off about which strategies will exhibit stability if any.

Evolutionary game theory situates games in evolutionary contexts and seeks to determine the dynamics and stability conditions of strategies.  There are roughly two approaches to evolutionary game theory.  The first is more mathematical, typically using differential equation to describe change in strategy frequency in a population. Strategies are still disembodied and generally, individuals in a population are assumed to be infinite in number and to randomly interact.  The second approach is the agent-based modeling approach, which place strategies back into agents, focuses on agent interactions, and strategy dynamics and stability are emergent properties of agent interactions.

There is no canonical way to create evolutionary game-theory models using agent-based modeling.  My preference is to create generically but minimally biologically realistic agents.  That is agents that are in space, can move, can aggregate, can play games with neighbors, reproduce with a mechanism of mutation, and invest in offspring.  Many more features can be added to such agents, but this is my starting point.

In evolutionary games, strategies evolve if agents using them tend to receive higher payoffs than agents that do not.  In the agent-based model that we develop, payoffs are turned into offspring that inherit a parent’s strategy. Agents that are more successful with their strategy produce more offspring and so the frequency of strategy use in a population increases.

Today we will create an evolutionary agent-based model for the prisoner’s dilemma.  We will begin by creating five classes.

List of classes:

Agent extends Replicator extends BasicAgent implements Steppable

We will start with BasicAgent, which has the capability of movement and aggregation.  I have created a second class Replicator, which has some of the basic methods needed for asexual reproduction.  Agent, which extends Replicator, will have a method for playing the prisoner’s dilemma with other agents and reproducing offspring.

Environnment extends ReplicationEnvironment extends BasicEnvironment Extends SimState

Each environment has parameters required for the functionality of its corresponding agents.

Experimenter extends Observer

The experimenter will collect and save any data and perform any manipulation or control on agents that we need to do simultaneously.

AgentsWithGUI extends GUIState

Provides a graphical user interface.

Probe

When we used Correlation in the Mate Choice model, it functioned as a probe for collecting data.  In general, I like to create probes that collect data where needed.  A probe is created in the Environment and passed to each agent as well as to the Experimenter.  The Experimenter then reads the probe and saves the desired data.

The code below is in the archived project EvolutionaryGT.zip and requires the archived project ABMToolsForMASONv2a.zip.

The Agent class

package egtAgents;

import java.awt.Color;
import basicMovement.BasicEnvironment;
import replicator.Replicator;
import sim.engine.SimState;
import sim.portrayal.simple.OvalPortrayal2D;
import sim.util.Bag;
import sim.util.Int2D;

public class Agent extends Replicator{

public int maxAgents;
public double cooperatorsPayoff;
pubic double temptationPayoff;
public double suckersPayoff;
public double defectorsPayoff;
public int playerSearchRadius;
public boolean played = false;
public boolean cooperator;
public Probe probe;
public int offspring = 0;

	public Agent(BasicEnvironment state, int x, int y, int age,  boolean cooperator, double resources) {
		super(state, x, y, age);
		Environment e = (Environment)state;
		maxAgents = e.maxAgents;
		cooperatorsPayoff = e.cooperatorsPayoff;
                temptationPayoff = e.temptationPayoff;
		suckersPayoof = e.suckersPayoff;
		defectorsPayoff = e.defectorsPayoff;
		playerSearchRadius = e.playerSearchRadius;
		probe = e.probe;
		if(cooperator)
			setColor(state,(float) 0, (float)0, (float)1, (float)1);
		else
			setColor(state,(float) 1, (float)0, (float)0, (float)1);
		this.cooperator = cooperator;
		this.resources = resources;
	}
	/**
	 * Handles a 2-person prisoner's dilemma game.  There is a cost in obtaining a cooperative
	 * benefit.  If two plaeyers cooperate and share the gain, then each receives of payoff of
	 * benefit/2 - cost.  If one player defects from sharing the benefit, then its payoff is the 
	 * benefit - cost and the other player is -cost.  If both do not cooperate, the payoff the
	 * non-cooperative payoff (i.e., typically 0). 
	 * @param state
	 */

	public void play(Environment state){
		if(played){  //already played this round
			return;
		}
		Bag players = state.space.getMooreNeighbors(x, y, playerSearchRadius, mode, false);
		//next find a random player who has not played
		Agent other = null;
		if(players == null || players.numObjs == 0){//there are no players nearby
			return;  
		}
		if(players.numObjs == 1){//only one
			other = (Agent)players.objs[0];
			if(other.played){//exit because other already played
				return;
			}
		}
		int start = state.random.nextInt(players.numObjs); //random starting point
		if(other == null){//If we get this far without a player
			for(int i = start;i< players.numObjs;i++){
				other = (Agent)players.objs[i];
				if(!other.played){
					break;
				}
				else{
					other = null;
				}
			}
		}

		if(other == null){//If we get this far without a player, keep searching
			for(int i = 0;i< start;i++){
				other = (Agent)players.objs[i];
				if(!other.played){
					break;
				}
				else{
					other = null;
				}
			}
		}
		if(other == null){//we did our best
			return;
		}
		//now to the game
		if(this.cooperator && other.cooperator){
			this.resources += cooperatorsPayoff; //cooperative payoff
			other.resources += other.cooperatorsPayoff;

		}
		else if(this.cooperator && !other.cooperator){ //cooperator/defector
			this.resources += suckersPayoff;//the cost, i.e. sucker's payoff
			other.resources += other.temptationPayoff;
		}
		else if(!this.cooperator && other.cooperator){ //defector/cooperator
			other.resources += other.suckersPayoff;//the cost, i.e. sucker's payoff
			this.resources += temptationPayoff;
		}
		else if(!this.cooperator && !other.cooperator){ //both defectors
			other.resources += other.defectorsPayoff;
			this.resources += defectorsPayoff;
		}
	}
	
	/**
	 * Agents reproduce asexually and locally.
	 * @param state
	 * @return
	 */

	public boolean reproduce(Environment state){
		Int2D location = this.canReproduce(state); //if it can, then return a location
		if(location != null){
			Agent offspring = new Agent(state, location.x, location.y, 0, this.cooperator, 0);
			//do mutation
			if(state.random.nextBoolean(offspring.mutationsRate)){
				if(offspring.cooperator){
					offspring.cooperator = !offspring.cooperator;//flip
				}
			}
			careAndplace(state, offspring);
			this.offspring ++;
			return true; //successful
		}

		return false; //failed
	}

	public void die(BasicEnvironment state){
		super.die(state);
		if(cooperator){
			probe.setCooperatorEnergyAtDeath(resources);
			probe.setCooperatorsDied(1);
		}
		else{
			probe.setDefectorEnergyAtDeath(resources);
			probe.setDefectorsDied(1);
		}
	}
	
	
	public void setColor(BasicEnvironment state, float red, float green, float blue, float opacity){
		Color c = new Color(red,green,blue,opacity);
		OvalPortrayal2D o = new OvalPortrayal2D(c);
		AgentsWithGUI guiState = (AgentsWithGUI)state.gui;
		guiState.agentsPortrayal.setPortrayalForObject(this, o);
	}
	
	public void adjustResources(){
		if(resources > maxResources){
			resources = maxResources;
		}
	}


	public void step(SimState state){
		super.step(state);
		if(age >= lifeSpan || randomDeathProb > 0 && state.random.nextBoolean(randomDeathProb)){
			die((Environment)state); 
			return; //nothing more to do
		}
		play((Environment)state);
		adjustResources();
		reproduce((Environment)state);
		age++;
		if(cooperator)
			probe.setCooperators(1);	
		else
			probe.setDefectors(1);
		}

}

The Environment class

package egtAgents;

import replicator.ReplicatorEnvironment;
import sim.field.grid.SparseGrid2D;

public class Environment extends ReplicatorEnvironment {
	public int cooperators = 100;
	public int defectors = 100;
	public double cooperatorsPayoff = 3;
        public double temptationPayoff = 5;
	public double suckersPayoff = -1;
	public double defectorsPayoff = 0;
	public int playerSearchRadius = 1;
	public Probe probe;
	
	
	
	public int getCooperators() {
		return cooperators;
	}

	public void setCooperators(int cooperators) {
		this.cooperators = cooperators;
	}

	public int getDefectors() {
		return defectors;
	}

	public void setDefectors(int defectors) {
		this.defectors = defectors;
	}

	public double getCooperatorsPayoff() {
		return cooperatorsPayoff;
	}

	public void setCooperatorsPayoff(double cooperatorsPayoff) {
		this.cooperatorsPayoff = cooperatorsPayoff;
	}

        public double getTemptationPayoff(){
               return temptationPayoff;
        {

        public void setTemptationPayoff(double temptationPayoff){
               this.temptationPayoff = temptationPayoff;
        {

	public double getSuckersPayoff() {
		return suckersPayoff;
	}

	public void setsSuckersPayoff(double suckersPayoff) {
		this.suckersPayoff = suckersPayoff;
	}

	public double getDefectorsPayoff() {
		return defectorsPayoff;
	}

	public void setDefectorsPayoff(double defectorsPayoff) {
		this.defectorsPayoff = defectorsPayoff;
	}

	public int getPlayerSearchRadius() {
		return playerSearchRadius;
	}

	public void setPlayerSearchRadius(int playerSearchRadius) {
		this.playerSearchRadius = playerSearchRadius;
	}

	public Environment(long seed, Class observer) {
		super(seed, observer);
		// TODO Auto-generated constructor stub
	}
	
	public void createAgents(){
		int locations = gridWidth * gridHeight;
		if(uniqueLocation && cooperators + defectors > locations){
			System.out.println("Too many agents! Please reduce the number of cooperators and defectors");
			return;
		}
		else{
			for(int i = 0;i< cooperators;i++){
				int x = random.nextInt(gridWidth);
				int y = random.nextInt(gridHeight);
				int age = random.nextInt(averagelifeSpan);
				double resources = random.nextDouble() * this.reproductionResources;
				Agent a = new Agent(this,x, y, age,  true, resources);
				place(a);
				a.event = schedule.scheduleRepeating(a);
			}
			
			for(int i = 0;i< defectors;i++){
				int x = random.nextInt(gridWidth);
				int y = random.nextInt(gridHeight);
				int age = random.nextInt(averagelifeSpan);
				double resources = random.nextDouble() * this.reproductionResources;
				Agent a = new Agent(this,x, y, age,  false, resources);
				place(a);
				a.event = schedule.scheduleRepeating(a);
			}
		}
	}
	
	public void start(){
		super.start();
		probe = new Probe();
		space = new SparseGrid2D(gridWidth,gridHeight);
		createAgents();
		
		if(observer != null){
			observer.init(space);
			((Experimenter)observer).setProbe(probe);
		}
	}
}

The Experimenter class

package egtAgents;

import sim.engine.SimState;
import sweep.Observer;
import sweep.ParameterSweeper;
import sweep.SimStateSweep;

public class Experimenter extends Observer {

	Probe probe;
	
	public Experimenter(String fileName, String folderName, SimStateSweep state, ParameterSweeper sweeper,
			String precision, String[] headers) {
		super(fileName, folderName, state, sweeper, precision, headers);
		probe = ((Environment)state).probe;
	}
	
	public Probe getProbe() {
		return probe;
	}

	public void setProbe(Probe probe) {
		this.probe = probe;
	}

	public boolean nextInterval(){
		data.add(probe.cooperators + probe.defectors);
		data.add(probe.cooperators);
		data.add(probe.defectors);
		data.add(probe.getCooperatorEnergyAtDeath());
		data.add(probe.getDefectorEnergyAtDeath());
		
		return true;
	}
	
	public void step(SimState state){
		super.step(state);
		if(getdata){
			nextInterval();
			probe.resetAll();
		}
		else{
			probe.resetCoopDef();
		}
		
		for(int i=0;i<agents.numObjs;i++){
			Agent a = (Agent)agents.objs[i];
			a.played = false; //set all to false for the next time step
		}
		if(agents.numObjs == 0){
			state.schedule.reset();
		}
	}

}

The Probe class

package egtAgents;

public class Probe {
	int cooperators = 0;
	int defectors = 0;
	int cooperatorsDied = 0;
	int defectorsDied = 0;
	double cooperatorResourcesAtDeath = 0;
	double defectorResourcesAtDeath = 0;
	
	public int getCooperators() {
		return cooperators;
	}
	public void setCooperators(int cooperators) {
		this.cooperators += cooperators;
	}
	public void resetCooperators(){
		cooperators = 0;
	}
	
	public int getDefectors() {
		return defectors;
	}
	public void setDefectors(int defectors) {
		this.defectors += defectors;
	}
	public void resetDefectors(){
		defectors = 0;
	}
	
	public int getCooperatorsDied() {
		return cooperatorsDied;
	}
	public void setCooperatorsDied(int cooperatorsDied) {
		this.cooperatorsDied += cooperatorsDied;
	}
	public void resetCooperatorsDied() {
		this.cooperatorsDied =0;
	}
	
	public int getDefectorsDied() {
		return defectorsDied;
	}
	public void setDefectorsDied(int defectorsDied) {
		this.defectorsDied += defectorsDied;
	}
	public void resetDefectorsDied() {
		this.defectorsDied =0;
	}
	
	
	public double getCooperatorEnergyAtDeath() {
		double average = 0;
		if(cooperatorsDied > 0){
			average = cooperatorResourcesAtDeath/cooperatorsDied;
		}
		return average;
	}
	public void setCooperatorEnergyAtDeath(double cooperatorEnergyAtDeath) {
		this.cooperatorResourcesAtDeath += cooperatorEnergyAtDeath;
	}
	public void resetCooperatorEnergyAtDeath() {
		this.cooperatorResourcesAtDeath =0;
	}
	
	public double getDefectorEnergyAtDeath() {
		double average = 0;
		if(defectorsDied > 0){
			average = defectorResourcesAtDeath/defectorsDied;
		}
		return average;
	}
	public void setDefectorEnergyAtDeath(double defectorEnergyAtDeath) {
		this.defectorResourcesAtDeath += defectorEnergyAtDeath;
	}
	public void resetDefectorEnergyAtDeath() {
		this.defectorResourcesAtDeath = 0;
	}
	
   public void resetAll(){
	   resetCooperators();
	   resetCooperatorEnergyAtDeath();
	   resetCooperatorsDied();
	   resetDefectors();
	   resetDefectorsDied();
	   resetDefectorEnergyAtDeath() ;
   }
   
   public void resetDied(){
	   resetCooperatorEnergyAtDeath();
	   resetCooperatorsDied();
	   resetDefectorsDied();
	   resetDefectorEnergyAtDeath() ;
   }
   
   public void resetCoopDef(){
	   resetCooperators();
	   resetDefectors();
   }
   
   public void printData(){
	   System.out.print(cooperators + defectors +"\t");
	   System.out.print( cooperators +"\t");
	   System.out.print(defectors +"\t");
	   System.out.print(getCooperatorEnergyAtDeath()+"\t");
	   System.out.println(getDefectorEnergyAtDeath());
   }
}

The AgentsWithGUI class

package egtAgents; 

import sim.engine.*;
import sim.display.*;
import sim.portrayal.grid.*;
import java.awt.*;
import javax.swing.*;
import sim.portrayal.simple.OvalPortrayal2D;

public class AgentsWithGUI extends GUIState {
	public Display2D display;
	public JFrame displayFrame;
	SparseGridPortrayal2D agentsPortrayal = 
                             new SparseGridPortrayal2D();

	public static void main(String[] args) {
		Environment e = new Environment(System.currentTimeMillis(), Experimenter.class);
		AgentsWithGUI ex = new AgentsWithGUI(e);
		e.setGui(ex);
		Console c = new Console(ex);
		c.setVisible(true);
		System.out.println("Start Simulation");
	}

	public AgentsWithGUI(SimState state) {
		super(state);
	}

	public void quit() {
		super.quit(); 

		if (displayFrame!=null) displayFrame.dispose();
		displayFrame = null;
		display = null;
	}

	public void start() {
		super.start();
		setupPortrayals();
	}

	public void load(SimState state) {
		super.load(state);
		setupPortrayals();
	}

	public void setupPortrayals() {
        Environment se = (Environment)state;
		agentsPortrayal.setField(se.space);
		display.reset();
		display.repaint();
	}

	public void init(Controller c){
		super.init(c);
		display = new Display2D(400,400,this);
		displayFrame = display.createFrame();
		c.registerFrame(displayFrame);
		displayFrame.setVisible(true);
		display.setBackdrop(Color.WHITE);
		display.attach(agentsPortrayal,"Agents");
	}

	public Object getSimulationInspectedObject() {
		return state;
	}
}