Predictive Power of Role-play Simulations in Political Science:
Experience of an e-Learning tool
Keywords: Role Play, Simulations, E-Learning,
Authoring Tool, Pedagogy, Fablusi, CASCON.
The scientific method has been accepted and valued as contributing to knowledge, to say the least, because it has offered a way by which to minimize uncertainty and enhance our predictive capacity in the domains in which it was successfully applied. Political Science which sought to integrate these methods to its field, and theories of International Relations in particular, have generally failed in providing much guidance in predicting the course of events of their chosen domain of the political – despite clearly serious, committed, consistent and imaginative scholarship. Political Science, it seems has contributed more to understanding the past than the future.
Analysts in these domains can usually do no more than provide historical, comparative, statistical and rational argument to support certain predictions, but unlike the physical sciences few can test these predictions with experiments of physical variables like in chemistry that help reduce uncertainty, and must wait for the course of events to prove them right or wrong. At most what can be achieved are thought experiments of various types and it is the aim of this paper to discuss one of these – the collaborative e-Learning thought experiment that ensues from web-based role-play simulations.
The lack of predictive power in International Relations theories has always been used as a weapon to discount or attack the utility of these theories. Still, the project of devising and improving analytical and empirical ways of understanding continues, as does the induction of young researchers into the field. At the Political Science Department at the university of Melbourne web-based role-play simulations have been used, as a teaching tool in some of its undergraduate courses in international politics and this paper will discuss these simulations, examine their potential as a tool for collaborative thought experiments and evaluate their predictive value.
The predictive power of simulations in political science arose for this writer as an issue in 1993 when simulations in Middle East Politics ran by Dr. Andrew Vincent at the University of Melbourne produced what seemed unrealistic outcomes, predicting specific events that were later proven to have been very realistic (assassination of Meir Khana, The Gaza-Jericho first agreement). Subsequent experience of creating and running simulations in courses on the politics of the Arab-Israeli conflict, politics of the Asia Pacific, Australian foreign policy, Russian politics and world politics have also seemed to produce similarly surprising and sometimes uncanny results.
This paper examines these simulations and evaluates some of these results attempting to understand how and why they were produced. Is it simply coincidence, noise in the discourse of political commentary or does this simulation tool have more utility than just pedagogy? The paper places these simulations in the context of the ICT revolution and compares the use of a web-based role play with some other ways in which ICT have been put into service for the prediction of future political events. In particular it compares the use of the role-play web-based simulations produced with the Fablusi software with rule-based software like CASCON produced by Bloomfield and Moulton from MIT and other rule based simulations used for pedagogy and prediction.
The paper concludes with recommending the use of simulations for both pedagogical and research in political science because as the simulations discussed in this paper seem to suggest, only when relinquishing the quest for realism in politics that one begins to catch a glimpse of political reality.
The latter half of the 20th century, as we all now know, has exponentially transformed the landscape of knowledge and practice. The tools that have been made available to Political Science, however, made little impact outside statistically based research and making access to information and collaboration quicker and easier. Both teaching and research (or learning in a wider sense) are mostly conducted using a bricolage strategy combining social, historical, economic, and sometimes psychological texts. These are cast in a web of empirical, analytical, normative, literary and statistical methods to construct more or less meaningful narrative interpretations of the political and instruct students in researching and producing such knowledge.
Clearly public information enabled by the tools is now instantaneous, vast and beneficial. Clearly also, such volume of information and search capabilities are useful. But it is also clearly problematic. It is not simply a problem of information overload or information distribution. For research, education and training the problem is how to evaluate, understand and organize information in meaningful ways that have implications for practice and more specifically pedagogical practice.
Pedagogical practice in the realm of the political has been slow in adopting the technology, though statistical and search tools for research have been incorporated. But even when doing so to varying degrees of effectiveness, it has mostly attempted to use the new tools in old ways and CASCON will provide us with one such example.
Unlike the hard sciences which have used the tools enabled by ICT to better model the objects of their study, Political Science has mostly used them to create, retrieve and disseminate information about the objects of their study - and with good reason. The objects for study and analysis are contested abstractions. Unlike the physical sciences, the researcher cannot physically manipulate the reality of the political. He cannot bombard it with electrons or bounce radio waves of its surface – indeed it is absurd to even wonder what such a surface would look like?
From a pedagogical perspective, ICT tools are mostly used in Political Science to provide information for the user – data bases, electronic texts, etc. - though it is abundantly clear to everyone that simply putting up information to read is not a particularly effective teaching strategy. It is however cheap and easy to do requiring little effort. This ‘shovelling information’ technique using higher or lowers levels of sophistication is what Clark Aldrich recently describes as the presentation of linear content as opposed to cyclical and open ended ways of handling it .
In the following section a comparative contrast is drawn between the CASCON and Fablusi platforms with regard to their predictive potential and the pedagogy they entail.
CASCON developed by Bloomfield and Moulton at MIT is one of the few tools enabled by the new ICT that has been created specifically for Political Science for both research and pedagogy. In essence it provides a historical case-based approach that enables researchers and students to study variables that lead to, or decrease, international conflict. Based on the Bloomfield-Leiss Dynamic Phase conflict model  it provides means by which to code, organize and compare historical examples of International Conflicts, it puts the researcher and student in a position to analyse and draw conclusion about similar cases current in the real world.
CASCON was of course never intended as a predictive tool but rather an analytical one. As the authors of this tool remind us it is only a supplement to the professional’s judgement, experience, understanding and skills . It is historically based and clearly a very creative way of handling and organizing the information of empirical case studies. As such it can perhaps be generalized to other areas of research and pedagogy. Any predictive value it has lies in the user’s ability to organize information, compare the factors of different cases using the software and make educated judgements.
However, though not intended as a predictive tool, in the background, the motive for using CASCON is to be able to learn which steps can be taken to prevent conflict from escalating into open hostilities. The authors recommend this tool to policy makers because it provides a systemic method for organizing information about specific cases so that recommendations can be made about current cases and their future development. The software itself may have no predictive power, but the analytical results generated by its use are aimed at not only predicting the course of events in a particular case but also to provide educated suggestions to alter the course of their trajectory.
By contrast the Fablusi platform was specifically developed for pedagogy in Political Science rather than as a research tool. At the basis of the approach lies the dictum that experience is the best teacher and the guiding abstraction about the object of study that communication is the organizing process of a community, including the communities of practice concerned with political practice, governance, administration, or research.
The aim was to get students to become familiar with current International realities by having them play the roles of different actors in that arena and thus have them experience facing dilemmas and potential courses of action that may be taken by such actors. In the course of researching these roles students inevitably gather relevant historical information to help them play their role in character.
Following in the footsteps of Andrew Vincent and John Sheppard  and based on Roger Schank’s goal-based scenarios , an approach we call ‘dynamic goal-based learning’ was developed  in which the actions taken by the roles feeds back into the learning space created by the platform and thus dynamically generating new scenarios necessitating further action. In other words testing a course of action leads to consequences that need to be addressed both analytically and practically in a dynamically changing environment.
CASCON enables the user to manipulate texts according to rules embedded in the software with linear outputs. Fablusi, by contrast, enables linear, cyclical and open-ended collaborative creation of texts within subject related communicative contexts.
The user of CASCON is presented with linear information, acts upon it by adding, subtracting and coding information and the output is reconfigured and presented in linear form. Thus it remains a ‘shovelling technique’ albeit a sophisticated one. Fablusi on the other hand distributes information unevenly between the roles, initial scenario, interaction spaces and resources , creating information gaps between users. On this basis users, as roles, input information they create and distribute it to the roles and interaction spaces in accordance with their objectives. Further they must then respond to the output effects their input had on other players’ activities. The choices as to which information they respond and which information they input remains open-ended as it depends on the objectives to be achieved by the role.
Because CASCON handles information on the basis of rules derived from a specific theoretical model of understanding the political there is no scope within the system for organizing the information and comparing it to different theoretical perspectives. Fablusi, by contrast, uses a generalized model of communication to study the political and therefore users can interact with one another on the basis of different and even opposing theoretical models of the political. Isn’t this what politics is all about?
In the first case the model of the world is determined by a theory that is embedded in the software, which in turn determines how information can be manipulated. In the second, the model of the world is partially determined by specific communication contexts provided by the creator and partially determined by the user’s creation, manipulation and distribution of information within these contexts. Thus the creator and user jointly construct what that world would look like – even if the theoretical abstractions of the user may be different from the creators.
The upshot is that pedagogically users of CASCON remain within the traditional information provider, or linear, paradigm of pedagogy despite using the new ICT, whereas users of Fablusi study their objects using a combination of linear, cyclical and open-ended processes in a collaborative and constructivist paradigm.
Even if the communicative contexts are provided, as a multi-user and open-ended environment, the Fablusi platform would be expected to generate some unpredictable models of the world and indeed many specific events and situations that were generated were fanciful and unlikely to say the least. But at the same time every simulation run has also produced events and situations that not only have similarities to the real, but sometimes uncanny results that seem to predict these events.
Eight simulations run between the 2000 and 2002 academic years at the university of Melbourne and one run at the University of Wales in Swansea have generated the data that we will now examine. All together, over 600 students were involved in playing some 300 roles generating over 22,600 messages.
Given this enormous volume of data it will be obviously impossible to present here a detailed picture. Indeed we are only at the beginning of this research. The examples here were selected out of only part of the data (the internal News reports of the simulations) after the first scan of the material. But it was so compelling that it clearly required some thought. Given space limitations this paper presents and discusses only one or two events abstracted from each of the different simulations and compares them to subsequent real world events adding a few observations for each.
1. In early April of 2000, two courses – Australian Foreign Relations and Theories of Power - played a joint simulation (97 students playing 52 roles). The scenario included the death of an Australian soldier in East Timor. The reason for this was that we wanted a dramatic event to which all roles would have to respond – the kick-start episode  – and given we were focusing on Australian Foreign relations (specifically with Indonesia and the UN) such an incident, we reasoned, would have to raise questions about these relations and policy. At the time, UNTAED forces were replacing InterFET forces as part of the transition to independence of East Timor under the guidance of the UN. On the 5th of April 2000 given our scenario, SBS (Sim World – hereafter SW) appropriately reported that an Australian soldier was killed in an ambush by militia gunfire near the border with West Timor. On the 24th of July 2000 the BBC (Real world – hereafter RW) reported the death of the first peacekeeper (from NZ), killed in an exchange of gunfire with Timorese rebels near the border with West Timor [U1]*. One could argue that this could have been foreseen given the volatile situation. While this may be true, given the bravado in government statements that usually accompanies such deployments, this possibility was muted in public discourse. In hindsight it is easy to say that this was probably going to happen – the reality is that nobody actually said it at the time. In the same simulation on the 19th of April 2000, after a week long of reported tensions and the deployment of Australian led international peacekeeping force, the BBC (SW) reported that the Malaitan Eagle Force (MEF) staged a coup killing the PM of the Solomon Islands, Mr Bartholomew Ulufa’alu with riots and sporadic gunfire on the streets of the capital Honiara. Just over 4 weeks later in June 2000 the MEF (RW) staged an attempted coup taking PM Ulufa'alu hostage and in October 2000 unarmed peacekeepers from Australia and New Zealand are deployed to supervise a negotiated peace deal [U2]. Again, at the time no analyst actually expected or publicly outlined such a scenario despite the tense situation in the Solomon Islands. Was it likely to happen?
2. In early May of 2000 we ran a simulation in a Worlds Politics in Transition Course (86 students playing 40 roles) focusing on China/Taiwan, India/Pakistan and Israel/Palestine issues. On the 11th of May 2000 the BBC (SW) reported that General Musharraf, who less than a year earlier seized power in Pakistan, declared that a return to a constitutional government would follow fresh election on the 14th of August (Pakistan’s independence day.) During Independence Day on the 14th of August 2001, the Guardian Newspaper (RW) reported that Musharraf surprisingly announced that elections will be held in October to return the nation to democracy [U3]. Both the role playing Musharraf and the real leader obviously understood the symbolic significance of the 14th of August as an appropriate time for such an event. Was this a lucky guess?
early October 2000, the Asia Pacific International Relations course ran a
simulation (72 students playing 37 roles) focusing on the Korean Peninsula with
a kick-start episode that has G.W. Bush as President and North Korea launching
a second test missile over Japan. At the time this was unlikely as evidenced in
the Wisconsin Project on Nuclear Arms Control update in October 2000 where it
reported that a “State Department official said the Clinton administration did
not expect North Korea to break its pledge” [made in 1999 to freeze missile
tests] and North Korea extended its ban on missile testing just a few months
earlier in June 2000 [U4]. We took a gamble and went with the unlikely scenario
to see how the region would react to another missile test and perhaps the less
contentious possibility that a bellicose Bush would be the next President of
the United States. A month later in November 2000 Bush was elected and a year
and four months later, March 1993, North Korea tested another missile (both
RW). In that same simulation and in the context of discussions between players
on US military moves and the North Korean missile test, on the 18th
of October 2000 CNN (SW) reported that Russia and China have formed a military
alliance in response to US attempts to undermine them in the region. Seven
months later and in the context of US upgrade to its national missile defence,
the ABC in July 2001 (RW) reported on an agreement signed between Russia and
China aimed at building stronger ties to counterbalance the ongoing use of US
power against both countries. “The Presidents of both countries have now signed
a treaty that stresses economic and military cooperation, based, in the words
of the document, on 'mutual efforts to support the global strategic balance and
4. In May 2001 the simulation in the Australian Foreign Relations course (75 students playing 36 roles) focused on Australian relations with Indonesia and events in Aceh, East Timor and the PNG. One element in the kick-start episode referred to sources in Jakarta who were eagre to negotiate a new treaty with Australia and East Timor to manage oil resources in the region. On the 25th of May 2001 the Jakarta Post (SW) reported that an agreement has been reached between Australia and East Timor, over the issue of the Timor Gap with the majority of the profits from the Timor Gap site going to East Timor, while Australia will be entitled to 12.5 per cent of the revenue. On July 5th 2001, slightly more than a month later the BBC (RW) reported that Australia and East Timor have signed a 30 year agreement whereby “East Timor should receive 90% of royalties from Australian mining operations for liquid petroleum and gas oil under the Timor sea between the two countries” [U6]. Our students were off by 2.5%.
5. In a simulation run in May 2001 for a Russian Politics Course (19 students playing 14 roles) the main focus was on the power relations between Moscow and the regions given President Putin’s creation of seven new federal districts, headed by 'presidential envoys'. To show the regions that they are dependent on Moscow, on the 23 of May 2001 our Putin (SW) decided that a sever earthquake in one of the regions, particularly one that occurred near a nuclear power station, and thus could possibly involve international aid from donor countries, would make the point clear. ITAR TASS (SW) ran the article and thus ‘created’ the event with 6000 people affected in Archangelsk. Five months later on the on the 10th of January 2002 PRAVDA (RW) reported an earthquake in Tajikistan that according to Russia's Ministry of Emergency and Calamity Relief killed and injured 1500-1600 people and that the International Committee of the Red Cross, UNICEF, and the United Nations Children's Fund were responding with assistance. [U7] Though not exactly the same place, nor the amount of casualties or the correct aid agencies, the means, the earthquake, Putin (SW) tried to use to demonstrate the power of the centre, had occurred in the real world to which international elements responded as expected. By the way, on the day of the real quake a bus overturned injuring 50 in Archangelsk [U7]. From a pedagogical and predictive point of view natural events can hardly be said to be political, the response to them, however, and how they may be used, is the point at issue here.
October 2000, one month after the 9/11 attacks in New York and Washington, and
therefore not surprising, the Global Politics Course (124 students playing 56
roles) ran a simulation on the effects of terrorism in world politics as a
central issue though not excluding other issues. On the 15th of
October 2001 CNN (SW) reported a number of bombings targeting US businesses in
Jakarta. The source was the Indonesian Defence Minister (SW) whose agenda was
to increase, perhaps justify, the use of the military against the ‘terrorists’
in Aceh. In December of 2001 (RW) Al Qaeda cells who were allegedly planning to
blow up US, British and Australian targets in Singapore were arrested. [U8] In
October of 2002 The Bali bombing and the Marriot hotel bombing in Jakarta
continued the trend. Two days after the Bali bombing on the 14th of
October 2002 the SMH (RW) reported in a headline that US ambassador to
Indonesia “saw the writing on the wall.” Mr Boyce, it said, warned of a
terrorist threat being hatched in Indonesia a month before the bombing, a
warning that has seen him repeatedly attacked by religious leaders and a host
of leading politicians, including Indonesia's Vice-President, Hamzah Haz. .
Our Mahfoud (SW) already saw the writing on the wall a year earlier and almost
two years before the Marriott hotel in Jakarta. This simulation also
established the ICC which occurred only in July 2002 (RW) and our IMF (SW)
rescheduled debt to Pakistan that only occurred in Oct 2002 (RW).
7. In October 2002 we ran a Worlds Politics in Transition Course simulation (60 students playing 31 roles), aimed at examining the ‘New Wars’ and the War on Terror. On October 15th 2001, the kick-start episode was published. It stated that Knesset member Benny Eilon from the right wing Moledet Party was assassinated - apparently by Palestinians - on his way home in the West Bank. The aim of the kick-start scenario was to prompt the ME actors to examine their position on terrorism – Israel has been assassinating Palestinian ‘terrorists’ commanders and leaders for some time and we wished to examine what would happen if an Israeli political leader was assassinated by Palestinians. Two days later, on the 17th of October 2001 the JERUSALEM POST (RW) reported the assassination of Tourism Minister Rehavam Ze'evi in Jerusalem hotel at 7:00 a.m. that morning. [U10] Ze'evi was regarded as the most right-wing member of the Israeli government. He entered politics in 1988 as the head of Moledet Party. Spooky!!!
8. Run in May 2002 in the Asia Pacific International Politics Course (60 students playing 31 roles) the simulation was aimed at exploring the disintegration and fragmentation caused by regional conflicts in Southeast Asia, and in particular Indonesia and the Philippines. On the 17th of May the Jakarta Post (SW) reported that President Sukarnoputri declared a state of emergency in Indonesia as a response to the upsurge of violence attributed to GAM in Aceh, and violence in the regions of West Papua and Maluku. Emergency powers were given to the military to quell the violence. Four months later in September 2002 TAPOL Bulletin Online 168 (RW) reported on the Indonesian government raising the issue of declaring an emergency in Aceh. That Bulletin argued that since July the government's move to raise the issue is more to do with the political situation in Jakarta where the TNI headquarters is increasingly calling the shots. [U11] By June/July 2003 the US, Australia and NZ have all declared travel alerts to Indonesia.
9. The final example comes from a ten day simulation that began on October 1st, 2002, in a Social Development Planning and Management/ Development Management Course at the University of Wales, Swansea (35 students playing 17 roles). The object was to examine the relations between civil society and government in developing countries. Venezuela was chosen because it exemplified a potentially rich country with large poor population, where the government was at odds with the middle classes. The kick-start episode opened with mass protests like the ones that brought a failed coup four months earlier in April. On the 11th of October, the day the simulation ended with Chavez still in power (SW), Newsmax.com (RW) reported that “As many as a million protestors marched through Caracas, Venezuela yesterday demanding that Castroite President Hugo Chavez resign. The rally was the biggest show of opposition to President Chavez since he survived a coup attempt in April.” [U12]
Perhaps separately each of the above examples is easy to explain away - but taken together a creeping suspicion arises that there may be more here than meets the eye.
As pedagogy it clearly demonstrates the utility of this open-ended approach that deals not only with past cases (upon which many scenarios were build) but also with possible future ones. Would such exercises not help organizations represented at PISTA in preparing for future events?
We cannot say why or which particular events and situations will arise in such simulations but we know that at least some will have a certain measure of accuracy.
In some of these cases above the initial scenario itself portrays events that actually occur not long afterwards (coup in Venezuela, the Australian soldier killed on the border with West Timor, the assassination of a leading Israeli political figure by Palestinians and others.) This occurred in other simulations not discussed here as well. In other cases the interaction between roles generates events that prove to be ‘prophetic’ like the Solomon Islands coup.
So how do we explain this? One possibility is that given the scenario in all cases was based on real events, the action by participants would, among other trajectories, also ‘incidentally’ generate ones that eventuate – as a normal distribution of probabilities. This seems the most likely explanation. But how exactly can we decide what a ‘normal distribution of probabilities’ looks like for a complex strategic activity. Do we consider each of the 22,000 messages an ‘action’ ?
A second line of explanation that perhaps expands on the first is that given the pressures that are modelled in the simulations participants enact solutions that real world actors eventually also enact. Political activity in the world is clearly not a random affair. It has clear trajectories based on the goals and interactions of actors and is couched within historical forces shaping their context. Similarly in our simulations the initial scenario that frames the interaction of roles, the interaction spaces, the role-profiles and resources undoubtedly limit and provide clues as to the sort of activities players can choose.
A third line of explanation accepts the first but in contrast to the second focuses on the fact that the multi-user approach - the collaborative thought experiment - allows possibilities that would simply not occur to the individual analyst/ researcher. As such events and situations that seem unrealistic at the time to an individual analyst, surface in the collaborative context - “The best way to have a good idea is to have a lot of ideas.” 
A Fourth line focuses on the credibility stakes for analysts making predictions. They do not particularly like giving precise predictions for particular events as it may stretch their credibility when predictions do not prove correct. Consequently we are unlikely to find such exact predictions in the public domain though they may have actually been considered.
We cannot yet determine which of these answers carries greater weight though all of them are probably involved. Nor can we be certain that a combination of these answers is sufficient to explain the phenomenon, or even some other answer.
A more detailed and systematic study needs to be carried out in order to give better answers. What is clear is that this pedagogy and platform does produce surprising results – some of which are fairly accurate with regard to real world events.
Can we design simulations that would better focus on specific events to be predicted? Can the dynamics in the production of political events and situations be better understood using these kinds of simulations? Can we assign levels of probability to some outcomes as opposed to others? Much more research needs to be done to be able to answer these questions – we are only at the beginning.
Today there is no theory or methodology to deal with such phenomenon and that give us answers. Thus we are left at the gates of might be a new field for the study of the political.
For this writer, the promise of better modelling techniques using this sort of pedagogy that enables better prediction of specific future events and situations is alluring and exciting. No doubt it’s a promise which if fulfilled will revolutionize the way the political is investigated by researchers, students and policy makers.
The ICT revolution has clearly impacted on the collaborative enterprise we know as science. The capacity of the tools this revolution has generated already made a significant impact on modelling objects for scientific study.
The exploration detailed in this paper suggests that we may be on the road of being able to better model the political using this type of collaborative thought experiment. If this turns out to be true, one can certainly wager that the political world as we know it will itself be transformed by the actions taken on the basis of such simulations.
Dr. Andrew Vincent at Macquarie University who introduced me to e-mail based role-play simulations, Dr. Peter Shearman and Dr. Derek McDougall of the Political Science Department at the University of Melbourne for using these simulations in their courses and Dr. Mat Sussex and Ali Jarman who moderated many of these simulations.
 C. Aldrich Simulations and the Future of Learning, John Wiley & Sons, San Francisco, 2004, pp. 23-29.
 L.P. Bloomfield & A. Moulton, Managing International Conflict, From Theory To Policy: A Teaching Tool Using CASCON, St. Martin Press, New York, 1997, pp. 98-114.
 As  pp.127-129.
 R. C Schank, & C. Cleary, Engines for Education, Lawrence Erlbaum Associates Publishers, Hillsdale, New Jersey. ("hyper-book") 1995
 A. Vincent, & J. Shepherd, "Experiences in Teaching Middle East Politics via Internet-based Role-Play Simulations". Journal of Interactive Media in Education, 1998 (11) (online: www-jime.open.ac.uk/98/11
 R. Linser, S. Naidu, & A. Ip, “Pedagogical Foundations of Web-based Simulations in Political Science”, ASCILITE Conference Proceedings, QUT, Brisbane, 5-8 Dec. 1999.
 R. Linser, & A. Ip, “Creating Learning Opportunities Using an RPS Authoring Tool” AUSWEB04 Conference Proceedings Surfers Paradise, Brisbane, 3-7 July 2004
 As 
 Dr. Linus Pauling quoted in  pp. 46.
‘REAL WORLD NEWS’ URL LIST:
For a full text comparison of the examples above and others see www.simplay.net/papers/data/ .
* U# see References to Real World News URL list at the end.