PCG has been used in interactive stories in the past, but usually in attempts to generate entire stories, plot points, or lines of dialogue from scratch. Rather than doing something so ambitious, instead I’m trying a sort of experiment: what’s the minimum amount of a generated text that could be inserted into an otherwise hand-authored story to produce something that’s both authorially sound and computationally interesting?

My approach with Goodbye is to generate the “satellite” sentences of a story (as opposed to the “kernel” sentences that move forward the plot) during dialogue scenes. These include all of the little bits controlling pacing between speakers (sentences like “He waited” or “There was a pause”) as well as the ones re-establishing the setting (“The moonlight shone on his face” or “Traffic growled from somewhere nearby”). While these sentences seem inconsequential at first, they can have a surprisingly strong impact on a reader’s perception of a scene. They are also relatively easy to procedurally generate compared to other types of prose sentences, and are amenable to variation based on the current narrative context (time of day, location, current speakers, mood, and the moment-to-moment rhythms of a conversation). The consequences of past player choices (such as changes undergone by the narrator) can also be factored in to the construction of these sentences. The result is (hopefully) a story that is personalized to the way you’ve been interacting with it in subtle yet constant ways, sentence by sentence… a different model than the large but infrequent consequences often seen in interactive narrative (i.e. getting one chunk of content instead of another).

You can play Goodbye at the link above; it takes about ten minutes to read through. The piece is a selection for “Avenues of Access,” an exhibit of new electronic literature that will be part of the Modern Language Association’s next conference, but I’ve received permission to post it online here early. Comments are always welcome. The curious can also read more about the technical details in a paper to be presented at the upcoming Workshop in Procedural Content Generation at the 2012 Foundations of Digital Games conference.

In 2010, the National Science Foundation and National Endowment for the Arts convened a historic workshop — it was their first jointly funded project. This meeting marked the beginning of a new level of national conversation about how computer science and other STEM disciplines can work productively with arts and design in research, creation, education, and economic development. A number of projects and follow-up workshops resulted in 2011. I was lucky enough to attend three of these events and, in the midst of all the exciting follow-up conversations, I couldn’t help but wonder: What about the digital humanities?

After all, the digital humanities have made it now. A recent visualization from University College London shows more than 100 digital humanities centers spread across the globe. There are dedicated digital humanities funding groups within the National Endowment for the Humanities and Microsoft Research. The University of Minnesota Press published a book of Debates in the Digital Humanities in January.

So why doesn’t the digital humanities have more of a seat at the table? Why is there the stereotype that, while computer scientists and digital artists have much to discuss, digital humanists only want to talk about data mining with the former and data visualization with the latter? I believe it is because the perception has developed, helped along by many in the field itself, that digital humanities is primarily about data.

Read more…

The set of lessons about how others see the book come mostly from reading reviews. A number have been published since my last post on Expressive Processing reviews. In the rest of this post I’ll post my favorite excerpts from reviews (including those behind paywalls) and then offer some thoughts.

While scholars of new media will no doubt find Wardrip-Fruin’s discussion useful, one goal of the text is to reach beyond the relatively small conversations of software studies (an emerging strand of new media scholarship) and digital fictions. Within this broader project, we might locate a promising expansion of the various political projects of open source and free software.

— James J. Brown Jr., from “Open Process Software” in Criticism (an unofficial-looking version is online). This essay looks toward Expressive Processing‘s call for more people to be able to think critically about software operations, and its observations of how some software itself can help develop our understanding of software processes, situating EP relative the goals of the free and open source software movements.

There truly is treasure buried in this land of geekdom, and not just a few nuggets, but enough to lay the foundation of an entirely new scholarly approach for the digital humanities…. If Manovich drew the map, Wardrip-Fruin has opened the mine, and what may be extracted will benefit not only those working in digital humanities or new media but scholars across the curriculum.

— Doug Reside, from a review in Digital Humanities Quarterly (open access!) which also focuses on the issues of understanding software, taking up particularly the question of what kinds of literacies are needed for interpretation and project guidance in the digital humanities.

I believe that the groundbreaking approach this book offers will help humanists and computer scientists alike discover the potential of computational processes and digital media for the advancement of digital humanities. An invitation to embark in this fascinating journey is what Wardrip-Fruin accomplishes with Expressive Processing.

— Carlos Monroy, from a review in Literary and Linguistic Computing which is unfortunately behind a paywall. It is interestingly one that comes from a computer science direction, though written by someone currently working in digital humanities.

In Wardrip-Fruin’s Expressive Processing, the field of “interactive entertainment” comes of age; its theories and methods are native to its medium, rather than borrowed from literature, film, or history…. He provides, then, a way to analyze this new kind of authorship that takes into account the scripting of dynamic and interactive processes.

— Annette Vee, from a thoughtful review essay about EP and Ian Bogost’s Persuasive Games titled “Procedural Rhetoric and Expression” for the composition theory journal JAC. Unfortunately, I don’t think it’s available except by getting a physical copy of the journal or doing a search through pay-access databases.

[Wardrip-Fruin's] wide interests and expertise, ranging from early computer games to artificial intelligence experiments and the most sophisticated electronic literature works, enable him to demonstrate the general value of the notion of expressive processing throughout various cultural and academic fields. As such, this book is the perfect volume to begin the new publication series in software studies. Rather than building the theory for software studies, it works as a model of how to do software studies.

— Raine Koskimaa, from “Reading Processes: Groundwork for Software Studies,” a detailed, thoughtful review in Game Studies (open access!).

Overall, I feel quite lucky to have had reviews in these five academic journals. Even just readership of the reviews has, I’m sure, helped expose more people to aspects of the Expressive Processing project. At the same time, it’s also interesting to note how the attempt to create a book that connected a number of different areas of concern allows for quite different interpretations. For example, some reviews treat the book’s political project as primary, while others explicitly see it as secondary (one actually says Wardrip-Fruin “buries the idea in the center of the book”). Similarly, some treat the theoretical ideas (e.g., the three “effects”) as the primary contributions, while others explicitly treat the theoretical work as secondary to the individual interpretations. Finally, some see the focus on games and digital fictions as unbearably geeky, while others see the book as primarily of interest in the discussions around those topics.

To me it is exciting to reach people who see the book from so many different perspectives. My hope is that I have also succeeded in at least planting seeds that will help grow an interest in other ways of seeing the subfields I hope to connect through Expressive Processing and future work.

We put our slides together on Jesper’s computer, so I don’t have them all, but here are mine with my presenter’s notes (what I actually said varied, of course).

The nice thing about teaching game history now is that we’re very close to agreeing on the list of essential games, from around the world, that students need to master in an introductory game class

Okay, not so much. How many of you have taken an “introduction to literature” class? You probably remember that the class was not a march through “the essential works of literature” the world has produced. Similarly, introduction to film courses are not a march through “the essential works of film” the world has produced. We need to give up on the idea of identifying the key games that students need to know. We can’t cover even the most minimal list in an introductory class, even if we could agree on how to make it. Our introductory classes should be about method and approach, not about becoming familiar with some list of material. And to teach such classes we need a different approach to game history than focusing on highly-influential games or systems.

One essential viewpoint is “diachronic” — changing through time. Types of gameplay, like other media styles and genres, influence each other over time. Tracing this is a way of practicing historical method, and getting into the specifics helps undermine the fetish of “first” (notice Jesper’s family tree of matching tile games has two top nodes). Tracing this is also a way of understanding the process of game design and innovation. Designers play other games, respond to them, and borrow from multiple traditions. Games are also part of wider culture. The favorite games of particular communities, historically, produce genre literacies and take on affective charge. Other games build on this much more effectively than, say, movies.

Another essential viewpoint is “synchronic” — in a particular slice of time. That slice doesn’t have to be a particular year or a particular decade. For example, it can be the time during which the 2600 was a popular platform or text adventures were a dominant genre. Synchronic viewpoints help us narrow in on design spaces, to understand the technical and artistic creativity that went into responses to them, and see how they are situated in history and culture. A synchronic view helps us see how amazing it was to take a machine made for Combat and use it to create Pitfall, as well as the economic and cultural forces that gave rise to ET. Understanding this can provide a new perspective on today’s game design spaces. For example, if Dance Central is Combat for the Kinect, what might be the path to Pitfall …. And what might be the path to ET?

Setup does work for you — seminar and lecture classes know about readings. Notice I don’t have to describe these assignments to you. Must set parameters, since students can’t finish Tetris, don’t need collaborators for most readings, etc. Must ensure legal access to all games, for all students, and give student timing flexibility.

Clara will say more about access strategies, including some used in this model. I worked with campus IT to install games, including ones that required being compiled in weird ways and purchased off the shelf, in general purpose labs. Some games could not be purchased in any legal way. I contacted rightsholders and got permission for some games. For example, Activision gave permission to install old Infocom games in the lab. Some games could not be installed in the lab — a disk was required. So I worked with my department’s existing equipment checkout to provide disks for students to check out. Some historical games were available commercially, so I just had students buy them from the bookstore, like they would any other readings.

Can you spot why I love the UC Santa Cruz library? It’s this: the sign says “Instruction Room, Gaming Lab, Reference Staff.” Game research is totally integrated into the library. Whitehead initiated ambitious work with library. Dedicated game playing room in library with historical systems always set up. Lending catalog of about 700 games and game systems. Just added what is probably the first-ever games-dedicated, library-lent iPad. A library research room and collection makes it easy for students to act like historians, but you can encourage the behavior without either, doing research online, etc.

One of the questions brought up after my talk at the Paris Game AI Conference was whether ABL is available for distribution and if there is documentation available. Currently, the ABL binaries are available for non-commercial use, but the source does not have a specific license assigned. There is documentation available for ABL, but it is spread across several sources. For an overview of the language and semantics, the ABL Wiki is a good starting point. For additional discussion on the usage of the language to author agents, there is Michael Mateas’ dissertation which includes a chapter on ABL, and shorter papers on authoring idioms in ABL for Façade and EISBot. ABL has also been discussed previously on the forums at AIGameDev.

The source code and ABL binaries for the tutorial are available here. The Zip file contains source for the game, ABL agent, and ABL interface as well as binaries for compiling and running the ABL agent. The goal of the tutorial is to provide a micro-domain for demonstrating the ABL interface for authoring game AI. The “game” is called ABL Chaser and involves movement and bullet shooting interactions. For this tutorial, I am creating an ABL agent that moves towards the player while simultaneously firing bullets at the player. ABL is used to control the Chaser NPC (red square), while the player controls the blue square. Features such as collision and score have been omitted to reduce the amount of code needed.

Compiling

ABL is a reactive planning programming language. Authoring game AI in ABL consists of defining sensors and actuators for the agent to interact with, as well as declarative behaviors which specify how the agent achieves goals. The ABL binaries consist of two parts: a compiler and a runtime component. The ABL compiler is a Java library that translates ABL code into Java code. An ABL agent which has been compiled to Java is referred to as a Behaving Entity. The ABL runtime component is a Java library for executing a Behaving Entity (ABL agent).

I used the Eclipse IDE to build this project, but it is not necessary to run the tutorial. The figure above shows the project structure, which includes a source directory for the game and ABL agent, as well as a lib directory which includes the ABL jar. The project also includes the higher-order java library (hoj.jar), which is necessary only during ABL agent compilation and not required by the ABL runtime component.

Compiling the ABL agent and game is a two step process. First, it is necessary to translate the agent code from ABL to Java. To perform this task, I have included a Compile class which is a simple script for compilation. The output of this process is a collection of generated Java files, which are placed in src/abl/generated. Second, it is necessary to compile all of the files, which include the game, interface, and generated code. Eclipse will handle most of the second step for you, but it is necessary to refresh the generated code directory after each ABL compilation to notify the IDE that a code update has occurred. Once everything has been compiled, the Chaser class can be used to run the game.

Agent Interface

An ABL agent interacts with a game world through the use of sensors and actuators. Sensors are used to perceive world state and store facts about the game world to the agent’s working memory, while actuators (primitive acts) are used to perform actions in the game.

The class Chaser provides several methods for interacting with the game. The following methods provide functionality for sensing game state and are used by the sensors:

• public Point getPlayerLocation();
• public Point getPlayerTrajectory();
• public Point getChaserLocation();
• public Point getChaserTrajectory();

To perform actions in the game, the Chaser class includes the following methods:

• public void setChaserTrajectory(Point trajectory);
• public void fireChaserBullet(Point source, Point target);

The middleware between the game and ABL agent includes defining working memory elements (WMEs) which are facts about the world, sensors for populating working memory with WMEs, and actuators for performing actions. The middleware provides an interface between the declarative behaviors specified in ABL code and the game.

There are several additional concerns in the middleware that are not covered in this tutorial. First, scheduling actions for execution is necessary for a reactive planning agent. In this domain we assume actions are performed instantaneously to reduce complexity. Second, the game has been simplified to minimize synchronization issues that can occur due to the ABL decision cycle thread running asynchronous to the game update logic.

Working Memory Elements (WMEs)

The base data structure in an ABL agent’s working memory is the Working Memory Element (WME). A WME provides a set of properties for describing an instance in the game world. The tutorial includes WMEs for tracking the player and the chaser objects. It is also possible to include additional game objects in working memory such as the bullets, but the current version of the agent does not incorporate bullets in its decision making process.

The src/abl/wmes directory contains the code for defining working memory elements. The two classes each extend the WME base class which provides functionality for storing data in the agent’s working memory. The PlayerWME has properties for describing the player character’s location and trajectory. Properties can be exposed to the ABL agent by defining Java Bean style getters.

Sensors

ABL sensors are used to perceive game state. In this tutorial there is a sensor for each type of WME that is monitored by the agent. The sensor code is located in the src/abl/sensors directory. Each time the sense method is invoked, the PlayerSensor creates a new PlayerWME which records the location and trajectory of the player character. While this example shows how to update working memory by creating new instantiations of WMEs, it is also possible to update the properties of an existing WME in the sense method. Working memory is accessed via static reference to the Behaving Entity. ThreadContext objects are used by the ABL runtime to enable multiple ABL agents to run within a single Java process.

Actuators

The actions the agent can perform are defined by actuators in the src/abl/actions directory. The project includes actions for firing, moving, and stopping the agent. Each action implements an execute method which invokes a method on the Chaser class.

An action has a set of input arguments, which are parameters that must be provided to the action when invoked by the ABL agent. The prototype of a specific action is defined in ABL Code.

In this example, the agent assumes actions complete instantaneously. The base action class in this example marks actions as completed as soon as the agent queries the completion status, which occurs at the beginning of each decision cycle. It is also possible to author durative actions by manually setting the completion status once a scheduler has completed executing the action.

Agent Behavior

The behaviors driving the agent’s goal pursing behavior are defined in ChaserAgent.abl. There are several parts to the file, which include package definitions, import directories, sensors used the agent, actions available for execution, and behaviors. Additional documentation about each part of the agent is provided in the ABL source file.

An overview of the agent’s active behavior tree during execution is shown in the figure above. The figure shows behaviors that have been selected for execution in pursuit of goals. The agent contains two main goals, which are to move towards the player and to occasionally fire bullets at the player.

Specificities and priorities are used to determine which behaviors to examine for execution. Priorities are used as a step modifier, and the step with the highest priority is selected for execution each decision cycle. Specificities on the other hand are specific to a particular goal, and when the agent retrieves behaviors for pursuing a specific goal, the behavior with the highest specificity and true activation conditions is selected for expansion. Vertical movement behaviors have a higher specificity than horizontal movement, which is why the chaser moves vertically first if necessary.

The ABL runtime applies a heuristic that prefers to evaluate the current line of expansion, which actually causes behavior thrashing in this simplified example. To ensure that the agent both fires and moves towards the player, a higher priority is assigned to firing, which includes a cooldown period.

Summary

This tutorial has provided an overview of the components and code necessary to interface an ABL agent with a game. More detailed comments are also provided in the source code. The complete system in action is shown in this video. While the example does not demonstrate the most complex behavior for this domain, it lays groundwork for incorporating complex decision making systems in game AI.

http://www.youtube.com/watch?v=p0-Gfpi7MWg

As we’ve posted before, Prom Week is a game where the player gets to shape the social lives of 18 highschoolers by controlling what social actions they take with one another. What each character wants to do, and how each character chooses to respond, is determined by over 5,000 social considerations.

Social considerations in Prom Week mirror the ways that characters think in fiction — the facts about the world that influence how they feel about each other. For example, if I am shy, I am going to be less likely to do something outgoing. Or if you’ve been mean to me, I’m going to be less likely to want to be nice to you.

Like real life, any single social consideration isn’t going to completely determine how a character in Prom Week will feel about another character. Social relationships are complicated, and sometimes people are mean to their friends, or feel compelled to flirt with their enemy. Prom Week‘s AI system, Comme il Faut, accounts for social considerations to bring this sort of richness to characters’ social feelings.

Come il Faut achieves this through the concept of the “social exchange.” A social exchange encapsulates a social action a character takes with another character (with the intention of changing their relationship) as well as how the other character responds. For example, someone might flirt with someone in order to make them feel romance towards them. A character’s desire to perform a social exchange is determined by social considerations. At any given point, each character has a ranked list of social exchanges that he or she desires to perform (Prom Week has over 40).

Once a player selects a social exchange for a character to perform with another, the second character, the “responder,” decides whether to “accept” or “reject” the intent of the exchange (e.g., someone might reject being asked out on a date, or accept someone’s compliment). Social considerations are also used to determine whether he or she accepts an exchange or not.

Each social exchange is associated with 20+ scenes of templated dialogue. Once the responder’s general response is determined, a scene is selected and instantiated to represent how the social interaction actually takes place (this will the the subject of a future post).

Social exchanges and considerations enable is players to choose from social actions that characters specifically desire to perform and the other characters will respond for specific reasons that respect the character’s personality and social context.

For example, below shows a social exchange where Cassandra tries to spread rumors about Naomi (one of her enemies) but Simon doesn’t believe her because he is better friends with Naomi than Cassandra.

Prom Week is can now be played on Facebook and Kongegrate! Stay tuned for more updates and information!

Everyone on the planet is rushing to add a social element to their game, picture app, music app, whatever app. Our widely held view of acceptable application development has narrowed to freemium social games that have to monetize. Somewhere along the way we forgot that games should be fun experiences and we became analysts. Let’s talk about that.

The talk is happening at 11:00 in the Simularium (E2 180), and promises to be an interesting take on the continuing social games phenomenon. It should be interesting to hear whether Graeme’s viewpoint on the subject agrees with Chelsea Howe’s, who spoke here two weeks ago about enchanting games and is a game designer for the social games company Social Chocolate.

The problem Emily tackles is one familiar to storytellers in interactive media: how is it possible to have meaningful interactions with other characters without creating an impossible authorial burden of endless branching conversation trees? In other words, how can we build a system that replicates something of the experience of interacting with other people, rather than authoring every possible interaction by hand in advance? The approach taken here is based on “social practice modeling,” tying NPC behavior to an expressive AI engine that understands interactions in a specific social milieu.

Designed to be something you could curl up by the fire and read on an ebook or laptop, the system presents an ongoing textual story that advances in real time as you read it. The player picks a character in the story whose role to adopt—perhaps the detective in a mystery for a very hands-on story, or a minor character (even the dog!) for a more passive experience—and can then take actions in the world based on the current social context and character’s place within it. For instance, as a guest at a dinner scene it would be appropriate to make small talk with those present, eat food, and share flirtatious glances, but not to abruptly rise and leave the house without provocation. Playing as a servant in the same scene, however, one might have very different affordances. (The player has some ability to go outside expected behavior through a pool of resource points.) The player’s available actions from moment to moment are also based on their relations with other characters and their personality: a shy person might have different conversational affordances than a loquacious one. These traits also drive the behaviors of the non-player characters, ensuring they take appropriate actions. Interestingly, the system is designed to support multi-player: the AI controls all remaining characters not played by a human.

Behind this framework lies a complicated map of characters’ desires and beliefs. Each character has a model of things they believe to be true, including opinions about other characters and their relationships with each other, like “Emma is nice but talks too much” or “Mr. Elton is in love with Harriet.” As characters observe other characters take actions that conflict with this model, these opinions are revised (“Mr. Elton is in love with Emma!”) and characters are able to verbalize or question these opinions in conversation. This allows for dynamic conversations that eschew a pre-scripted conversation tree in favor of multiple agents each changing the social landscape with each “move” they make, both advancing and evolving opinions about each other and the plot from moment to moment and scene to scene.

The system also tries to solve the problem of embedding these moment-to-moment actions into a larger narrative structure. For instance, characters are in part defined by traits, but at dramatic moments in a story, players can push characters to change by abandoning traits or acquiring new ones. For instance, a shy character trying to get up the nerve to propose marriage to a girl he loves might spend some of his in-game points to overcome shyness just to propose, or might spend a large amount of it to overcome his shyness permanently, for presumably a much more dramatic proposal. Players have the option of saving the developed version of their characters to play in other stories.

It’s interesting to note that many of the approaches in this project mirror those taken in Prom Week, released this week here at UCSC; Emily commented on these similarities several times in her talk. It’s not surprising, as both of us were trying to solve similar problems of building playable systems from social interactions. It’s great to see powerful expressive systems driving story and character starting to move beyond theory and into real playable experiences—in academia, the indie game scene, and the world of commercial games, too.

Play it now!!!