A literature review of Empirical Evidence on Procedural Content Generation in Game-Related Implementation <s> Source Explanation <s> Balancing a game is a long process and relies mainly on subjective feedback from human testers and selective interpretation from game developers. As a first step for completely automate game balancing, we propose a methodology to algorithmically choose features and calibrate their parameters for the procedural level generation of a simple runner game based on testers' feedback.This methodology is used in a 30 seconds game demo with survey and each playthrough is recorded and fed to a reinforcement learning algorithm. We show that the average fun grade steadily grows, proving the effectiveness of the proposed method. The collected data can be further analysed for insights on new features and other major changes. <s> BIB001 </s> A literature review of Empirical Evidence on Procedural Content Generation in Game-Related Implementation <s> Source Explanation <s> Self adaptive video games are important for rehabilitation at home. Recent works have explored different techniques with satisfactory results but these have a poor use of game design concepts like Challenge and Conservative Handling of Failure. Dynamic Difficult Adjustment with Help (DDA-Help) approach is presented as a new point of view for self adaptive video games for rehabilitation. Procedural Content Generation (PCG) and automatic helpers are used to a different work on Conservative Handling of Failure and Challenge. An experience with amblyopic children showed the proposal effectiveness, increasing the visual acuity 2-3 level following the Snellen Vision Test and improving the performance curve during the game time. <s> BIB002 </s> A literature review of Empirical Evidence on Procedural Content Generation in Game-Related Implementation <s> Source Explanation <s> This paper proposes an automatic way of evolving level generators for arbitrary 2D games, which are described in the Video Game Description Language (VGDL). The process works as follows: a game described in VGDL is interpreted and transformed in a set of rules defined in Answer Set Programming (ASP), along with other general and customizable rules. Although a set of rules described in ASP can generate multiple levels, not all of them will be playable or well designed. Therefore, an evolutionary process is run to determine the values of the parameters of those customizable rules. The different level generators are evaluated with general video game playing agents, which are able to play any game and level in the framework. The aim is to maximize the difference between their performance in the levels generated, under the assumption that levels are better designed if good skilled players play better than poor agents. This work presents some initial experiments that suggest that it is possible to evolve interesting level generators using this technique, and outlines some lines of future work. <s> BIB003 </s> A literature review of Empirical Evidence on Procedural Content Generation in Game-Related Implementation <s> Source Explanation <s> To understand the impact of emotionally driven games on player experience, we developed a procedural horror game (Vanish) capable of run-time level, asset, and event generation. Vanish was augmented to interpret players’ physiological data as a simplified emotional state, mapping it to a set of adaptation rules that modify the player experience. To explore the effects of adaptation mechanisms on player experience, we conducted a mixed-methods study on three different versions of the game, two of which integrated varying biofeedback mechanisms. Players’ affective experiences were objectively measured by analysing physiological data. Additionally, subjective experience was recorded through the use of the Game Experience Questionnaire. Our study confirmed that biofeedback functionality had a statistically significant effect on the ratings of player experience dimensions: immersion, tension, positive affect, and negative affect. Furthermore, participants reported noticeable differences in player experience, favouring the added depth present in the biofeedback-enabled iterations of the game. In the future, these conclusions will help to develop more immersive and engaging player experiences. <s> BIB004 </s> A literature review of Empirical Evidence on Procedural Content Generation in Game-Related Implementation <s> Source Explanation <s> In this paper, we propose a hybrid method to procedurally generate terrains in Scorch Earth. It features both the fast computing speed of random techniques and the wide variety of details of fractal techniques. We indirectly evaluate terrains by analyzing the gameplay data. We create an AI agent to model players' behaviors to produce gameplay data. The agent is capable to shoot and move in different scenarios. Finally we introduce metrics for gameplays to select good terrains. <s> BIB005 </s> A literature review of Empirical Evidence on Procedural Content Generation in Game-Related Implementation <s> Source Explanation <s> In this paper, we propose a game adaptation technique that seeks to improve the training outcomes of stroke patients during a therapeutic session. This technique involves the generation of customized game levels, which difficulty is dynamically adjusted to the patients’ abilities and performance. Our goal was to evaluate the effect of this adaptation strategy on the training outcomes of post-stroke patients during a therapeutic session. We hypothesized that a dynamic difficulty adaptation strategy would have a more positive effect on the training outcomes of patients than two control strategies, incremental difficulty adaptation and random difficulty adaptation. To test these strategies, we developed three versions of PRehab, a serious game for upper-limb rehabilitation. Seven stroke patients and three therapists participated in the experiment, and played all three versions of the game on a graphics tablet. The results of the experiment show that our dynamic adaptation technique increases movement amplitude during a therapeutic session. This finding may serve as a basis to improve patient recovery. <s> BIB006 </s> A literature review of Empirical Evidence on Procedural Content Generation in Game-Related Implementation <s> Source Explanation <s> Abstract TrackGen is an online tool for the generation of tracks for two open-source 3D car racing games (TORCS and Speed Dreams). It integrates interactive evolution with procedural content generation and comprises two components: (i) a web frontend that maintains the database of all the evolved populations and manages the interaction with users (by collecting users evaluations and providing access to the evolved tracks) and (ii) an evolutionary/content-generation backend that runs both the evolutionary algorithm and generates the actual game content that is available through the web frontend. The first prototype of the tool was presented in July 2011 but advertised only to researchers; the first official version which generated tracks only for TORCS was released to the game community in September 2011; due to the many requests, we released a new version soon afterwards, in January 2012, with support for Speed Dreams (the fork of TORCS focused on visual realism and graphic quality) that has been online since then. From January 2012 until July 2014, TrackGen had more than 7600 unique visitors who visited the website around 11,500 times and viewed 85,500 pages; it was employed to evolve more than 8853 tracks, and it was used to download 12,218 tracks. Some of the tracks evolve by our system have been also included in the TORCS distribution. <s> BIB007 </s> A literature review of Empirical Evidence on Procedural Content Generation in Game-Related Implementation <s> Source Explanation <s> While personality is known to moderate the kind of video games people like to play, the link between personality and game mechanics has not been explored. Finding such a link can enable customizing game mechanics based on personality, potentially making games more enjoyable and suited to a wide range of players. The present work investigated the relationship between the Big Five personality traits and four different implementations of difficulty adaptation, a popular game mechanic, in a first-person shooter game. In Study 1, a linear regression model was derived to relate the five personality traits with enjoyment (ENJ) and gameplay duration (DUR) in the four difficulty adaptations. In Study 2, this regression model was used to construct a predictor that chose a difficulty adaptation which maximized ENJ and DUR based on player personality. The predictor was tested against dynamic difficulty adjustment (DDA), which matched difficulty to user performance. ENJ and DUR were significantly higher in the prediction group than the DDA group. The present work highlights the importance of difficulty adaptation as a game mechanic, and suggests that personality could also be related to other game mechanics. Accordingly, a framework for personality-based game mechanics customization is proposed to foster future research. <s> BIB008 </s> A literature review of Empirical Evidence on Procedural Content Generation in Game-Related Implementation <s> Source Explanation <s> This paper presents a framework and an initial study in general video game level generation, the problem of generating levels for not only a single game but for any game within a specified range. While existing level generators are tailored to a particular game, this new challenge requires generators to take into account the constraints and affordances of games that might not even have been designed when the generator was constructed. The framework presented here builds on the General Video Game AI framework (GVG-AI) and the Video Game Description Language (VGDL), in order to reap synergies from research activities connected to the General Video Game Playing Competition. The framework will also form the basis for a new track of this competition. In addition to the framework, the paper presents three general level generators and an empirical comparison of their qualities. <s> BIB009 </s> A literature review of Empirical Evidence on Procedural Content Generation in Game-Related Implementation <s> Source Explanation <s> Abstract Would it be possible to bring the promise of unlimited re-playability typically reserved for Roguelike games to competitive multiplayer shooters? This paper tries to address this issue by proposing a novel method to dynamically generate maps at run-time almost as soon as players press the Play button, while ensuring the features what players would expect from the genre. The procedures are simple and practically feasible to be employed in actual computer games. In addition, the work experiments the possibility of incorporating asynchronous game-play element into a multiplayer shooter with human imitating bots where the players can let their bot/avatar replace them when they are not around. The algorithms are implemented and evaluated with a playable game. The evaluations prove that playable 3D dynamic maps can be generated in order of seconds using game context data to initialise the parameters of the algorithm. The paper also shows that asynchronous game-play element is a possible feature for consideration in next generation multiplayer shooters. <s> BIB010 </s> A literature review of Empirical Evidence on Procedural Content Generation in Game-Related Implementation <s> Source Explanation <s> This paper presents a coevolutionary framework for procedural generation of NPCs in MMORTS games. In this context, players need to defeat environmental troops in battle to gather resources and achieve their goals. The benchmarked game has several balance problems related to these battle activities, mostly caused by the handcraft design of complex game content. To solve this problem, the algorithm takes player modeled heroes as input and returns a solution evolved to win. By this way, the players need to think better in a new way to conquer the victory, adding new levels of challenge, keeping the game enjoyable. We present a new mathematical model to evaluate the solutions, based only on the number of soldiers on the input and output, making it easy to extend to other contexts. The results show it is possible to procedurally generate thousands of new efficient and fair builds, without violating the game rules. Moreover, our analysis of the results was able to identify unbalanced characteristics in the game design and we suggested simple way to fix it. <s> BIB011 </s> A literature review of Empirical Evidence on Procedural Content Generation in Game-Related Implementation <s> Source Explanation <s> The use of statistical and machine learning ap­proaches, such as Markov chains, for procedural content generation (PCG) has been growing in recent years in the field of Game AI. However, many of these level generation approaches account for only the structural properties of the levels. We developed multi-layered representations of levels, where each layer is designed to capture distinct gameplay information. Specifically, we experiment with representing levels using three layers: a structural layer that captures the object placements in the level, a player path layer that captures the path of an agent through the level, and a height layer that captures sections of the level as various height groups. We test our approach by generating levels for Super Mario Bros. with a multi-dimensional Markov chain approach. We compare the levels sampled with our multi-layered representation with those sampled using a single- layer approach that only considers the structural layer. <s> BIB012 </s> A literature review of Empirical Evidence on Procedural Content Generation in Game-Related Implementation <s> Source Explanation <s> The construction of dungeons in typical action-adventure computer games entails composing a complex arrangement of structural and temporal dependencies. It is not simple to generate dungeons with correct lock-and-key structures. In this paper we sketch a controllable approach to building graph-based models of acyclic dungeon levels via declarative constraint solving, that is capable of satisfying a range of hard gameplay and design constraints. We use a quantitative expressive range analysis to characterise the initial output of the system, present an example of the degree to which the output may be altered, and show a comparison with an alternate approach. <s> BIB013 </s> A literature review of Empirical Evidence on Procedural Content Generation in Game-Related Implementation <s> Source Explanation <s> Abstract Current video games use simple methods to deal with interactive narratives and the enormous variety of player types. In this paper, we propose a novel approach to interactive storytelling in games, in which the quests and the ongoing story are determined in view of individual personality traits and behavioral attitudes in a nondeterministic way. Our method starts the process employing a new technique to assess the player’s personality traits according to the well-known Big Five model. These traits are then used by a nondeterministic planning algorithm to define adaptive goal hierarchies. In addition, an artificial neural network is trained to predict player behaviors in real-time, allowing partial-order planning operators to use player behaviors and personality traits as logical terms in their preconditions. With this approach, a richer individualized experience is provided to the player, while preserving consistency with the conventions of the chosen genre. <s> BIB014 </s> A literature review of Empirical Evidence on Procedural Content Generation in Game-Related Implementation <s> Source Explanation <s> Procedural Content Generation (PCG), i.e. how game content can be created algorithmically, is an increasingly important area and currently one of the most active topics within the software games industry and game research. One of the crucial aspects of PCG is the capacity to maintain the player engaged and in flow. In this work, we explore how player skill progression could be used by PCG to create more appropriate challenges for each player and propose a model for content adaptation that takes this concept as its core feature. Our approach introduces the player as an active element in the adaptation process and assumes both the player and the game should have an equal and active role in this process. Our adaptation explores how modelling the evolution of multiple dimensions of a same challenge while the game is played helps creating a better game experience for the player. To evaluate our approach, we present a novel validation process embedded in the game itself, with the purpose of providing a more direct and seamless way to analyse player preference. The results of the evaluation of our approach in the context of an endless running side-scrolling platformer game revealed that players have consistent and specific preferences regarding how difficulty should evolve over the course of a game, which should be taken into account when designing an engaging game progression. <s> BIB015 </s> A literature review of Empirical Evidence on Procedural Content Generation in Game-Related Implementation <s> Source Explanation <s> Generating feasible levels for physics-based puzzle games is a complex and time-consuming task. This is because the mechanics of these games are based on realistic physics, so simulations are required to evaluate the playability of the generated levels. Recently, a few generators have been able to produce considerably complex levels in the context of the Angry Birds game, a very famous title of this genre. However, none of these generators are able to guarantee the playability of their produced levels. This paper presents Tanager, a level generator based on a genetic algorithm that is capable of producing feasible levels for the Angry Birds game. Evaluating playability in this game requires checking both the stability of the stacked blocks and the possibility of killing all the pigs with the given amount of birds. These two components are handled by the algorithm through a simulation. The first one is calculated by measuring the overall velocity of the blocks and the second is defined by an intelligent agent, which plays the levels. Three sets of experiments are conducted to evaluate Tanager. The first one measures the performance of the genetic algorithm underneath Tanager. The second one explores the expressivity of the generated levels considering their structural characteristics. The third one measures design quality of levels via an online user study. Results show that Tanager is capable of generating a considerable variety of feasible levels that are as engaging and enjoyable as those manually designed. However, the generated levels are less challenging than the hand-authored ones. <s> BIB016 </s> A literature review of Empirical Evidence on Procedural Content Generation in Game-Related Implementation <s> Source Explanation <s> Automated content generation for educational games has become an emerging research problem, as manual authoring is often time consuming and costly. In this article, we present a procedural content generation framework that intends to produce educational game content from the viewpoint of both designer and user. This framework generates content by means of genetic algorithm, and thereby offers designers the ability to control the process of content generation for various learning goals according to their preferences. It further takes into consideration how the content can adapt according to the skill of the users. We demonstrate effectiveness of the framework by way of an empirical study of human players in an educational language learning game aiming at developing early English reading skills of young children. The results of our study confirm that users’ performance measurably improves when game contents are customized to their individual ability, in contrast to their improvement in uncustomized games. M... <s> BIB017 </s> A literature review of Empirical Evidence on Procedural Content Generation in Game-Related Implementation <s> Source Explanation <s> A key challenge in procedural content generation is to automatically evaluate whether the generated content has good quality. In this paper, we describe an approach that uses nonexpert workers to evaluate small portions of levels generated by an off-the-shelf generation system for the game of Infinite Mario Bros . Several such evaluated portions are then combined to form full levels of the game using a mathematical progression arc model. The composition of the small portions into full levels is done by accounting for the human-annotated information. We evaluated the approach using computational metrics as well as surveying human subjects playing the levels. The results show that the human computation approach is able to generate levels that are perceived by people to have better visual aesthetics and to be more enjoyable to play than existing approaches. Another contribution of our paper is a dataset of the small annotated levels that can be used in future research for learning models for evaluating machine-generated content. <s> BIB018 </s> A literature review of Empirical Evidence on Procedural Content Generation in Game-Related Implementation <s> Source Explanation <s> We present a search-based approach to generating boards and decks of cards for the game Ticket to Ride. Our evolutionary algorithm searches for boards that allow for a well-shaped game arc, and for decks that promote an equal distribution of desirability for cities. We show examples of two boards generated by our algorithm and compare our results to those of the actual components of the game. Our approach creates game content that is specifically designed towards metrics that can affect gameplay in an impactful way. <s> BIB019 </s> A literature review of Empirical Evidence on Procedural Content Generation in Game-Related Implementation <s> Source Explanation <s> Current research on adaptive games has mainly focused on adjusting difficulty in a variety of ways, for example, by providing some control over adaptive game world generation. These methods, however, are mostly ad-hoc and require quite some technical skills. To the best of our knowledge, so far, there has been no adaptive method that is truly generic and explicitly designed to actively include game designers in the content creation loop. In this paper, we introduce a generic method that enables designers to author adaptivity of game world generation, in a very expressive and specific fashion. Our approach uses adaptation rules , which build atop gameplay semantics in order to steer the online generation of the game content. Designers create these rules by associating skill profiles , describing skill proficiency, with content descriptions , detailing the desired properties of the specific game world content. This game content is then generated online using a matching and retrieval approach. We performed user studies with both designers and players and concluded that adaptation rules provide game designers with a rich expressive range to effectively convey specific adaptive gameplay experiences to players. <s> BIB020 </s> A literature review of Empirical Evidence on Procedural Content Generation in Game-Related Implementation <s> Source Explanation <s> Designers structure a game to provide a desired range of player behaviors: the play space of a game. Any given game is one instance from a design space of alternatives. Navigating a design space to achieve a designer's goals requires knowledge of how design choices shape the play space in a game. We present algorithms to automatically measure play patterns using statistical models that predict how design choices alter player behavior. We present Monte Carlo tree search as a way to sample behaviors from a play space, action metrics to automate play space measurement, and predictive modeling techniques to model design spaces. We demonstrate these techniques in two simplified, perfect information, adversarial game domains based on Scrabble and Hearthstone showing their use for automated design space modeling. <s> BIB021
Answer Set Programming (ASPP BIB013 , BIB003 Answer set programming (ASP) is one of the existing programming approaches that can be used to deliver PCG contents, it is a specific programming approach that is used for solving combinatorial search problem, such that involves a search tree based on known facts related to the problem and rules. Such method has been  observed to be able to create contents such as a dungeon level generation BIB013 . Another research also has been done by combining ASP with evolutionary algorithm to generate and optimize maps in a level generation by utilizing ASP logical nature BIB003 . Artificial neural network (ANN) BIB014 Artificial neural network (ANN) is a form of computing system that is designed to mimic how the human brain works, instead of calling it a model or an algorithm, it works more like a framework due to its nature of applying multiple machine learning algorithm on its process, nevertheless, on this context, ANN is considered to be a method of delivery to create a PCG outputs. As a framework, the work is usually done by combining the said framework with multiple other algorithm, one work combines Big-Five model, nondeterministic planning algorithm, along with ANN to create a personalized interactive storytelling, by using the player behavior to generate player-specific quests BIB014 . Coevolutionary genetic algorithm (CGA) BIB011 Coevolutionary genetic algorithm (CGA) is an advanced form of evolutionary algorithm that utilize a subjective fitness value, such that the evaluation of the individuals in the algorithm are evaluated based on its interaction with other individuals. CGA usage in delivering PCG are more focused on decomposing complex design problem by using such algorithm to generate a solution while avoiding early convergence state, thus creating a more optimal result. Creation of A.I. in game by using CGA has been proven to be optimal BIB011 , creating a more challenging and fun game to play with. Dynamic Difficulty Adjustment (DDA) BIB001 , BIB008 , BIB002 , BIB004 Dynamic Difficulty Adjustment (DDA) is a popular form of PCG deliverance method due to its simple mechanism, more focusing on creation of logical aspect of the generated content in form of game difficulty, DDA has been seen to be used in multiple researches over the years. DDA mechanism works by using player's ability in the game and use it as the input to adjust the game difficulty dynamically, targeted to create an engaging game experience that is neither boring or too difficult for the player. One of the works that has been done focuses on using DDA by combining it with Big-Five model to create a level with various difficulty based on player enjoyment and duration BIB008 . DDA is also used to balance the game by punishing player by spawning obstacles based on player's tendency to abuse a certain game mechanic BIB001 . DDA has also been proven to be useful on rehabilitation games where therapy session difficulty is adapted by player's current achievement on finishing a task in the rehabilitation BIB002 , such that the difficulty does not always goes up incrementally, but instead adapts to the player physical condition and result of previous therapy. A combination of DDA with indirect biofeedback (IDF) has also been done to create an immersive procedural horror games by using player's biological and emotional reaction to generate events in the game BIB004 . DDA adaptability is thanks to its simple mechanism of adapting a certain type of method and easy adaptation with other algorithm or models to deliver PCG. Experiencedriven procedural content generation (EDPCG)  , BIB005 , BIB015 Experience-driven procedural content generation (EDPCG) is a more generic PCG which uses player experience to generate a content, rather than specifying it like DDA or evolving a solution like CGA. As such, any input that is experienced by the player such as basic gameplay experience BIB005 , activity and skills BIB015 , or even an external data such as after-game questionnaire  can be considered to be a EDPCG. Due to its generic choices, EDPCG is rather too flexible if not defined or focused into a more specific approach, but its usage has been proven to base other existing methods. Genetic algorithm (GA) BIB016 , BIB017 Genetic algorithm (GA) is a more general form of CGA which uses the concept of natural selection and rules to generate solution based on given population (called chromosome) and changing its value by using mutation and crossover, in PCG creation, rather than using the player action as an input like DDA and EDPCG, GA uses existing level as a base to generate more level with similar difficulty or similar content BIB016 . While it is also not impossible to use player input as a form of chromosome for GA selection process, a research has been done in this form of GA implementation in an educational game BIB017 to generate questions by using PCG by using GA approach. Human-in-theloop BIB018 Human-in-the-loop approach is a form of PCG creation method that uses human input to interfere the result and adjust the result rather than using algorithm to fully do the generation process. This might sound counterproductive, but it has been proven to be able to create a much more refined result by increasing the aesthetics and difficulty of the PCG output. Due to the rigid nature of algorithms, the aesthetics of the level generated by an algorithm looks less appealing albeit having a considerable level of difficulty, human-in-theloop approach improves such problem by doing a minimal adjustment to create a visually pleasing output BIB018 . Monte-Carlo tree search (MCTS) BIB021 , BIB006 Monte-Carlo tree search (MCTS) is an algorithm consisting of selection, expansion, simulation, and backpropagation used in a combinatorial problem for games with multiple viable solution that changes every with every action done such as chess. Due to its stochastic nature, MCTS are able to measure play patterns and create a design space based on the patterns Method/ Algorithm/ Approach Name Source Explanation BIB021 . MCTS also has been applied on rehabilitative games to create a game quest as well BIB006 . Search-based procedural content generation (SBPCG) BIB009 , BIB007 , BIB019 , BIB010 Search-based procedural content generation (SBPCG) is the counterpart of EDPCG which focuses on the searching aspect of PCG rather than using the experience as the driving factor to create a PCG. As such, it is a very general type of PCG which usually combined with multiple algorithm albeit not necessary. SBPCG has been done to generate levels BIB009 and maps BIB007 , BIB019 , BIB010 , but often are more geared towards a pilot study due to its nature of searching instead of using player experience as the major decision choice. Platforming Games BIB016 , BIB020 , BIB012 , Based on the result, majority of the papers researched are focused on platforming games, the usage of PCG on the papers found are focused on several key contents of the