Carrie Yang

I have been working on an Mixed Reality project in the ETC called MediSIM (Medical Simulated Interactive Mannequin). It is a medical simulation tool using the HoloLens in conjunction with Abe, a physical abdominal simulator, designed for medical students to practice abdominal examination and prepare them for future examinations on patients during real world situations.

Since Microsoft only began shipping the development kit of the HoloLens on the 31st of March, we were working with the first generation of Meta, another augmented reality headset 2 years older than the HoloLens.

However, there were many limitations we faced compared to the HoloLens. The Meta 1 only offers a much smaller field of view which restricted the display by a considerable margin. It was difficult to see the display using the headset due to the resolution and it was extremely uncomfortable to put on even for a short period of time. Despite that, even while developing on this preliminary headset, we received overwhelmingly positive feedback from play testers. Medical students who tried our prototype were very excited about it and felt that it would be immensely useful to aid in their learning experiences. The faculty at ETC also felt that the experience will be meaningful and practical.

Since I have been working with Augmented Reality, I would like to share some of my thoughts about it.

First of all, it is very suitable for educational or medical purposes, where learning by practicing in context is essential. By using Augmented Reality, those contexts can be created and put into real world simulations. Hence, I think transformational games will work really well in Augmented Reality since it provides the opportunity to make things happen in front of you or make impossible possible. If combined with a physical simulator, like Abe, or physical props, the experience will be even more immersive and realistic, providing a way for the player to grasp the knowledge or skill faster. For example, in our project, we superimposed a virtual patient in the augmented world with detailed symptoms so that medical students can not only practice their abdominal examination skills on the physical simulator, but also diagnose the virtual patient. This simulation cannot be realized in the real world because it is impossible for students to find patients readily.

The power of Augmented Reality lies in the private experience the guest has while still being aware of everything in the real world. Even with the headset on, they will still be able to interact with everything around them. This makes it interesting to create social or multiplayer games using Augmented Reality. Imagine playing a multiplayer game with friends, while each of you can see “personalized” information on your own glasses and at the same time, seeing a shared part of the game. Players can even do sneaky things right in front of others’ faces. As game designers, we will need to think about the information that can or cannot be seen by players in order to create different feelings/moments so that the experience can be unique every time.

A unique aspect of the Hololens compared to the Meta 1 is that it can detect space and objects around you. This is a feature that helps improve the experience in Augmented Reality. Based on this feature, characters in the virtual world can “play” with you in your own room by being spatially aware of your environment. You can hide behind an object like a sofa to dodge attacks or “change” the world that you are in by mounting defenses on your dining table. Artificial characters can move around the space, which allows every player to get a unique gameplay experience since their real world environment is unique. Take Microsoft’s RoboRaid game for example, it allows you to blast virtual holes in your walls. Young Conker, Microsoft’s first platformer game, tailors each of the levels you play to your real world.

The common comparison or confusion with Augmented Reality is Virtual Reality. There are several key differences between both technologies.

In Virtual Reality, you are brought into a totally different world where everything is synthetic. You can be transported to a fantasy world or any environment possible. On the other hand, for Augmented Reality, you are placed in a familiar environment while interacting with it in different ways. This results in very different experiences when designing for both technologies. Virtual Reality tends to be more successful in creating fantasy like experiences compared to Augmented Reality. Hence, the inverse should also be true, although not yet proven, Augmented Reality should be more successful in creating experiences that model reality. Due to the nature of the technology, Virtual Reality experiences are primarily isolated whereas Augmented Reality may feel more social.

In terms of presence, they are kind of different. As we can see, VR has already done a quite good job trying to really bring you into the virtual world by letting you see your own “hands” in VR or even do natural hands and feet movements (room-scale VR). The AR technology might still have some way to go before it can really make you feel completely convinced.

I’ve tried and been working with the HoloLens for days. It actually works fairly well and the holograms are uncannily real. However, there are still limitations like the field of view is still somewhat narrow and the device cannot detect the position of the hand yet (I guess they have that but just haven’t released it). It would need to cover much more of the world than HoloLens allows right now. I am looking forward to see what Magic Leap can achieve to bring the technology a huge leap forward.

Magic Leap’s latest video

 

 

M. C. Escher would have never thought that his geometry artwork could become digital games 60 years later.

Escher was a famous Dutch graphic artist in the 20^th century, who created impossible realities in his drawings, including impossible stairs/objects and multiple visual and gravitational perspectives. Escher used the tribar or Penrose triangle repeatedly in his lithographs, one of which is a building that appears to function as a perpetual motion machine called Waterfall. His art pieces are the combination of art and mathematics, which makes them indeed impressing.

I was once really into this type of visual illusion and did a bit research myself. I never knew that impossible geometries came be put into games and let me really play with it.

I found out this game called “hocus.” recently, which is similar to the architecture design in Monument Valley. Most of you have probably already played or heard of Monument Valley, in which the developers recreated Escher’s most famous artwork, Waterfall, to honor him. It is an outstanding puzzle game with pretty art and a subtle “lost” story.

“hocus” on the other hand looks simple and clean. It’s just a puzzle game, which doesn’t have a story. It uses the simplest elements such as black, gray, red, cube, etc., and the only goal in this game is to control this little red cube in the game and find the way to the destination. Because the cube has 6 faces, so there are 6 possible directions. The directions that player can move will be shown around the black circle as a hint of possible movements.

All the simple elements doesn’t really make the game simple For being a one-man development and programming “team,” this visual illusion puzzle game is probably the best one in the genre at showcasing to casual gamers what makes these kinds of puzzles so much fun and satisfying to solve. It’s definitely the most quick and accessible of it’s kind, making it great at filling in any brief downtime during the day, but more than that, it’s the kind of game that makes you glad to find yourself suddenly stuck in line or in the waiting room because you’ll have an excuse to solve that next puzzle you’re on.

In all these games, the unique aspect is that the path can be altered by rotating the shapes and viewing them from a different perspective. For instance, if a gap or obstacle is obscured, the character will behave as if the path continues behind the object which currently obscures the gap or obstacle from view. Similarly, if discontinuous shapes or parts of the same shape appear, from the chosen camera angle, to form a continuous path, the character will traverse from one to the other.

As for a game designer, designing these puzzles/levels is quite fun because you are creating something very different and impressive. In addition, one thing that we should think about is the changing of perspectives. Even when designing other games not using impossible constructions, visual illusion can still be used to create interesting game environments. This would also be interesting if putting into Virtual Reality since the player is in that environment so that he can explore and rotate himself. It would be amazing seeing a different shape or even a different world in a different perspective.

Gone Home is one of the my favorite first person storytelling and exploration games. In a way, it is not really a “game”. It is more like an adventure for players to explore and experience. The player played as Katie Greenbriar, who left home to Europe a year ago and comes back to her new family house expecting to see her mom, dad and sister Samantha, but no one is at home. Where is everyone? This question is the motivation for the player, aka Katie, to find out the story happened in this house. Throughout the exploration process, the player will study various objects scattered around this big fine house, and dig out the family story.

The game starts from the player, Katie, standing in front of a house, which is also a tutorial phase to teach the player how to pick up things and explore, since the player needs to find the key under a toy in a cabinet in order to open the door. The note on the door also suddenly draws player’s attention and give him the motivation to get into the house and find out the story.

Gone Home gives players a lot of feeling of freedom to explore things by themselves. Exploring the details in this game is not just some tedious repetitive work. If players explore the house carefully, they will discover hidden stories and some humor in there. Not like some other games where players could only interact with things that is “meaningful”, Gone Home provides this kind of experience where as long as the object is not nailed to the floor, players can always pick it up and examine it. This simulates the situation in real life where you would grab and look into everything that looks suspicious. In the process of doing this, players will recall their own memories to imagine the story behind the object , in which way they can learn more about the family members’ lives, personalities and relationship between them(based on their own life experiences). For example, many books that have father’s name on it indicate that Katie’s dad is a writer; the employee’s ID card and some brochures show that mom is working in a forest; punk music tapes, posters and some notes Sam left tell us that she is a rebellious teenage girl.

As the player continues exploring in Greenbriar’s house, he will find a lot of phone records, a piece of paper and family members’ diary. Most of the clues point to a dark corner in the family. The player will see Katie’s sister Samantha’s growth, complex marital relationship with parents, and finally player will know the reason why she left home. The whole process makes a good interest curve and a rich story to show the tension. Despite that some of the scenes are scary while finding the secrets in the house, there are many surprises. For example, in exploring the study, player will find a box filled with dad’s second unsuccessful novel. But if player takes away the top layer of the novels, he can find his magazine collection under there. Some similar circumstances that appear many times throughout the game will make player feel warm and nostalgic.

One of the high points on the interest curve happens in Sam’s bathroom. When the player walk into the bathroom and see the bathtub filled with water with some red drops on the edge, player will immediately think of the worst situation. The player will then notice the spray bottle down beside bathtub, and find out that it is nothing but some red hair dye. Although the player might feel that he has been fooled, he will still feel happy that Sam is alright. At the main time, Sam’s clever interpretation of narration changes player’s emotion from fear into relief.

Besides that, the moment when player opens the locker in Sam’s room, there shows Lonnie’s photo, from which player will know that the relationship between Sam and Lonnie is not just friends. This is a twist point in the story.

 

The highest point on the interest curve, I assume, is when Lonnie decided to leave Sam. Player would really feel the heart-broken emotion of Sam, which happens right after the player enters the attic. Player will find out about Sam and Lonnie’s happy memories in the attic while listening to Sam’s journal at the same time says Lonnie is leaving.

However, after that, Lonnie suddenly realized how important Sam is to her. So she told Sam to pack up and leave home to go find her. It is also the moment when player realizes the true reason why Sam is not at home.

 

Apart from the story, discovering the hidden places are some other exciting moments in the game. Player can only see the place in the map after they discover that place. For example, there are several secret panels on the wall that players can open and find hidden objects in it. Discovering the way to the basement is a scary but exciting part where there are many spider webs in a dark and narrow environment, however, player will also be rewarded with a lot of clues once he gets into the basement.

Although Gone Home is a story exploration game, the flow of this game is also well designed. In the beginning, the player is allowed to do things such as picking things up, examining photos, etc., in which stage player will get familiar with the environment and family members so that player establishes some kind of relationship with “their parents and sister”. In the second stage, players will have to solve some puzzles, such as lockers that need combinations to open, so that player needs to pay more attention in order to find these specific clues to reveal more story. After that stage, the player is going to discover some secret locations such as panels, basement, attic, the secret room behind the foyer, etc. Players have to find keys for some of these places.

One of the most compelling things about Gone Home is the sense of immersion. Not only because this is a first person perspective game that you share the same vision and feelings as the protagonist Katie while exploring, since everything in this new house is alien to both the player and Katie, but also the entire house settings give player the familiar feeling back to 1990s that many players had some similar experiences. In addition, the music and sound effects in this game help a lot to build up the atmosphere and emotion. Gone Home uses a gloomy background music that heightens the atmosphere. There are many Riot Grrrl band’s hit songs. Player will find many cassettes that can be put into the recorder to play, which is a great and very natural interaction for players to play with. The punk music creates another kind of emotion to the atmosphere. It seems weird to put these two types of totally different music style together, however, they match surprisingly well that not only embodies the adolescent rebellion of this teenager Sam, but also provides a sense of mystery.

The best part of Gone Home is finding clues such as notes and letters, and piece them together into a complete picture. There are some certain objects in the game that will trigger different pieces of Sam’s journal, which is the main story line in this game. The player will have a strong sense of accomplishment every time they discover something useful.

The layout design and art are very nice and detailed in this game. Rotate a soup can, the full trademark can be seen; look at the tape storage box, player would notice the typical taste of a 1990s family; open the closet and there will be full of tapes for games and videos. The color tone in Gone Home is slightly yellow, which makes this house much more lifelike.

Looking back to the start scene, the note on the door matches perfectly with what player finds out in the end. It says, “It is impossible to be there to see you”, “We’ll see each other again someday”. Furthermore, the ending of this game would make player reflect on his own life, and think about every crazy decision he has made or maybe wishes that he would have done something crazy to make his life different other than just sitting there and complaining about life.

Whatever platform we are using to play games, console, mobile or computer, their engineers all keep trying their best to provide players with faster operational speed and better image quality. However, in this case, will you still be attracted by a text based adventure game without any pictures? Or, let me ask this way, can a game still be good without any image?

Is it a sign of regression?

“Lifeline is a playable, branching story of survival against all odds. You will help Taylor make life or death decisions, and face the consequences together.”

These three messages on the picture above are what you will receive when you open this game called Lifeline on your phone for the first time. It seems like an urgent call from an astronaut. It makes you feel like you are really holding a radio transceiver and all of a sudden you receive a signal that comes from outer space.

The mechanic of the game is really simple that the player interacts with Taylor “through the radio” by receiving messages from him and choosing the words to reply. There is no image in this game, only texts, and because of that, the player will have to learn about the story and current situation according to Taylor’s messages. In addition to just talk and chat with him, the player needs to provide his suggestions to help Taylor get through his ordeal in a cold and lonely planet.

A text-based adventure game is like an interactive story where the player reveals the story through the conversations and influence or change the story by making choices at several key moments. Some of you might know about gal game as one of the famous types of TAVG where the drawings of characters and texts of conversations are well-combined. It seems like Lifeline is a sign of historical regression that goes back even before gal games, but it’s actually not. It simulates the form of telecommunication which doesn’t have any pictures originally. It also makes good use of the features of mobile devices that the player will have to wait for Taylor to reply other than constantly playing the game and will get a notification when Taylor replies as if it’s all real.

       

Notification on iPhone &  Apple Watch

Atmosphere + Imagination

To create an immersive experience is to create the right atmosphere for players which includes both right context and right emotions.

Unlike other TAVGs before where texts were just used as a form of presenting contents like conversations or narratives which were disconnected with the game itself, the text messages and mechanic in Lifeline are so natural to the player because he has been trained to use his phone to send and receive messages for a long time. In other words, the feeling of immersion is weak in previous TAVGs, but in Lifeline, as one of the TAVGs, is truly engaging.

In order to evoke emotions from the player, the main character is an essential key. Taylor is talkative, sometimes annoying but also funny and optimistic. He is a vivid and distinctive character with whom the player would be emotionally connected. He is alive in those texts and becomes a friend of player’s. Thus, whenever he is in need, the player would be willing to help him solve the puzzle or give suggestions, and whenever he is in danger, the player would feel anxious and upset.

Sometimes less is more. Since there are only texts in Lifeline, it leaves plenty of room for players to imagine things themselves, such as scenes, objects, Taylor’s facial expressions, his movements and so on. In other games with images, players’ imagination also plays an essential role in the whole experience, which is the reason why all games are not total realistic. Like what Scott McCloud said, people tend to put emotions and feelings into objects and mask themselves into cartoon characters because they are not fully detailed, which leaves room for imagination.

In this case, no matter what elements or technologies a designer is using to create a game, as long as he could design a right atmosphere by using those elements and technologies properly and leave room for players to imagine, he would be able to create an immersive experience and in the meantime, let players create their own stories.

Game designers are always trying to figure out this question: how to control players throughout the game while giving them the feeling of freedom at the same time.

I recently played “Her Story”, the director of which also directed “Silent Hill: Origins” and “Silent Hill: Shattered Memories”. This is an interactive movie game where players search and sort through a database of video clips from fictional police interviews, and use the clips to find out the true story and solve the case of a missing man. Players are presented with an old desktop, which contains several files and programs. Among the programs are instructional text files, which explain the game’s mechanics. One of the programs which automatically opens on the desktop is the “L.O.G.I.C. Database”, which allows players to search and sort video clips within the database. However, players can only see at most 7 video clips every time they search. This means that players might be stuck at some point if they couldn’t get enough clues. There are several ways of indirect control in “Her Story” that make it possible for players to explore and ensure progression of the game.

KEYWORDS

We are very familiar with searching systems by inputting keywords since everyone uses Google or search engines all the time. So when players see some words come up quite frequently, they might perform a search of suspicious words. Besides, there are some keywords that are not only clues but also metaphors, such as “mirror”.

In addition to the keywords that come from what she said, players can find clues from plots or movie clips as well, which are also good ways of providing the keywords. For example, in “Her Story”, the main character will drink something at the beginning of every investigation. From this observation, players could search “coffee”, “tea”, “cup”, or “sugar”, etc., to get more information. After players have watched a certain amount of videos, they will find out that there are subtle differences in hairstyles, clothes, tattoo and even facial expressions in different clips. All these implies that they are clues and players can search keywords according to this. Once players detect clues, they will be curious and dig out the story on their own.

HOW PLAYERS WANT TO PLAY

The clues will not only make the progress and the story advance, but also give players freedom to play in their own ways. It is not hard to notice that there is a timer on each video clip so that some people might sort the video clips based on time sequence.

Moreover, in this rather limited interface, there are still ways to help players sort video clips by “User Tags” and “User Session”. This let players investigate whatever they want and also   find out the truth.

SHIFTING GOALS

In the beginning of “Her Story”, it only teaches you how to use the system but never tells you what the goal is. What the players can see is a keyword, “MURDER”, already in the search bar. This certainly arouses players’ curiosity to solve this murder case. But as players advance through the game, they will feel that the murder case may not be as important anymore, and be more curious to find out who she is. Thus the goal shifts from “finding out who killed the man” to “looking into the story of the girl”.

Other than this, “Her Story” is a very immersive game where would feel like they are actually playing the role of a detective. However, in the end, they will be given another identity (I won’t spoil). This identity intensifies the emotion that the player had before and gives the player a strong motivation of exploring the whole story.

While I was playing the game, I shifted my goal three times. At first, I wanted to know the truth of the murder case, which means I needed to know who she is. Due to the mechanism of this game, I found out the truth of the murder quickly because I searched some right keywords that led to it. Along the way, I gathered some information of the girl and gradually found out who she is. But after figuring out her identity, I became more curious about her story so I kept looking for more video clips that would constitute the whole story. These three goals are actually connected and are also progressive layers of the story.

 

“Her Story” doesn’t have any tutorial or direct control in the game. The game uses frequent keywords to control players indirectly and some other hints to provide various ways of playing this game. Allowing players to pick and choose among a diversity of goals will keep players engaged in the game constantly even if they are repeating the same simple mechanic.

“The uncanny valley is a hypothesis in the field of aesthetics which holds that when features look and move almost, but not exactly, like natural beings, it causes a response of revulsion among some observers. The “valley” refers to the dip in a graph of the comfort level of beings as subjects move toward a healthy, natural likeness described in a function of a subject’s aesthetic acceptability.”

When talking about uncanny valley and game, people tend to associate the theory with 3D models or animations, from which we can tell that graphic means a lot to players and has a strong impact on players’ feelings. But here, I want to talk a bit about the uncanny valley in gameplay.

People who create games are actually creating pre-made models of the reality. Games are more or less simulations of reality modified by game creators. The challenges or sometimes the gameplay in a game are to some extent reflections of the problems in reality. The reason why people like to play games is that it can not only give people the satisfaction of human instinct to discover and solve problems, but it also provides people with a place to do something they want to do that might not be allowed or able to do in real life.

In the process of playing a game, players would always be able to learn some skills concerning solving a certain type of problems, which is considered the training function of games.

When the relation between a game and reality is weak, the training or the challenge might just be player’s reaction or the coordination between eyes and hands. Players would still be having simple and original fun experience like when they are playing Tetris or Bejeweled.

Once a certain background story is added, everything starts to become more interesting, even if the mechanics and training purpose are the same. PacMan made a big step towards simulating the reality because the AI for different ghosts is different which makes them look like that they have a plan to catch you.

With the advance in technologies nowadays, there are much more that we can achieve. Thus people are becoming much more dedicated to the simulation of reality. However, problems appear, which never occurred before. People start to step into the uncanny valley of gameplay, which is, in most of the cases, the gap between the AI design for the enemy and the game experience.

Especially after watching the (CGI) trailers, and having some knowledge as well as the expectation of what might happen, the pattern for AI or the feeling while playing the game might pull you out of the game. It means that the feeling of you knowing how to solve the puzzles because this is a typical way to solve that kind of puzzle in a game. This kind of feeling can easily break the immersion and expose, so to speak, the “code”, in front of the player.

We might think that the simulation in games with ancient background or fantasy background is not as good or as real as games with the background that we are familiar with, however, players won’t feel the same. This is because people can use their imagination to add to the story just in order to convince themselves. In other words, the level of the simulation and the graphics are similar. People would easily realize that “this is a virtual world”.

However, this kind of game is also been made into more and more realistic style, such as Witcher, Dark Souls, etc.. It will be facing the same problem, the uncanny valley in gameplay. There are some ways that I can think of to solve this problem, in the long term. First, getting better in simulating the reality, which is quite possible since there are games like shooting or flying simulation for real training purpose in military, or even using some physical simulator/props, especially together with VR experience. Second, change the art style to match with the immature AI system, for instance, lower the realistic feeling in order or find a unique style of its own. The second solution is sometimes useful with the game still looking good/pretty no matter whether it is realistic. 

Portal, however, did a great job because the game developer made the opponent a real AI so that people wouldn’t feel uncomfortable; on the contrary, people know that it is not a real person and think GlaDOS is a really funny character.