Last week in the first part of our storytelling feature, we talked with big name developers Gearbox and Telltale about the importance of getting feedback for the stories in their games. For a slightly different perspective in part two, we sought out some indie developers to see what it was like incorporating feedback in smaller projects. Pinkerton Road founder Jane Jensen and Dave Gilbert of Wadjet Eye Games were kind enough to talk about their experiences in story crafting and how players help shape their games.
Jensen, as you may know, is famous for having worked on the Gabriel Knight series at Sierra Online, and she reflected that things have changed a lot since then. Ultimately, while Jensen sees great value in player feedback, testing can be taken too far, she said.
"We didn't do a lot of player feedback back then - we did beta testing back in the day," Jensen remarked. "We'd do some testing back in the day, arrange two-day play sessions, watch them play and video tape that, in addition to an alpha and a beta test. Bigger companies that I've worked for recently have done extensive player insight with everything from game titles and whether the main character is male or female... I think that's too much sometimes. Taking an element like that and presenting it out of context can be deceptive - if someone actually saw the full game and they liked it, maybe they'd understand why the main character is a female."
"It's always good to have fresh eyes looking at something. Every week there's something new in the game so I do a new call for new testers"
While Gilbert's path to game development has been different, fan feedback has been no less valuable for his indie projects. "After I did Blackwell Convergence, and I put a lot of effort into the art where I paid a studio to do the background, I asked for feedback. When I released the game, it made a bit more money than before, but since I paid more money to have it made my economics were different. I took my question to the fans instead of second-guessing myself. I asked if they liked the graphics and would they have bought otherwise. I started various conversations on the topic and I determined the story was more important than the background graphics."
Gilbert stressed that when getting feedback on a game, it's important to use a fresh set of eyes, rather than relying on the same testers over and over.
"It's always good to have fresh eyes looking at something. Every week there's something new in the game so I do a new call for new testers," he said. "When someone has been testing something from the beginning, they can't approach it with the same level of freshness. Most of them aren't professionals, so I get new people testing all the time."
One of the great things about feedback is that it sometimes leads to completely unexpected results, which can be quite useful, Gilbert said.
"For first two Blackwell games, there's a notebook and one thing you can do is use it like an inventory item and combine clues together to get new conclusions. It works for a mystery game, but players found it annoying. They didn't think it was intuitive, since they could come to a conclusion and the character wouldn't do that until they had the right clues. I thought people would like when I simply removed it in the third game but they didn't! They liked using the clues, just not the way I thought. So in the fourth game, the conclusions from the clues weren't something you could come up with on your own."
Jensen's new project is a Kickstarter funded project called Moebius, and for Jensen, getting early feedback on the project was crucial from the start - in fact, the decision on which particular game she'd work on came through a vote from backers.
"We had the idea for a fun thing to do for our Kickstater backers, to vote on the next title," Jensen noted. "We kind of stacked the deck in that I liked them all and I kind of had an idea for one that would never fly. It was interesting for me because I had a game called Gray Matter in 2010 and I was curious to see how the Gabriel Knight fans felt about that whether they wanted that or something totally new. But it's not like we're going to let users vote on everything."
As important as it is to listen to fans, Jensen asserted that the majority of design decisions will rest with her and the other members of Pinkerton Road.
"It's not like we're going to put the design for every scene or what should happen when you click on his door," she said. "The process is what it's always been for me - I tear my hair out finding a story and puzzles for a few months, and then we run with those ideas from there. But it is valuable to get feedback from people for those classic tier adventure games and we haven't spent a lot of time or money investing in examining gameplay features yet, like should items be a bar or a radial menu."
"We're going to have 'swipe to reveal' hotspots on easy mode and figured that we'd have a hardcore mode where it would be more like classic adventure games. As it turns out, we had people who wanted the 'swipe to reveal' in hardcore mode as well. Getting feedback from Kickstarter is not too different from beta testing, but you can see trends, like if people say they hate a cutscene, at least you're forewarned and maybe you can assuage it by the final product."
Unlike Jensen, Gilbert has been a bit more open about letting players have a certain degree of control in driving story. He got some vital story feedback when it comes to Blackwell Deception. "During beta testing I asked for feedback from gamers in terms of story and flow. The original ending of the game was honestly kinda bad - just a placeholder because I was struggling to come up with something good," he admitted.
"I still have to feel like there has to be a creative vision; I've never been a believer in design by committee"
"I guess the best way to describe it in broad times, is there was this antagonist that had been hinted at all game, and with barely any interaction, he was defeated. I couldn't think of a way to make it interesting, but I decided to code it in there to put it out there and put it in front of testers, because maybe they can think of something better," he described. "There's this ability that's been foreshadowed... and the main bad guy uses the ability in the game. That was the crowbar that was needed to open it up and make the final sequence more threatening and interesting."
Jensen, meanwhile, was very assertive about the necessity of creative control, even with crowd-funded titles. "I still have to feel like there has to be a creative vision; I've never been a believer in design by committee," said Jensen. "So we intend to stay true to that. Like the Mass Effect 3 controversy about the ending and the response to that... there are things I feel passionate about. It could be for business reasons or creative reasons, people get upset at a lot of things. You have to temper player responses at least 60 percent of the time."
Regardless of how fan feedback is incorporated, connecting with the players is of utmost importance in today's industry. It's why Jensen started Pinkerton Road in the first place.
When asked if it was scary and exciting to be simultaneously founding a new studio, going to Kickstarter for funding, trying out new IP, and being more public than she has been, Jensen responded, "All the above! When we decided to do the studio, the large part of wanting to do it is to connect with fans personally. I was working with a casual developer and had to subsume my personal brand but with Pinkerton we're going to do adventure games and I'll be taking a more prominent role. It's scary to make yourself visible since you know you're going to get critiqued and see the whole spectrum of responses. It's intimidating to do, but I'm glad that we did. We have established a core group of players who are interested in this and will help get the buzz and PR going, and Kickstarter helps accelerate that as well. It's been stressful and flattering all at the same time."
In closing, Jensen pointed to what's perhaps one of the best uses of feedback for her new studio: "I think that people are excited that the people from Sierra are back and seeing that and having it thrive is great. What's pleased me is the three concepts that we had people voting on - the subliminal message is that we have three games we could work on. A lot of people wanted to play all three of these games in terms of positive feedback that means the most to me."