Corporate Anthropology

A Standing Ovation

2005-03-26T04:57:00.000-08:00

Allen was a manager of a software engineering team, and had discovered an ethnographic methodology. He recuited a core team from the ranks of his own engineers, and from other teams, including writers, testers, and support specialists. They wanted to know how to improve user account management functions in the administrative console for NetWare 4. They conducted about 16 interviews in the contiguous U.S. states.

It was time to consolidate the customer models created during the interpretation sessions. The team created an "affinity diagram" from the 1,000 notes taken from interpretation. From this diagram, they detected a number of key patterns. They consolidated the sequence, context, flow, and physical models. All of these consolidated models were helfpul in providing new insights into their users, but the sequence models stood out from the rest. It was clear that network administrators had to perform unnatural acts to create and manage user accounts.

Administrators had to skip around to different functional areas in the administrative console to create the account, create a directory, grant rights, and set account restrictions. The work experience had been fragmented by the wrong functional design.

Based on the information collected and consolidated, over the ensuing months the team redesigned the user account management functions to be more cohesive and simple. Development went quickly, and with few arguments, because the engineers shared a common, rich understanding of the customer experience.

They finished their work just before Novell's annual Brainshare conference came around. Allen was scheduled to present Novell customers and partners with an overview and demo of the latest enhancements to the administrative console, and his session was to follow one of the plenary morning sessions in the University of Utah events center.

Thousands of people attend the morning sessions at Brainshare, and several hundred stayed in their seats to see what the latest changes were to the administrative console. Allen was nervous. He walked through his presentation, and gave a demo of the user account management enhancements at the end of his presentation.

Attendees had come to his session from around the U.S. and the world. As they saw the changes, they stood up and gave him a standing ovation. The team's ideas, grounded in thorough research and analysis, had been validated.

Allen knew he was onto a powerful methodology.

Expediting Contextual Inquiry

2005-02-08T18:02:00.000-08:00

As I mentioned in the preceeding posts, Allen's engineering team had learned about CI from the Word Perfect people and had decided to try it out. He recruited a handful of people to get the training and become instant ethnographers. Dennis, from Word Perfect, had developed a tight relationship with Karen, and understood the process well enough to train the team.

One challenge was to get the research done in time. Karen Holzblatt had just started her consulting business, and was still working out the details of her methodology. The Word Perfect CI projects took about a year each to complete.

Being a pragmatist, Allen started thinking of ways to tighten the process and bring the project in on time. He had a release coming up, and he needed answers fast. He pushed the team to get their customer visits scheduled - and to make each one count. Individual team members were responsible for setting up and conducting two or three visits each. He got the whole project done in three months.

Another challenge was to bring the team's skill levels up to snuff. Allen dealt with this by pushing them to do the work and learn as they went.

Being a skunk works project, the team only had money for trips within the contiguous 48 U.S. states. They decided to spread the visits out between the East and West Coasts the Midwest and the Mountain West. They also tried to get companies in different industries such as manufacturing, finance, services and so on. Each team member would have to make some calls and get his or her visits set up.

For each visit the designated person went to the customer site at the appointed time, and watched the customer perform the work related to the focus (how people manage user accounts). Each team member spent about 4 hours watching a network administrator use Nwadmin to work with user accounts, set access control rights to files and so on.

After each visit the interviewer flew back to the office and held an "Interpretation Session" while the information was still fresh in mind. During this session the whole team participated, with different team members modeling the different aspects of the information the interviewer rolled out to them. Each model showed a different facet of the customer experience: social interaction, information flow, work steps, physical layout, and artifacts. In addition to creating powerful models of the customer experience, the Interp also gave the whole team to live through each interview vicariously. The information from all 16 interviews was in everybody's head.

Team members were surprised by some of their findings from their visits. They held off on making design decisions; they would need all their concentration to capture the right information at the visit, and then holding their Interp sessions. In spite of this voluntary suspension of design ideas, they started to see some use patterns. One of the common work sequences was as follows:

- A manager hires a new worker
- The manager calls the Information Systems Help Desk
- The help desk logs the call in their trouble ticket system. The email had the user's name, boss department and so on
- Help desk sends an email to the administrator requesting to create an account for the user
- The admin fishes the email out of his in-box, and starts Nwadmin
- The admin uses the Nwadmin search function to see if the user is already there
- Admin fiddles with Nwadmin to find the right file server and volume
- Admin creates a home directory for the user
- Admin fiddles with the directory portion of Nwadmin to create the user account
- Admin performs some additional functions to assign rights

In this example, it became clear to the team that it was possible to create a user account using Nwadmin, but it wasn't smooth. In fact, it was just painful! Another interesting thing was that no sales rep, systems engineer or even a customer (at a trade show or usability test) had ever expressed that this was how they created user accounts. They probably assumed that it was obvious, and didn't bother talking about it.

After they finished their visits and Interpretation Sessions, the team was ready for the next big step - consolidation. Now they had to take the models from each Interp and pull them all together into a composite. This would help them understand what feature priorities and design ideas they would use to drive the next release of the tool.

Engineering Team Discovers Contextual Inquiry

2005-02-05T06:21:00.000-08:00

As I mentioned in the previous blog, Allen took over an engineering team responsible for the Windows administrative console for NetWare 4. The console focused on allowing network administrators manage user accounts, file systems, servers and any other services managed in Novell's new directory services. The enhancement requests were rolling in from everywhere. It was hard to tell what was a good idea and what was bad, what was most urgent, and what could wait.

Trent, a designer on the team, had made some contacts with the Word Perfect people (Novell had merged with Word Perfect some months earler). There was a pocket of people at Word Perfect who, having dealt with the endless list of possible enhancements, had met up with one Karen Holzblatt, formerly of Digital, who together with Hugh Beyer, had developed a methodology for understanding user needs. Karen was facilitating a research project for a team at Word Perfect.

Trent and a few others on the Nwadmin team wanted to try out Holzblatt's methodology on network administrators and see what insights they would gain about better feature design and prioritization. Dennis, one of the Word Perfect people, had immersed himself in the Holzblatt-Beyer methodology and would help the small team of engineers apply the methods to their research.

Allen appointed a few of his own lead engineers to participate on the team, and recruited some form outside his team from documentation, testing, and support.

Their first challenge was to set a focus: what it was they wanted the research to tell them. They decided the focus statement would be: "how people manage user accounts." They planned on "shadowing" network administrators as they created, managed, and deleted user accounts. The goal wasn't to sit and ask the subjects a series of questions, but rather to watch administrators as they dealt with creating and managing user accounts.

The team sequestered a "war room," a small conference room near the engineers' offices. This is where they would meet and debrief (in Holzblatt's parlance, it's an "interpretation session") after each interview conducted.

The group started setting up interviews ...

Discovering Contextual Inquiry: requirements engineering

2005-02-01T16:39:00.000-08:00

I'd like to post a series of blogs about how I discovered ethnographic research, and why I'm such a fan.

I started out as a programmer for I.B.M. and worked on PC connectivity tools for the System/36 and then the AS/400. Within a few years I moved from programming to dealing with customers and planning features. I kept hearing about how Novell, an upstart in Utah, had far better file service performance numbers than an AS/400 with a PC attached. I could see a massive wave of innovation coming, and with its conservative, top-down approach to development, I.B.M. was going to watch this wave slide by. I hopped over to Novell.

Fast Forward a few years. I had convinced Novell's management on the idea of a single, extensible, Windows-based utility for managing all NetWare functions. I had built a great team of engineers, and we were in our third release of the Nwadmin console. We were quite pleased with ourselves on the Nwadmin team: we were the first to use C++ and Object-Oriented Programming; we were developing on Windows 3.0 (DOS was still big then); and we were the primary tool for managing NetWare 4. Being engineers, we thought quite highly of ourselves.

As is the case in any software development, we had a long list of enhancements to make. Suggestions came from everywhere: testing, sales, customers, support, documentation, other engineering teams, and of course, our own favorite notions. We didn't know how to prioritize, and our marketing team was little help to us. (of course, we knew what we thought was "cool"). Marketing helped us set up visits and usability tests with customers, and these were helpful, of course, but they didn't help us get the big picture.

After a major release, I held an one-day off-site meeting with my team to give them a chance to blow off some steam and tell me what we could all do better. I learned a lot from that meeting, but the most important point to make here is that it was clear to me that I had hadn' t been careful enough defining requirements, and this had caused us to make costly mistakes.

I became obsessed with the problem of requirements engineering. I gave up my engineering team and moved to a newly-formed Product Management to push the ideas further. Allen, one of my best engineers, took over managing the team for me. This is where the story begins ...

Bad Business Bets in Las Vegas

2005-01-26T16:13:00.000-08:00

The January 21 Wall Street Journal has a front-page article about some bad business bets in Las Vegas casinos (Why High Rollers Shun Private Digs In Las Vegas Casinos, by Christina Binkley). According to the story, the state legislature loosened some 50-year old laws prohibiting private betting. By law, any casino room had to be open to all comers. With the new rules, the casinos placed their own bets; they built opulent private rooms for wealthy Chinese high rollers. For example, Mandalay Bay spent $3 million on new private salons. Caesar's Palace rolled out new Roman-style private rooms.

There was one flaw in the whole theory, however: gamers had to fax names and background information to regulators, to prove to jittery bureaucrats they were on the up-and-up. This scared off the Chinese: they didn't want to disclose this information and end up having their own government know what they were up to.

The $3m room has had one visitor so far -- $3m down the tubes. The Caesar's Palace rooms have had a couple of visitors. This bet was a huge bust, all because of a small detail.

How can businesses avoid these sorts of disasters? One benefit of ethnographic research is that you get a clearer picture of your prospect, and know what keeps them from signing on the dotted line. The best model for identifying these sorts of sales barriers is the Context Model, which shows relationships between people, bosses and governments. If this relationship had been modeled, business decision makers could have seen clearly the barrier:

Nevada regulator -> gamer: "you must register to play"
Chinese governement -> gamer: "the peace-loving people of China frown on gamblers"
Gamer ->casino: "I won't play if I have to register"

Seeing sales barriers is what a context model can do. Better luck next time.

The Decline of Wintel

2005-01-25T17:32:00.000-08:00

In the December Fortune Magazine, Brent Schlender asks, "Is Wintel Out of Gas?" Wintel, of course, is the Windows-Intel hegemony? Why is it out of gas? Because the PC market is out of gas.

People use PCs and Windows at work because they have to -- their livelihood depends on it. Businesses spend thousands of dollars a year just keeping the surly, uncooperative systems running. (a large percentage of calls to the help desk are just to recover deleted files.) Windows is a hothouse orchid, only suited to be coddled by IT teams.

Consumers won't put up with Windows. My boys have an Xbox and a GameCube in their bedroom. The GameCube stays busy, and the Xbox is gathering dust. We loved Amped, the snowboarding game, but it froze all the time. They have NBA for both systems, and prefer to play it on the GameCube. It's tougher, simpler. No freezing, crashing or dying, like the Xbox. My brother, the first to get any gadget, bought Windows XP Media Center as soon as it came out. He dumped it. Same problem. Windows isn't consumer-strength.

What about Intel? They've lost the leadership. Itanium is a niche processor. The latest Pentiums and Xeons support 64-bit software using the same instruction set AMD developed for the Opteron and Athlon processors (code-named Sledgehammer). It's not just the instruction set, though. AMD's processors also have on-chip memory and I/O controllers, for better overall performance. While Intel build an incompatible 64-bit chip, and contented themselves with clock speed sleights-of-hand, AMD looked at the bigger picture. In a month or so AMD will ship two processor cores on each chip.

Meanwhile, the PowerPC charges ahead, running on a variety of different systems, and dual cores have been announced also.

The open source movement isn't just about Linux. Take BSD, retrofit with MacOS, run it on a PowerPC, and you have a Macintosh with OS-X. Add the human factors team at Apple, and you have something consumers really want. The disruption to Wintel will show up in the new markets: phones, music players, video players home, game systems, entertainment systems and more.

Wintel will continue to dominate the PC market. Word Perfect still dominates the DOS word processing market, too. It's all the other stuff on www.gizmodo.com that won't be dominated by the Wintel hegemony.

The Creative Power of Metaphors

2005-01-18T14:30:00.000-08:00

Even after research team members have done their work well -- learned well how to research, observed the right people doing the right things, modeled the information, consolidated the models and created a tight cartoon summary -- they still have the greatest hurdle to climb: turning their understanding into compelling products or services. The next logical step is to use metaphors.

This is the point where literal-minded team members can lose interest, so it's important to warn everyone that they're going to try a new kind of approach. I think metaphors work so well because they help people use their diverse intuitions to deal with the problem; they handle new problems by treating them as old ones they already understand.

It usually takes about 90 minutes to come up with the right metaphor. For my last project, the library team decided to compare their patrons to golfers. We explored the metaphor for its strengths and weaknesses. Golfers are like patrons because: even after lots of practice, they can still slice or hook the ball; everyone on the course is a self-appointed "expert," and much of the advice is contradictory; time on the driving range helps; golfing can be a sporadic thing for some, and a frequent thing for the fanatics. The comparisons went on and on.

Once you have the metaphor, you decide how you would solve the toughest problems. In the case of golfing, how could you make everyone a proficient golfer? We decided that in a world of golfers, every neighborhood would have a small driving range; golf courses would be loaded with knowledgeable, helpful experts; people would know the difference between a real golfing coach and a straw coach.

After exploring the metaphor and the fantasy of a perfect world in that metaphor, it's time to go back to solving the real problem. In the case of the library, we decided to use the model of caddies and pros. We had the stimulation we needed to create an improved world in the library, drawing on our golf metaphor.

Cartoons to Summarize

2005-01-07T04:54:00.000-08:00

A few weeks ago I had dinner with an anthropologist, and told him about the the latest research project I facilitated for a genealogy library. I described the process to him, and mentioned that I always have my teams summarize their findings with cartoons. The idea of summarizing with cartoons struck a chord with him. Communicating research findings to a large audience is a challenge -- after all, they weren't there for the work in the trenches. It's always a challenge to create a tight summary of what the team has discovered.

(I use a process based on Holzblatt-Beyer Contextual Inquiry, which helps members of product and service teams become anthropologists, observing their prospects in their natural work environment, analyzing what they see, and devloping ideas for compelling new products, features and services. Using this process teams produce quite a bit of data showing social, work and environmental aspects of the customer experience.)

I have learned that even though CI's consolidated models are rich sources for market analysis, it's important to create a simpler summary for executives, peers and anyone who might have an interest in the findings. The massive amount of customer details contained in the consolidated models are gold for product and service designers, providing needed detail to inform their frequent design decisions; however, for anyone else, it's just too much.

The cartoons don't need to be product quality. Stick figures are fine. However, I have discovered that every team has some people who are good at drawing. Cartoons provide a number of benefits over raw data: they cause the team to build a narrative, which improves their own understanding and lets them tell a story based on a single premise; it drives the team to compress the gobs of data they collected and focus on what's most important; the humor, pictures and team interaction involved in creating cartoons exploits previously-untapped creative portions of people's minds.

2005-01-04T09:47:00.000-08:00

Near my home in Ivins, Utah.

Listening is not the same as understanding

2005-01-04T08:39:00.000-08:00

Last October John Porcaro, a Microsoft employee, posted a blog titled, "We're Listening ..." and went on to say that MS is doing more listening to customers than ever before. As an example, John said colleagues talk more about the impact to the customer, participate in online communities, and still do focus groups and surveys.

What's wrong with listening to customers? Lots. First, even when you listen, it doesn't mean you understand. Second, you miss what the customer didn't tell you. Third, the rest of your team now has to get the customer story, second hand, from you.

Instead of just listening to customers, you need to observe your customers in their work environment. Your whole team needs to understand the customer's job and challenges, and make sensible decisions about the most important features and programs to offer based on that understanding.

Ethnographic research is a way of immersing a team in their customers' world. Last year I led a team from a genealogy library through the process of observing their patrons as they used resources in the facility. Most of the team members were reference consultants, and provided guidance daily to patrons. In spite of having listened to their customers for years, they were suprised by our findings. Instead of simply listening, we followed 20 different patrons around the library as they tried to set objectives, find books and microfilms, use computers and analyzed their materials. These consultants were surprised by how often patrons didn't find useful information, and wasted hours trying to navigate through the library's resources. We identified several simple changes that would make the library a much more usable place.

In his book, The Inmates are Running the Asylum, Alan Cooper, a human factors consultant, points out that business computer users are a captive audience: using the products their employers have purchased is a condition of employement. Consumers, on the other hand, have more choice. To bridge to consumers, Microsoft and others will have to rely more on observation and human factors design; listening isn't enough.