Fast Company

Customer Service: EMC Corp.

"When a customer believes in you ... they'll stick with you almost no matter what." -- Mike Ruettgers, EMC Corp.

Recognized For: Customer Service
Home Base: Hopkinton, Massachusetts
Year Founded: 1979

Mike Ruettgers was apologizing again -- and wondering why the hell he had joined EMC Corp. in the first place. It was 1988, soon after Ruettgers had signed on as executive vice president of operations and customer service. The company's products were failing left and right -- and EMC itself was on the verge of bankruptcy. The biggest crisis involved a batch of faulty disk drives that EMC had shipped to customers. Ruettgers had joined the company as a high-tech troubleshooter, an expert in straightening out these sorts of kinks. But this kink threatened to put the company out of business.

Life and work turned into a blur as he crisscrossed the country, meeting with customers in a desperate bid to contain the damage to EMC's reputation. He became a kind of executive punching bag, absorbing as much anger as he could in an hour-long meeting and then moving on to the next one. It was a punishing, humiliating routine. When he wasn't listening to tirades about EMC's products, he was apologizing for the problems associated with those products and promising to make amends.

Now, as he sat in another windowless office in Salt Lake City (or was it Denver?), he felt himself descending into a new and special hell. The man on the other side of the desk, a manufacturing executive about the same age as Ruettgers, had broken down in tears because he was going to lose his job as a result of EMC's failures. His company's business had come to a screeching halt because no one could get to the crucial information stored on EMC's data-storage systems. Big, expensive computers -- and all of the myriad operations that relied on them -- had been sitting idle since the refrigerator-sized EMC machines, where the data was kept, had seized up. It was as if the heart of the company had stopped beating.

When Ruettgers resumed his bleak rounds, he arrived at a bold comeback strategy. To make customers whole again, he insisted that they be given a choice: receive a new EMC storage system, or take one made by archrival IBM -- but paid for by EMC. So many customers opted for IBM that during one quarter in 1989, at the height of the fiasco, most of the storage systems shipped by EMC were actually made by its biggest competitor.

Inside EMC, some people wondered what the hell was going on. But customers recognized EMC's extraordinary commitment to stick by them, and once Ruettgers had installed rigorous quality controls, many of them bought from EMC again. "What that proved to me, to all of us, was that when a customer believes in you, and you go to great lengths to preserve that relationship, they'll stick with you almost no matter what," says Ruettgers, who is now 58. "It opened our eyes to the power of customer service."

EMC still hasn't blinked. Thanks in large part to how he handled that make-or-break crisis, Ruettgers was named CEO in 1992. And the company's reputation for fanatical devotion to service has been at the heart of its torrid growth and prosperity ever since. Back in 1988, when Ruettgers joined EMC, the company had 910 employees, annual sales of $123 million, and a net loss of $7.8 million. Today, EMC has more than 23,000 employees, an annual revenue of $8.9 billion, and a stock-market value of roughly $70 billion. It has earned a spot, along with Cisco Systems, Sun Microsystems, and Oracle, as one of the "four horsemen of the Internet." Lately, like all technology companies, that has meant EMC has been hit hard by the economic downturn. Its stock (about $32 per share when this article went to press) is down from an all-time high of nearly $105 per share. But compare that with the split-adjusted price of 33 cents per share, which is what it traded for when Ruettgers became chief executive.

EMC has done a lot of things right on its way to the top. The company is filled with superb technologists who are prepared to obsolete products in their prime and to churn out new ones with clockwork regularity. A do-or-die sales culture pushes top-line growth even in tough times. But it is impossible to understand EMC's 10-year rise without appreciating its commitment to customer service. The company boasts that its customer-retention rate is an astonishing 99%. When Forrester Research surveyed 50 big companies about their various technology suppliers, "EMC came out looking like God," says Carl Howe, a director of research at Forrester. "It had the best customer-service reviews we have ever seen, in any industry."

Consider how EMC responded in the winter of 1999, when a bank in Wisconsin suddenly lost access to its data-storage facility. In quick succession, the screens in the bank's computer center started flashing "data unavailable" -- a message that might as well have read "closed for business." Within minutes, customer-service engineers at EMC headquarters had retrieved, remotely, the logs of EMC's storage systems at the bank and had started combing through them. But the cause of the problem didn't jump out -- the way it often had.

Four hours later, when Leo Colborne, VP of global technical support, was alerted, the engineers who had designed the machine joined the effort. They re-created the bank's setup in a $1 billion facility that EMC has set aside to do such simulations: same computers, same kinds of networks -- a mirror image of the data center, including a double of the EMC machine that was screwing up. Only then did they zero in on the problem: a glitch in the operating system. They hammered out a patch and zapped it to Wisconsin.

Problem solved -- which, for EMC, meant that it was the beginning of the story, not the end. Colborne knew that he had some fence-mending to do. So that same day, he jumped on a plane to Wisconsin, and a few hours later, he walked into the bank's boardroom, where 20 people sat around a table. Colborne, now 46, has been in this kind of hot seat so many times in the past 15 years that it's become second nature to him. Invariably, his opening line is an apology, on behalf of EMC and from himself personally. As he took the empty chair across from the bank's CEO, he reviewed in his mind one more time the questions that he tries to answer to help restore faith in EMC: What happened? Why did it happen? How do we make sure it doesn't happen again?

"You can't really lose if you go to a customer after something's happened and tell them exactly that," Colborne explains. "It's very simple. But it's not easy. Customers have to go through a healing process before they can move on. The goal is to get the customer to say, 'I'm really not happy about what happened, but I can't thank you enough for the support you provided to get me through it.' That's turning the situation around. If we get to that point, we become sales enablers."

Colborne reached that point with the bank CEO when they began to talk about how other EMC customers had avoided the nasty experience of losing access to their data by using an EMC software package that builds a mirror copy, which is always available. Turning to his team, the CEO told them to set up a meeting that afternoon with EMC sales engineers to figure out how they could add that redundancy to their systems. EMC revenue: $7 million. Says Colborne: "Everyone in the room was looking at me like, 'What's he doing? We're here to defuse the situation, and he's turning it into a sales call!' "

Smart Service: (Don't) Do It Yourself

Consider the challenge: Your customers include some of the biggest outfits in the world: banks, phone companies, automakers, oil giants, Internet-service providers, and other e-commerce heavyweights. They stand to lose millions of dollars if your equipment falters even for a few minutes. Every week, they're changing the already-complex environment of computers and software programs that rely on your equipment for the care and feeding of their crucial information. And those customers are scattered across North and South America, Europe, Asia -- all around the world.

In this situation, delivering great service would seem to hinge on developing world-class problem solvers on the front lines. At EMC, just the opposite is true. The biggest mistake you can make as a customer engineer -- a career-threatening one, in fact -- is to try to solve a problem by yourself. EMC doesn't want heroes in the field; it wants team players who will draw on the company's collective expertise to solve a problem quickly and elegantly.

"The hardest thing about customer service is to get people in the field to have a real sense of urgency," says Ruettgers. "The average guy would die with a problem before calling for help. And some people just don't like to face irate customers, so things can get kind of buried." What happens next, Ruettgers knows all too well: "Senior management doesn't have a sense of what's going on until you get some blowup where a customer calls and says, 'Hey! Do you know I've been down for three days?' "

So one of Ruettgers's first moves at EMC was to centralize the dispatch and management of all service activities. "If you don't do that, and you're providing service in 20 countries," he says, "then there's no chance in the world that you'll be able to understand what's really happening with customers or to intercede fast enough when something isn't going right."

To achieve that kind of speed, EMC's customer-service engineers follow a rigid procedure for escalating a customer's problem. Leo Colborne gets paged about any EMC product anywhere in the world that's been out of commission for four hours. If the problem goes on for six hours, Colborne alerts his boss, Joseph F. Walton, EMC's senior VP of global customer services. If the problem is still unresolved after eight hours, the quaint interval of a traditional business day, Colborne notifies Ruettgers, who recently became the company's executive chairman, and Joseph Tucci, who took over as its CEO.

Colborne hates making that call. He goes to great lengths -- puts in long hours of planning, drills his field-service organization of 4,000 people to an amped-up state of readiness -- in order to maximize the odds that his group can solve a problem before Walton, Tucci, or Ruettgers enters the picture. But five, maybe six times a quarter, Colborne submits to protocol and grudgingly dials the phone to report that EMC's reputation for unfailing reliability and unparalleled customer service is being tested again.

The escalation process is so ingrained at EMC that it has the eerie quality of being self-propelled. At least that's how it seemed to work at Sears, Roebuck and Co. In the summer of 1999, the giant retailer was renovating its data center in Columbus, Ohio one Saturday when a couple of workmen tried to move a refrigerator-sized EMC box. They edged too close to an open floor tile and the machine suddenly tipped at a sickening angle, then crashed through the opening to the subfloor below, pinning a workman's leg underneath.

The guy's leg was freed a few minutes later, badly broken but still intact. The EMC storage unit was another story. An EMC customer-service engineer was paged before anyone at Sears knew about the accident. He rushed to the scene and managed to stand the box back up. The cabinet was badly damaged, but it wasn't until he gingerly removed some of the high-capacity disk drives, one by one, that he realized he had a potential showstopper on his hands. Sears relied on the EMC system to store data on daily sales and inventories for every Sears store in the United States. Without it, the entire merchandising-and-replenishment system was blind -- spelling thousands and thousands of dollars of lost sales opportunities each day it was down.

By some miracle, the disk drives and the all-important data they contained had survived the crash. But the system that tied them all together was broken beyond repair. Jonathan Rand, 53, Sears's merchandise-information director, heard about the accident within an hour. By then, two remarkable things were happening. First, because of the speed of EMC's response, dozens of people were already working out a plan to preserve the data. Second, and more amazing to Rand, every well-rehearsed action by EMC seemed to flow from an understanding that approached empathy for Sears's plight. "They were not going to put us in the role of solving their problem," says Rand. "They took ownership of it."

EMC escalated the problem through four layers of the organization, all the way up to Leo Colborne, the vice president in charge. He got authorization to steal a new system off the assembly line and use it as a replacement for Sears. "There were never any questions about the impact on revenue," says Colborne. By Monday, when Sears's top executives started looking through the merchandising reports from Saturday, it was as if nothing ever happened.

Would You Like Us to Read Your Mind?

Of course, the best way to address a customer problem is to fix it before it happens. And for every hour that EMC spends mending faulty hardware and software, it spends nine hours anticipating and preventing such meltdowns. "Most of the time, we address problems before the customer even knows that there's an issue," says Walton.

EMC likes to call it "service and support mind reading." Sensors that are built into its storage systems monitor things such as temperature, vibration, and tiny fluctuations in power, as well as unusual patterns in the way data is being stored and retrieved -- over 1,000 diagnostics in all. Every two hours, an EMC system checks its own state of health. If everything is running smoothly, the log file is stored away. If the machine spots something that it doesn't like, it "phones home" to customer service over a line dedicated for that purpose.

Every day, an average of 3,500 calls for help reach EMC's call center in Hopkinton. But it's not people who are calling in to ask for help -- it's machines. Even so, it takes an ability to deal with pressure to be one of the 80 people who provide responses to machine-generated inquiries. In the span of a few minutes, EMC's systems at Ford Motor Engineering, Chase, GE Plastics, Citibank, ITT Hartford, and Ericsson dial in to request attention. About one-third of the calls from EMC's machines trigger the dispatch of a customer engineer to lay hands on the box.

"Mind reading" on that level isn't just a way to increase the speed of EMC's customer-service response. It also dramatically shortens the feedback loop on new products for EMC's engineers. It's what Ruettgers means when he says that EMC stands for "Everyone Meets Customers." (Actually, the name is derived from the last names of the company's three founders.) At most companies, service technicians live on the other side of a wall that separates them from the engineers who create products. EMC has torn down that wall. The call center sits right in the middle of the building where the software and hardware engineers work. If a machine phones home with something acute, the design engineers are summoned. "Usually, when there's a problem with a piece of code, it's personal for the engineer who wrote it," says Rick Espanet, 42, a product-support manager who oversees the call center. "It's amazing what that buys us."

Great Service Pays -- If You Lose the Profit Motive

For all of the skill that EMC brings to bear on customer service, its most important innovation doesn't reside in its "mind-reading" technology or in its procedures for escalating problems. It lives in Ruettgers's insight, now implanted in the company's DNA, that customers get better service because EMC doesn't expect to earn a profit from fixing their problems.

Customer service is certainly run like any other business unit at EMC, with a budget of its own. But the absence of the profit yardstick changes the logic of how that business runs. EMC invests more in customer service than it gets in revenue from maintenance agreements, more than the $780 million it spent on R&D in 2000 (though it won't say how much more).

Consider one striking example: Kathie Lyons, who headed up a team of 20 people responsible for moving EMC spare parts around the world, understood that her unit would be under the gun during the 1996 Atlanta Olympics. The city center -- including Atlanta's business district, where several EMC customers housed their computer operations -- was snarled by traffic and security checkpoints for weeks. So Lyons stationed bicycle messengers at two dozen locations and stocked each one with EMC spare parts. As a backup, she hired two helicopters and kept them on standby during the games. "She never had to use them," says Frank Hauck, who ran EMC's customer service at the time. "But nobody ever said to her, 'Waste of money.' "

Doesn't Ruettgers have a right to expect an immediate payoff from that sort of spending? He doesn't think so. "What happens," he asks, "when one of my customers runs into a problem, and it would help if I flew five people out to help them through that situation? If I'm running a service organization, and I've signed up to deliver a certain profit line, then I've got conflicting objectives. The way we're set up, those five people are on that plane."

Dan Gaffney, 39, COO of eTrue Inc., which provides face, voice, and fingerprint recognition over the Internet, knows what it feels like to be treated that way. Organizations such as NASA pay eTrue to keep unauthorized users off their networks. His is not a storage-intensive business, but Gaffney does need nonstop access to the face and fingerprint data that's kept at eTrue's headquarters, in Southborough, Massachusetts, in order to make exact matches in less than five seconds. Last November, just before he was due to start testing eTrue's new service, Gaffney lost confidence in the system he had ordered from Sun Microsystems.

For EMC to get the business, it had to locate a machine at the end of the quarter and have it up and running within three days. "I'm still not sure how they did it," says Gaffney. "I'm a pretty tough customer." Gaffney was on the phone with EMC's sales group while the machine traveled by truck to eTrue's data center. A team of EMC engineers met the truck, unloaded it after regular business hours, and worked around the clock for the next two days to get it online.

Gaffney says he choked when he first heard EMC's price, but he soon became a convert. "There was no question of cost after I bought the machine," says Gaffney. "No one ever said, 'Installation costs are so much, and if we go over and above that, we'll have to charge you.' I literally said, 'I need you guys to be there at 12:30 tonight,' and they worked until 4:30 in the morning without ever saying, 'Gotta go.' Yeah, it was painful to write the check. But what I got in the end made me understand why I paid a premium."

"Guilty Until Proven Innocent"

No competitive advantage lasts forever, of course, and EMC's rivals are betting that its commitment to service may be less meaningful as competition in the storage field heats up. The company's phenomenal growth has drawn intense scrutiny from Compaq, Dell, Hewlett-Packard, IBM, and Sun, among others. And all of them are hammering away at what they call EMC's greatest vulnerability: the relatively high cost of its storage systems.

Up till now, as in the case of eTrue, EMC has gotten away with charging premium prices because its service has been so exquisite. But mold breakers like Dell and upstarts like Network Appliance are asking a new question: Why pay more in order to manage complexity? Why not simplify storage technology, with lower-cost products that are easier to install and maintain, so that service itself becomes less crucial? "If we make our products simple for customers to order, install, and manage, they won't need the same level of complex support and service," declares Bruce Kornfeld, worldwide marketing director for Dell PowerVault storage. "It's plug and play."

Mike Ruettgers is unmoved: "I don't see it. The equipment will continue to get better, but whenever you put these things together, it gets pretty complex. Most customers don't have the time or the expertise to do it themselves." Indeed, Ruettgers turns the competition's argument inside out: The storage landscape is shifting so rapidly, he claims, that great service -- and the window it opens to the needs and habits of the companies that use the technology -- will become even more important going forward.

There's intriguing evidence to support his argument: EMC has become so proficient at solving its customers' problems, so prepared to do whatever it takes, that more and more customers are calling EMC to help fix problems that aren't even caused by its products. And EMC is prepared to tackle those problems too -- in accordance with a principle it calls "guilty until proven innocent."

The principle works like this: J.P. Morgan Chase & Co. was having a problem with a big database just before New Year's. The bank does millions of transactions every day, which requires a ton of computer hardware and software, but it appeared as if the problem originated with the EMC machines where the data is stored. When Chase called, EMC began its drill of breaking the problem down and re-creating it in the lab. After 12 hours, all signs pointed to a piece of software in the IBM computers that run the databases. It was a huge investment of time and resources -- only to find that the problem originated with someone else's equipment.

What did EMC do next? It didn't send a bill to Chase. Instead, EMC's customer-service and engineering teams called their counterparts at IBM in Austin, Texas to share the information that they gathered, which helped IBM resolve the problem more quickly.

Needless to say, customers love that kind of treatment. In fact, they love it so much that more than 20% of the problems tackled by EMC wind up originating with some other company's equipment. Is that a cost or an opportunity? Does EMC get credit for its efforts, or does it foot the bill for its competition? "I'm not that sure we get a direct payback," says Ruettgers. "But it's part of our philosophy. We have a deep belief that if you provide good service to customers, you will, in fact, get rewarded for it -- even though it's hard to measure."

Paul C. Judge (pjudge@fastcompany.com) is a Fast Company senior editor.

Sidebar: The Customer-Service Agenda

Who: Nigel Alderson
What: Weekly change-control process
Why: To maximize flexibility and to minimize errors with on-site product upgrades

Nigel Alderson, 47, oversees the most ticklish part of EMC's customer-service operation: the 1,800 changes that get made each week to EMC systems in the field. The vast majority of those upgrades are performed at the same time -- in the dead of night on Saturdays, about the only time that big customers can do without access to their crucial data.

The process of getting ready for Saturday used to begin with a six-hour conference call on Thursday, known as "change-control day." District service managers dialed in to the war room in Hopkinton, where Alderson and his team sat with stacks of paper that listed every job scheduled for the coming weekend. Each report included a list of planned steps, regardless of how seemingly simple. The six people in the room would go through each item, line by line, each asking the same question: What could go wrong? On about 6 out of every 10 jobs, Alderson's team flagged issues that had to be answered by noon on Friday for the process to move forward.

As EMC has grown over the past five years, the conference call dragged on, sometimes for as long as 18 mind-numbing hours. So Alderson's team has automated the change-control process and moved it to the Web. Instead of filing a piece of paper with the folks in the Hopkinton war room, customer engineers now log on to a Web site and complete an online form.

The payoff from all of this work? Back in 1995, when EMC initiated the change-control process, field engineers were getting it right about 75% of the time. Today, Alderson claims a success rate of 99.5%. The goal is 100%, according to Alderson. "Do the math," he says. "Today, 5 to 10 changes that we make fall short and affect customers in some way. In our business, that's unacceptable."

Add New Comment

0 Comments