• Flight Today

Airliner Repair, 24/7

Boeing's traveling fix-it team has one goal: Get it airborne.

• By Stephen Joiner
• Air & Space Magazine, November 01, 2008

Jim Testin, director of Airliner repair services at the Boeing factory in Everett, Washington, made a prediction in July 2007 based on 27 years of experience. “I can tell you this: Something will always happen on Christmas Eve,” he said with the certainty of a man who has missed many family gatherings.

Five months later, on the night of December 24, a tow tractor pushed a Boeing 767 away from the gate at a busy European airport. And pushed. (Air & Space  agreed not to publish the name of the airline or any location.) Passengers heard “a very loud noise” and were invited to disembark via roll-away stairs. An ocean and a continent away, at Boeing’s Commercial Airplanes Operations Center in Seattle, an immense video screen displayed the status of the airliner shoved tail-first into a 14-foot blast-diversion fence. It was officially AOG—Airplane On Ground. For an airline with tickets to sell, that is exactly where you don’t want an airplane that can earn more than $200,000 a day.

From the dents and dings incurred on crowded taxiways to a jumbo jet bobbing in a Tahitian lagoon, Boeing AOG teams have seen and repaired it all. On call 24/7/365, ready to go anywhere around the world, Jim Testin’s quick responders keep over 12,000 extravagantly complex airliners airborne.

“We’re not bashful about promoting our ability to do that,” he says. “When we have a product that’s down and losing revenue, our number-one thing is to get that plane back into service.”

The Boeing team isn’t called in for everything. Airlines have incidents every day, Testin says, and most airport ground crews can handle routine repairs—a window cracked by a bird, a component failing calibration. An AOG intervention is required for the big things: an airplane off the end of a runway, landing gear accidentally retracted, a flaming tail drag, two 150-ton behemoths kissing wingtips on the ramp.

The AOG team is the most expensive roadside service on the planet. One call activates hundreds of people on a single wavelength: urgency. “I’ve been on calls where I could still hear the sirens of the emergency equipment in the background,” Testin says. Another aircraft might be rushed out of routine maintenance or temporarily diverted from another route to fill in for a disabled airliner. But in the house of cards that is today’s airline schedule, yanking even one aircraft from service can cause delays and cancellations.

From an office complex secluded behind tall evergreens on the Duwamish River, the Seattle operations center keeps tabs on Boeing airliners grounded around the world. “In a hospital analogy, this is the emergency room,” says center director Bruce Rund. It looks more like a downsized mission control. Concentric rows of consoles and monitors accommodate 30 controllers, project leaders, and engineers. On a 30-foot-wide video display, the sunrise is tracked across a world map in one screen. In another, blue icons representing AOGs in progress advance along a timeline, turning yellow, then red, as they approach the center’s deadline for action within 24 hours. Controllers conduct telephone triage to establish the severity of each situation and whether the airline wants a permanent repair or a temporary fix that will get the aircraft home. “We need to know quickly: How AOG are you?” Rund says. “Are you sitting in a gate with passengers loaded, or is this problem something you found during an overnight inspection?” The center defines an AOG as any incident requiring a response in less than 24 hours. But when carriers major and minor queue for critical care, everybody knows who goes to the head of the line: “Whoever’s got passengers on board,” Rund says. “It’s understood throughout the industry that we always look at those scenarios first.”

Rund’s staff takes 125 calls a day, and by the time an airline calls, it’s usually tried everything it can to fix a problem itself. The heavy morning departures in Europe and the afternoon rush in the Middle East create problems that show up during Seattle’s witching hours. “It gets really interesting in here when it’s 2 a.m. and we’re the only game in town,” Rund says.

And increasingly they are. As airlines downsize workforces, a busted airplane far from its corporate hub may not be swarmed by a phalanx of mechanics in company jumpsuits. Sometimes, says Rund, “There’s one avionics guy with a screwdriver.”

Owners of damaged airliners occasionally call with a one-item wish list. “They just want to know if they can fly the plane without doing anything,” Rund says. He cites an airline maintenance director pressing for flight approval after sustaining a hammering in a hailstorm. Boeing engineers determined that wing components were damaged beyond limits. Ten minutes of a carefully worded reality check, plus an offer to rush replacement parts to the site, persuaded the impatient carrier to fix instead of fly. “Part of our job is to be the voice of reason,” Rund says.

Incident reports advance to the center’s resident engineers. With broad experience across the aircraft types in Boeing’s fleet, these frontline troubleshooters huddle to prescribe the most immediate relief. In cases of structural damage, which make up 60 percent of the calls, the solution usually involves collaboration with specialized engineers elsewhere in the company—what Rund terms “the brain surgeons.”

“We get the customer on the line, we get the brain surgeons of Boeing on the line, and we sit in a collaboration room and videoconference,” he says. Plasma screens with smart boards facilitate sketches on digitized blueprints and photos. By now, an AOG survey team in Washington State has been alerted and is standing by. After each engineer confirms his understanding of the damage and votes a course of action, the plan is presented to the airline.

Order is supplanting chaos. Just getting the aircraft into Boeing hands has a therapeutic effect on stressed-out airline execs. It’s also a healthy antidote to denial when damage is severe. “By this point, in most cases the airline recognizes that— and will tell us—‘This is beyond our capabilities,’ ” Rund says. “Then it’s time to get Jim’s team involved.”

“This, By Far, is the best job in Boeing,” AOG mechanic Bernie Dalien shouts over rivet guns and rock ’n’ roll. “The boring, the mundane, the everyday thing—that’s not our gig.”

We’re in an enormous halogen-lit hangar in Western Europe, standing between two separated segments of a 767. Earsplitting music thunders from an iPod boom-box. Widebody X is the one recently spindled on the fence.

As the last of its deplaned passengers straggled to their destinations, the AOG survey team was en route. The four-member first wave is often in the air from Seattle on just four hours’ notice. Says Paul Amrine, quality assurance supervisor on this project, “Sometimes we go to work in the morning and end up having to ask our wives to bring us a packed suitcase.” Amrine himself arrived at the hangar after back-to-back surveys of incidents in Shannon, Ireland, and Taipei, Taiwan. The team appraises the aircraft, documenting what Boeing calls “discrepancies” (a torn-off wing, for example). Man-hours, parts and resources, and a time-flow to a rock-hard completion date are calculated. Back in Everett, a contract is drawn up that includes a firm price. “And the customer either says yea or nay,” Amrine says. In the death match of airline competition, the yeas usually have it.

The reckoning takes into account more than just the repair. A critical shortage of “lift”—the pool of aircraft on the market to replace one scrapped—is a big factor. The waiting list for most Boeing models is three years, and used airliners for sale or lease are scarce. “The book value of the plane, plus the fact that you can’t get a new one for another two or three years, is what dictates whether you fix it or not,” AOG engineer Craig Oppedal says. For a 1998 Boeing 767, it’s cheaper to keep it.

Up on a hangar balcony, a smattering of airport office workers watch the drama of deconstruction. “Most people have never seen a large aircraft come apart like this,” says Mike Carpenter, project team leader. But looky-loos glimpse only airplane-incognito-on-ground. Among the first implements of an AOG team is brown paper to mask identifying airline logos on the airplane during repair. Boeing maintains doctor-patient confidentiality with customers not eager to have their brand name associated with an embarrassing incident—much less advertise the fact that passengers will be boarding an airplane that lately has been in two pieces.

Photos of the incident conveyed only skin-deep gashes on the underside of the empennage. I’d seen as much inflicted on cars in mall parking lots. But the survey team recognized that this was no mere panel-bender. “Just by the external location of the damage, we pretty much knew what to expect,” Paul Amrine says. For a ground-handling incident, it could hardly have been worse. A fence stanchion penetrated at the precise spot to puncture one of the largest, most critical components on an airliner: the rear pressure bulkhead. The 16-foot-diameter dome-shaped aluminum barrier is sandwiched between the fourth and fifth fuselage segments and seals in life-supporting cabin pressure. These bulkheads are constructed as integral units, so when they are substantially damaged, they must be replaced, not repaired. The instructions have only three steps. Pull the $120 million airliner completely in two, insert bulkhead, put halves back together again. In three weeks.

Spread across the hangar floor is a half-acre of cranes, jacks, crates, and tool cases. “This is our portable factory,” says Mike Carpenter. The gear, all on casters or pallets, is designed for transit, rapid setup, and tear-down. Once the contract is signed, the AOG operation’s second wave—the mechanics, engineers, and inspectors, plus the portable factory and a cargo hold of parts—descends en masse.

“I’ve got to get 36 people and all logistics on site in a matter of days,” Carpenter says. He picked his team from AOG staffs at the Washington facilities (another small group in Long Beach, California, tends the McDonnell Douglas fleet). Some have expertise specific to the 767, “but most can work on any Boeing plane any time,” he says. “Structure is structure.”

AOG team accommodations range from tents beside dirt runways in underdeveloped countries to a blur of bland airport hotels. Anniversary and birthday no-shows, chronic jet lag, continual room service sandwich platters—all part of the job.

It’s not for everyone. Within Boeing’s rank-and-file, Testin’s group tends to be conspicuous as self-directed overachievers. “The cream of the crop,” Bernie Dalien says bluntly. “There’s a lot of animosity toward us in the factories because AOG is so difficult to get into. A ton of guys back there would love to have this job.”

Dalien would know. On the arc to AOG, he paid a decade of dues on the 737 and 757 production lines, accumulating skills like merit badges. Boeing’s average production employee carries seven job certifications; the average AOG member, 28. A competent electrician in Washington you may be, but in AOG you’ll also need to drive a rivet and drop an engine with the best of them. And play nice with your fellow Type A’s. Candidates for a vacancy are sent on tryout repairs to far-flung locales, not only to test their skills but also to gauge how they relate to others in the tight-knit team. “They’ll bring us a guy who really shines in the factory,” Dalien says. “But take him out of his comfort zone and put him in a situation like this, and you find out his personality’s not cohesive with the rest of us. So he’s weeded out.”

Nor is an AOG career designed for your inner nine-to-fiver. “We work two 12-hour shifts, around the clock, seven days a week, until we’re done,” Jim Testin says. “We don’t take holidays. My first 11 years in AOG, I missed Christmas nine times.” He pauses to tally the number of passports he’s filled—six—then shakes his head. “I told my wife I would only do this for three years.”

Still, for an average of just one opening per year, Testin gets as many as 500 applicants. Part of the draw is the mystique. Some of the feats the Boeing AOG operation has performed are commercial aviation legends. In 1988, a 747 aborting a takeoff bellied into a mud flat adjacent to New Delhi airport. Fully 70 percent of the airplane required AOG repair or replacement, at a total cost of $75 million. Then a mechanic, Testin worked 126 days straight in a circus-size tent dubbed the New Delhi Dome. Boeing returned the resurrected jumbo as pristine as one just off the assembly line. Two decades later, it’s still flying.

As it turns out, pulling a 767 in half is a straightforward procedure. “Everything about these aircraft is designed to be taken apart,” mechanic Fred Chadwick says. Before the tail section could be separated from the rest of the fuselage, the towering vertical stabilizer had to be removed. Overhead clearance in the hangar was three feet short, but resourceful AOG mechanics hyper-extended the airliner’s nose gear to tilt the tail down. A gantry crane hoisted the enormous fin to within inches of the rafters, then shuttled it over the airplane and lowered it to a jig on the floor. The tightness of the quarters made it a hold-your-breath procedure that dragged through an entire shift. But AOG tool engineer Jason Lockwood was over his first hurdle.

“When they told me I was going to be on this job, I didn’t sleep for a month,” he admits. Lockwood designs and directs heavy lifting operations. On this repair, heavy lift included the startling act of removing the 48 section—the unwieldy 12,000-pound final fuselage segment that includes the vertical fin, the horizontal stabilizers, and the damaged pressure bulkhead. A 48 separation was a career first for Lockwood. “I’d seen the videos of all the things that can go wrong,” he says.

Just aft of the rear lavatories, the 48 section meets the 46. At a join ring encircling the fuselage, the two fifths of the airplane are held together by 200 bolts—heat-treated, aircraft-grade fasteners (though they look unsettlingly similar to some you might have in a coffee can in the garage). A cable yoke descended from the ceiling crane and attached at five lift points on the tail section. After the crane applied 12,000 pounds of lift, the bolts were removed in a pre-determined sequence. Since the cable tension matched the section’s weight, “theoretically, we should just be able to take out all the bolts and the section will just sit there,” Lockwood says. In practice, determining the suspended segment’s exact center of gravity required some fine-tuning. A few bolts were left loose but threaded as final tweaks were made to the pitch of the yoke. “Once we’ve convinced ourselves it’s in a neutral attitude,” Lockwood says, “we take out those last bolts. Hopefully, it just hangs there.” If it pitches, binds, or pendulums, it could damage the precisely machined mating surfaces of the sections.

With barely a twitch, the fuselage de-mated. The crane backed off, bearing the 48 section, exposing the ruptured bulkhead, and providing mechanics in the now-gaping 46 with a view you hope you never get on your way to the lavatory.

Lockwood slept better thereafter. “It’s actually easier to put it back together than it is to take it apart,” he says.

The intimacy with the way Boeing airplanes dismantle now influences how they’re built. “We put three AOG team members on the 777 design teams, and we’ve followed that up on the 787 and the next-generation 737,” Jim Testin says. Specific design changes were integrated to minimize damage in common incidents and expedite AOG repairs afterward.

Still, damaged aircraft rarely go back together exactly as they rolled out of the factory. Change—as routine as replacing a stripped bolt with an oversized substitute—happens. “Any time you change the configuration of anything in the structure, you have to document it,” Craig Oppedal says. Every deviation from “drawing config,” the blueprint gospel of an aircraft as originally constructed, is subjected to his scrutiny and triggers a Field Rework Record (FRR). Oppedal’s been on AOG repairs that produced just a few FRRs, and others that resulted in 280. But there’s always something. All FRRs become part of the voluminous documentation that follows an airliner around for life.

The replacement pressure bulkhead has no deviations to document. In 1978, a Boeing AOG team repaired the bulkhead of a Japan Airlines 747 damaged in a tail-drag incident. Seven years later, the repair failed in flight, resulting in an explosive depressurization that tore off the vertical fin and severed all hydraulics systems. Some 30 minutes later, the aircraft slammed into a mountainside; 520 people died in the second worst airline disaster in history. Investigators determined that the AOG repair did not comply with Boeing’s own Structural Repair Manual. Boeing accepted 80 percent of the liability for the crash, while JAL accepted the remainder for neglecting signs that the repaired bulkhead was weakening.

Every night at 7:30, the day crew logs out and the second shift “ties in.” The playlist mellows, and reassembly continues around the clock. Mangled skin is replaced with new aluminum. The new bulkhead is sealed into the 48 section, fuselage segments are reunited, and the  vertical fin is dropped back into its slot.

The critical task of reconnecting the control cables and hydraulics that operate the tail’s rudder and elevators belongs to rigger Randy Pratt. He’s required to adjust the 175-foot tungsten steel cables back to Boeing factory specs to produce the flying characteristics the airplane came off the production line with—no matter how far out of whack the airline flew it. “They’ll say ‘Hey, what did you do to my airplane?’ ” Pratt tells me. Adjustments made for the flight preferences of particular pilots, or an accumulation of skin patches that skew the airplane’s aerodynamics, produce differences from manufacturer’s suggested settings. Soon after recouping their airplane, airline mechanics typically set about undoing Pratt’s precision work, adjusting cables and neutral positions to customize control to taste.

On day 18 the airplane is towed out into a squall of rain and snow. “We’ve got a page and a half of functional tests to do,” Mike Carpenter says. Control surfaces aft of the separation point are actuated and electrical components energized. The auxiliary power unit then over-pressurizes the fuselage for the “high-blow” test. Made of expandable, credit-card-thin aluminum, the pressure bulkhead in normal service holds seven pounds per square inch. In addition to testing the join ring seal, the 12-psi high-blow stretches the new bulkhead’s elasticity nearly to its limit. The test also results in instant break-in: Stretching and fatigue during routine lower pressurizations are thus minimized.

It’s day 20, and make-readies continue down to the last minute. At a sit-down with airline officials, every item on the survey list is verbally closed out. Mike Carpenter and Paul Amrine sign their names to a document attesting that the aircraft has been repaired to the standards of the Boeing Company and the Federal Aviation Administration (FAA standards are published online; the AOG team references them constantly) and is ready to return to service. Then the whole movable factory packs up.

Like now. “We’ve literally had passengers with tickets in their hands looking out the window of the gate at us as we were boxing up our equipment to leave,” Jim Testin says.

Corporate carriers with three-figure fleets and thousands of flights daily rely on Boeing’s AOG teams to discreetly get airplanes flying again, ASAP. But its real clients show up in ones and twos, clutching boarding passes and tripping over shoelaces untied for security checks. “There’s a tremendous effort that’s put forth on behalf of the traveling public,” Testin told me back in Everett, “to make these planes the safest in the world. If an airline calls, we’ll have somebody there.”

At the operations center, when I suggested that future airliners made of advanced impact-resistant materials might never become tomorrow’s AOGs, Bruce Rund had a prediction of his own. “Somebody will figure out a way.”

Comments