When the Error Message Comes From 50 Million Miles Away

…spacecraft engineers have to get creative.

The Galileo spacecraft was already far from Earth when problems cropped up with its antenna. (NASA)

Robotic spacecraft have traveled the solar system for more than 50 years, exploring places astronauts can still only dream of visiting. But with no human aboard, when something goes wrong, it’s up to ground control to pinpoint the problem—and fix it from millions of miles away. Sometimes all the ingenious engineering in the solar system wasn’t enough to keep a mission from being lost in space.

In a small, windowless room at Florida’s Kennedy Space Center on October 18, 1989, NASA project manager Bill O’Neil was monitoring the launch of space shuttle Atlantis. He was close enough to feel the vibrations but he focused on the special cargo being carried aloft: the Galileo spacecraft, finally on its way to Jupiter and its moons. Astronauts took a picture of the spacecraft (above) shortly after it left the cargo bay of Atlantis.

“I was practically numb,” O’Neil says, describing that day. Galileo had been delayed many times since 1982; it was even packed up at Kennedy, ready for a May 1986 ride into orbit, when Challenger exploded that January. New safety rules in the wake of the accident forced the team to replace the hydrogen-fueled Centaur upper stage that would rocket Galileo toward Jupiter in a speedy two-year journey with a safer but less powerful solid-fuel upper stage and a new trajectory—one that would take six years and require slingshotting around Venus and, twice, Earth for gravity assists.

For a year and a half, Galileo sailed smoothly through the inner solar system. On April 11, 1991, the operations team sent a command to deploy the mission’s most important piece of communications hardware: the high-gain antenna, a towering umbrella with a powerful transmitter. Within minutes, they knew there was a problem: The antenna had failed to deploy.

Three of the extension ribs in the umbrella were stuck. For the next year, the Galileo team worked to troubleshoot the spacecraft. They sent commands to alternately turn the vehicle away from its sun orientation, then back again, hoping the heating and cooling would expand and contract the metal in the antenna tower and “walk” the ribs out of their lockdown. They tried “hammering” the deployment motors by pulsing them repeatedly. Nothing worked. “At first there was a lot of optimism that we would figure it out and get the [antenna] open,” says O’Neil. “That turned out not to be true.”

Fortunately, Galileo itself had a fix. Compared to its high-gain companion, the spacecraft’s low-gain antenna was 10,000 times slower in data transmission, but it could have been worse. The spacecraft’s engineers made radical changes to both its software and its ground control receivers, increasing the low-gain antenna’s initial transmission rate by 100 times.

On December 7, 1995, Galileo reached Jupiter and became the first spacecraft to settle into the giant planet’s orbit. “We had undertaken an audacious tour of Jupiter’s miniature solar system, and despite all the challenges, we never missed an encounter,” O’Neil says. The signals may have been weak, but NASA received them—including evidence suggesting that under the surface of the moon Europa, there are liquid saltwater oceans—until Galileo was finally deorbited in 2003.

See the gallery below for more stories of dramatic spacecraft saves.

The Curiosity Rover

(NASA / JPL-CalTech)

The Curiosity Rover, NASA’s Mars Science Laboratory, has been grabbing attention since August 2012, when it landed in Gale Crater in a spectacular fashion. The rover began its trek toward Mount Sharp, about five miles away, and almost immediately began sending back discoveries from the Martian surface. Then one night last February, project mission manager Jim Erickson was awakened by a call. “Ground control said they were getting some interesting telemetry from the Mars rover relay,” Erickson says.

Team members at NASA’s Jet Propulsion Laboratory in California had noticed the problem when Curiosity failed to send back the data it had recorded for the day and go into regular sleep mode. That meant something was wrong with the rover’s main computer. The engineers woke up Erickson and switched the rover over to its secondary computer, which they call the B-side, a move that automatically puts the rover into safe mode so the team can root around for the problem.

Erickson was concerned, but he was a veteran at space hardware troubleshooting, having worked on the Mars rovers Spirit and Opportunity—and Galileo. Sure enough, his team located damaged memory hardware in the A-side primary computer. The team could reprogram the A-side to not recognize the damaged parts, but Curiosity was working just fine using the B-side, so they decided to keep it as the rover’s primary system. In the space business, it’s always good to fly with a spare.

Comment on this Story

comments powered by Disqus