Who was right? Turns out they both were. The moon appeared different at different times, mostly depending on the sun angle. When the sun was high, the surface tended to look brown. At low sun angles, it appeared more gray.
Mike Collins of Apollo 11 later recalled "a cheery rose color" near noon. Jim Irwin of Apollo 15 wrote (in his 1973 book To Rule the Night) that the moon was "all ochers, tans, golds, whites, grays, browns—no greens, no blues."
Well, not so fast. When orbiting over the night side of the moon in Earthshine (the equivalent of moonlight, but brighter), the surface looked "pale blue," according to Apollo 16's Charlie Duke. Collins also described a "bluish glow," which his crewmate Neil Armstrong thought beautiful. "It's a view worth the price of the trip," he radioed to Mission Control during one nightside orbit.
Even walking on the surface, the astronauts saw a surprising amount of color change. In their postflight science report for Apollo 12, Pete Conrad and Alan Bean commented:
The first day, everything appeared to be dull gray. If we looked very closely, of course, now and then it was possible to observe a white rock or, in an area where we had disturbed the soil, perhaps a slightly different shade of gray. Between the first and second days, definite color change accompanied the Sun-angle change. On the second day, everything that had appeared to be gray on the first day started looking either a dark or a tannish brown.Earth, not surprisingly, presented a more varied pallette when viewed from the moon. "The red Earth colors were easy to distinguish, but the greens and grays were difficult to distinguish from the blues," reported the Apollo 12 astronauts after their return.
That crew also was treated to the stunning sight of a total eclipse of the sun by the Earth, just a few hours before their splashdown in the Pacific. The planet itself was pitch black. "All you can see is this sort of purple-blue, orange, some shades of violet, completely around the Earth," Bean radioed to the ground. While the planet was in darkness, the crew saw lightning flashes in the atmosphere, the lights of cities, and the bright light of Venus, just off the Earth's limb. Bean, who'd just returned from walking on the moon, said, "This has got to be the most spectacular sight of the whole flight."