I never knew the answer to this, until now.  I only remembered a reference to stercobilin from Silence of the Lambs.

“Bile comes from your gall bladder and helps your body digest food,” said Anish Sheth, M.D., assistant professor at Yale Medical School and author of the book What’s Your Poo Telling You? “It’s metabolized by the bacteria in your large intestine, leaving behind a byproduct called stercobilin—and it’s that stercobilin that gives stool a brown pigment.”

Without stercobilin, your poo would actually be a sort of pale, off-grey color, like white clay. This really does happen from time to time, Dr. Sheth said, when something is blocking a patient’s bile duct, so that bile can’t get from the gall bladder into the intestinal tract. The cause could be as simple as a gall stone, or as ominous as pancreatic cancer.

Science Question from a Toddler: Why is poop brown? Boing Boing.