Chatmon was born in Bolton, Mississippi. Chatmon's family was well-known in Mississippi for their musical talents; Chatmon was a member of the family's string band when he was young. He performed on a regular basis for white audiences in the 1900s.
The Chatmon band played rags, ballads, and popular dance tunes. Two of Sam's brothers, fiddler Lonnie Chatmon and guitarist Bo Carter, performed with guitarist Walter Vinson as the Mississippi Sheiks.
Chatmon played the banjo, mandolin, and harmonica in addition to the guitar. He performed at parties and on street corners throughout Mississippi for small pay and tips. In the 1930s he recorded both with the Sheiks, as well as with sibling Lonnie as the Chatman Brothers.
Chatmon moved to Hollandale, Mississippi in the early 1940s and worked on plantations in Hollandale. He was re-discovered in 1960 and started a new chapter of his career as folk-blues artist. In the same year Chatmon recorded for the Arhoolie record label. He toured extensively during the 1960s and 1970s. He played many of the largest and best-known folk festivals, including the Smithsonian Festival of American Folklife in Washington, D.C. in 1972, the Mariposa Fest in Toronto in 1974, and the New Orleans Jazz and Heritage Festival in 1976.
A headstone memorial to Chatmon with the inscription "Sitting on top of the World" was paid for by Bonnie Raitt, through the Mt. Zion Memorial Fund and placed in Sanders Memorial Cemetery, Hollandale, Mississippi on March 14, 1998.