Charles A. Peabody was born in Woodbury, Connecticut on July 29, 1810 to William Henry Peabody (1760-1841) and Ruth Bulkley (1773-1843). He was apprenticed to a tailor. When he moved to Columbus in 1833 he and his brother George H. Peabody operated a clothing and tailoring store in Columbus until it was destroyed by fire in 1846. After the crash of 1837, he moved to a piece of wild land he owned in Alabama. There he discovered his true calling and devoted his energy to horticulture and experimental agriculture.

In 1832 Charles Peabody married Frances Harriet Williams ( 1812-1885), a niece of a former resident of Columbus, Edward E. Powers. They were married 52 years and had nine children. Charles A. Peabody became the agricultural editor of the Muscogee Democrat in Columbus in 1848 and was one of the founders and editors of the Soil of the South, a monthly agricultural journal published in Columbus from 1851 to 1856. He became the horticultural editor of American Cotton Planter until 1858. He emphasized the use of native plants, trees, flowers and shrubs.

Charles A. Peabody died October 1, 1884 and was buried from Trinity Episcopal Church in Linwood cemetery.