This restored home was the residence of that unreconstructed Confederate statesman and soldier, Robert Toombs, former brigadier general in the Confederate army who served briefly under President Jefferson Davis (a man he despised) as Secretary of State of the Confederacy. On May 11, 1865, Yankee troops arrived at his home with orders to hang the former leader from an oak tree on his front yard. His wife, Julia, stalled the troops until her husband could escape on horseback. He fled into the Georgia mountains and became a fugitive in his own country, eventually escaping to Cuba where he made his way to Paris, there to live a bitter life. He returned home just before Christmas in 1866, upon learning of the death of his last living child, Sallie. He refused a presidential pardon from Andrew Johnson. "A pardon?" Toombs asked. "I have done nothing to ask forgiveness for, and I have not pardoned them yet!" He spent his remaining years denouncing the "carpetbag rule" of the Reconstruction South, participated in state politics, and was a major force in rewriting the Georgia Constitution. Twenty years after the end of the Civil War, Toombs died at the age of 75.

The frame Federal-style house with a Greek Revival portico is filled with Toombs memorabilia and antique furniture. The house was originally built in 1797 but went through five architectural face-lifts: Federal, Plantation Plain, Greek Revival, Victorian, and Neoclassical. Relatives of the Toombs family lived here until 1973, when the state purchased the house and hired Edward Neal, a Columbus architect, to restore it as it was at the time of General Toombs's death in 1885. Visitors can view a dramatic film, portraying the elderly Toombs relating his sad story to a young reporter.