Haller Nutt was a Southern American planter born on February 17, 1816, on Laurel Hill Plantation in Jefferson County, Mississippi. His father was Dr Rush Nutt, and his maternal grandfather was David Ker, the first presiding professor of the University of North Carolina at Chapel Hill. Nutt was educated at the University of Virginia from 1832 to 1835 and then returned to Mississippi to help his father manage the Laurel Hill Plantation.

He owned several plantations, including Araby, Evergreen, Winter Quarters, Cloverdale, Laurel Hill, and Longwood. Nutt mainly grew cash crops, including cotton and sugar cane, which brought him considerable wealth. He made a net profit of more than $228,000 from agricultural enterprises in 1860 and owned 43,000 acres of land and 800 enslaved people. Nutt's fortune before the Civil War was more than three million dollars.

During the Civil War, Nutt suffered significant financial losses from destroying his cotton fields and real estate. However, General Grant spared the Winter Quarters plantation because Nutt was pro-Union. Nevertheless, the appropriation of stores and supplies by the Union and Confederate armies led to the foreclosure of Nutt's plantations in Louisiana. After the war, he filed documents with the federal government to compensate for the loss of assets due to the Union army.

Nutt married Julia Augusta Williams in 1840, and they had eleven children. They decided to begin construction on Longwood in the spring of 1860 and hired Philadelphia architect Samuel Sloan to design the multistory octagonal in the Oriental Revival style. Construction of the exterior was completed by the beginning of the Civil War. With the threat of the Civil War looming, Sloan's artisans soon halted their construction, fearing for their safety, and fled back to the North. The basement story was completed by slave labour and was ready for occupancy by 1862. Longwood is the largest octagonal house in the United States.

Nutt died on June 15, 1864, of pneumonia, leaving his family to continue living at Longwood plantation. There were some tragedies in the Nutt family, as their daughter Caroline Routh Nutt died in infancy, and their son Haller Nutt Jr. died at 13 in 1861.