During the American Civil War, the Windsor mansion played a role in Confederate and Union forces. Confederate forces used the mansion's cupola as an observation platform and signal station. Later, Union General Ulysses S. Grant and his troops took control of the mansion in 1863 as part of his Vicksburg campaign. The mansion was used as a hospital and observation station by Union troops, while the Daniell family was allowed to live on the third floor during the Union occupation.

Despite surviving the war, the mansion eventually suffered a tragic fate. In 1890, a fire broke out on the mansion's third floor, possibly caused by a guest dropping ashes from a cigarette or cigar into construction debris. The estate was destroyed, leaving only the columns, balustrades, cast iron stairways, and pieces of bone china.

For over a century, the mansion's outward appearance remained a mystery until an 1863 sketch of Windsor mansion was discovered in the papers of Henry Otis Dwight, a former Union officer of the 20th Ohio Infantry. It is believed that Dwight sketched while his unit was encamped on the mansion's grounds. Today, the Windsor Ruins site is administered by the Mississippi Department of Archives and History and contains 23 standing and five partial columns.

The ruins of Windsor Mansion have been featured in two movies. Raintree County, a 1957 film starring Elizabeth Taylor, Montgomery Clift, and Eva Marie Saint, featured the ruins as a backdrop in one scene. The ruins were also featured in Ghosts of Mississippi, a 1996 movie about the assassination of civil rights leader Medgar Evers. In the film, Windsor ruins represent the home of Evers' assassin, Byron De La Beckwith.