The best spite is always spiritual, as it aims for eternity.

From an account by the always-excellent Cotton Boll Conspiracy of a small-town South Carolina Episcopal Church:

Also buried at the church is Pvt. Alpheus Cushman, a New Yorker who served with Co. B of the 7th US Cavalry Regiment. The 7th US Cavalry was among military units sent to Upstate South Carolina during Reconstruction following the declaration of martial law in response to Ku Klux Klan violence in the late 1860s and early 1870s. 

Cushman, a farrier, was said to have fallen in love with a Union County girl, but grew ill, and his illness prevented him from marrying her, though it could also have been possible that the girl’s parents weren’t keen on their daughter being betrothed to a Yankee so soon after the war. 

Whatever the case, Cushman is said to have taken his own life out of despair, on May 20, 1871. 

After his death, the members of his company asked that they be allowed to give their compatriot a Christian burial. Locals agreed, but stipulated that they would choose the plot. 

Cushman was not only buried in the far corner of the cemetery, but his grave was placed north-south, unlike typical Christian burials, and every other one at the Episcopal Church of the Nativity, which is east-west. 

Of course, the 7th US Cavalry would gain notoriety a little more than five years later, when more than 260 members of the unit were wiped out at Little Bighorn.