Debutant Mark Hughes' 39th minute strike was cancelled out in the final minute by Gary Mulligan, before substitute Dean McDonald struck a sublime volley to seal an unlikely win for the home side.
Despite the glorious outcome, Gillingham's play for most of the game was characterised by sloppy passing and mistimed tackles.
The visitors began the stronger of the two sides and in the 18th minute Derek Niven narrowly failed to turn in Kevan Hurst's low cross.
Hurst was proving a threat from set-plays and, a minute after Niven's missed chance, delivered a vicious corner that Gills' keeper Kelvin Jack missed in the air. Alan O'Hare headed the ball across goal only for Colin Larkin to hook the chance over the bar.
As Gillingham struggled to gain any rhythm, they were hit by a sucker-punch six minutes before the break. Hughes picked up a loose ball 20 yards out and fired a low strike that took a slight deflection on its way past Jack.
But on the stroke of half-time Jack came to the home side's rescue, producing a fine low save to turn away Larkin's shot after the striker had burst through on goal.
Gills boss Ronnie Jepson made a double substitution at half-time, replacing the ineffective Guylain Ndumbu-Nsungu with McDonald, and hauling off Andrew Crofts for Alan Pouton.
But the visitors continued to threaten and on 76 minutes thought they had won the game only to see Caleb Folan's close-range strike ruled out for offside.
McDonald then saw a header loop narrowly over, before Leon Johnson came even closer, heading Mark Bentley's cross against the bar.
But the equaliser finally came when Matt Jarvis skipped past full-back James Lowry and found Mulligan, who tapped home from close range to seemingly secure a share of the spoils.
McDonald then stunned the visitors moments later, chesting down Bentley's cushioned header, before unleashing an unstoppable volley past Barry Roche into the far corner to break Chesterfield's hearts.