Savage headed home his eighth goal of the season in the 80th minute when he met a left-wing cross from Walsall old boy Edrissa Sonko.
It gave Tranmere the win that they needed to leapfrog Scunthorpe and move into sixth place.
But it was a result that was tough on Walsall goalkeeper Rene Gilmartin. He had earlier pulled off three stunning saves to deny Tranmere but his resistance was finally broken by Savage as Walsall slipped to a second successive defeat while Tranmere have now lost just once in their last eight games.
Having been frustrated by the woodwork when Michael Ricketts slammed a cross from Sofiene Zaaboub against the bar in the 19th minute, Walsall were indebted to Gilmartin for ensuring they went in at half-time on level terms.
Gilmartin first came to the rescue of Walsall after 21 minutes when Charlie Barnett intercepted a mis-placed pass from Zaaboub.
Barnett then released Moore who only had Gilmartin to beat but the Republic of Ireland under-21 international spread himself and beat away the shot with his legs.
Gilmartin pulled off an even more impressive save in first-half stoppage time after Walsall failed to fully clear a corner from Barnett.
The loose ball fell invitingly to Antony Kay who let fly with a fierce volley. But Gilmartin was again perfectly positioned to block the shot.
Gilmartin repeated his heroics nine minutes into the second half when he tipped a goalbound drive from Kay around the post.
That third let-off sparked Walsall into their best spell of the game only for Jabo Ibehre to waste two golden chances while Troy Deeney was denied by goalkeeper Danny Coyne.
When Kay then failed to hit the target with a close-range header, it looked set to be a frustrating stalemate for Tranmere before Savage came to their rescue.
Tranmere could then have gone on to record a more convincing win but Gilmartin denied them a second goal with another save - this time from Moore.