Perhaps he needed to clear his head or maybe he was simply enjoying the moment after his team had pulled off the biggest of upsets in defeating Darlington.
The club that was playing Saturday afternoon park football four years ago had beaten their former Football League neighbours 2-1.
Two goals from Danny Earl earned a victory that ended in drama with an 88th-minute penalty save by Aycliffe keeper Scott Pocklington.
“We didn’t expect that,” admitted Oliver. “I’d have been delighted with a point. Darlington missed a few chances, but Danny’s second goal was good enough to win any game.
A chip over the keeper, it was class.
“It’s probably the best result in the time I’ve been here.
For me, it’s an historic result for the town to beat Darlington.
“I think Darlington were treated dreadfully, being demoted four divisions, but they knew at the start of the season how tough this league is and I still think Martin (Gray) and Brian (Atkinson) will get them promoted.”
There was another manager at Moore Lane who will have been just as pleased as Oliver – Spennymoor Town’s Jason Ainsley.
Not only did he see his team’s primary challengers for the Northern League title beaten for the fourth time this season, he will have noted how suspect at the back Darlington are.
Quakers have conceded 25 goals in 23 games and the manner in which they gave goals away on Saturday meant they had only themselves to blame.
As in the defeat at Team Northumbria a fortnight ago, Aycliffe soaked up the pressure and hit Darlington on the break.
Both Earl goals saw him latch on to long balls, the first on the half hour seeing him shrug off Stephen Harrison before slotting past Jack Norton.
He did it again before halftime, this time evading Gary Brown before executing a sublime chip.
Gray’s team have not kept a clean sheet in 14 league matches and he’s had enough of it.
He said: “We got done on the break and it wasn’t good enough so things are going to change. The players have been told.
“We’ve got to get back to being solid, getting clean sheets and w i n n i n g games that way.
“ P l a y e r s need to start listening and taking on board what’s expected of them. If they don’t, then changes will be made within the club. You can’t give goals away and expect to come back.
“No disrespect to Aycliffe, but we can’t afford to be coming to these places and losing.
“Give any team two goals, in what is their cup final and they’ll defend for their lives to hang on to their lead.”
While conceding goals has been an issue for some time, Quakers are no longer finding scoring goals easy.
Between Earl’s two goals, Amar Purewal headed past Pocklington to briefly provide parity, but in the second half, when Darlington were entrenched in the opposition half, they missed a handful of chances.
David Dowson, Stephen Thompson and Craig Gott were all at fault, while playmaker Thompson’s input was diluted by Aycliffe’s manmarking.
Oliver added: “We didn’t think we’d match Darlington man-to-man in a 4-4-2, so we thought we’d crowd the midfield and play on the break.
That worked for both of Danny’s goals.”
Thompson had a chance to provide a late leveller from the penalty spot, after James Oliver felled Dowson, but Pocklington guessed correctly, diving to his left to become Aycliffe’s hero.
It was a sweet moment for a keeper who was in Quakers’ youth team and also had an unsuccessful trial at Darlington last year.
Oliver may believe that Darlington will still be promoted, but many more results such as Saturday’s will leave them playing for second place.
Their lead remains four points, but second-placed Moors have four games in hand on Quakers, who have lost three of their last five games.
Last year the top two – Spennymoor and West Auckland – lost only five games each, albeit in a 42-game season.
Gray added: “I don’t think we’ve lost any momentum, but I’m telling you now there’ll be a massive improvement at Durham next week and believe me the players know about it.
“You see what happens next week. Under no circumstances will they not come out of the blocks.
“Some strong words were said by me and what I say goes – it’s as simple as that. But we’ll stick together because we’ll have little blips like this during the season.
“But we can’t afford any more days like this if we want to challenge for the league.”