“They had a shot and I got nowhere near it,” he says. “It went past the post and I just thought: ‘This isn’t working’. I couldn’t lift my hand up.”
All that treatment, all those injections since landing on his neck tumbling over former team-mate Bob Latchford in a league match the previous Friday, had all been in vain.
Just nine minutes into the European Cup final between Villa and Bayern Munich, Rimmer signalled to the bench. His night was over.
Rimmer, the man who had missed only one match in five years for the club. Rimmer, the man who had watched from the bench when Manchester United defeated Benfica at Wembley to lift the trophy 14 years previously.
“It was a long wait, ‘68 through to ‘82,” the 74-year-old remarks in wistful tone down the phone from his Swansea home.
The obvious question to ask is whether any part of him regrets not playing on? But he cuts it short.
“You can’t do that,” he says. “We got to the final as a team. It wasn’t just about me.
“Tony Barton, the manager, had been great. He wanted me to go out there. I’d had injections in my shoulder and neck. We’d done everything to keep it quiet.
“But in the end I had to say no. I couldn’t be the Jimmy Rimmer people would expect me to be and I didn’t want to let the team down.
“If I’m 100 per cent healthy and make a mistake, I make a mistake. It happens. But on that day I knew I couldn’t do it. Not against Bayern Munich. I had to think of the other lads. They were my mates. I would have been letting them down. In the end, I’m glad I made the right decision for the club.”
There can be few arguments there. Nigel Spink, who replaced him off the bench, wrote himself into Villa folklore with a legendary performance to shut out the German champions.
Rimmer, of course, wishes it had been him. Yet any regret has evaporated over the following 40 years and besides, it has still been quite the life: A decade spent at Old Trafford, where he played in the same youth team as George Best, being named player of the season at Arsenal and then, after hanging up his boots, a long stint coaching abroad in China and Canada.
“I only went to China for three months and 18 years later, I was still there,” he says.
Still, Rimmer admits what happened in Rotterdam came as a crushing blow at the time.
“I went into the dressing room with the club doctor David Targett and cried my eyes out,” he explains. “I remember the doc saying: ‘I’ve never seen you do that’.
“I took a bit of convincing to go out and watch the rest of the match. Luckily, I saw Peter Withe’s goal.”
What helps Rimmer is the knowledge he played such an integral role in Villa’s success. An ever-present in the team which won the First Division title in 1981, the club’s first foray into the European Cup would very likely have ended at the second round stage were it not for the saves he produced against Dynamo Berlin.
Rimmer remains immensely proud, meanwhile, of the fact he regularly tops supporter polls to find Villa’s greatest-ever keeper.
“They’ve got a good one now in Emi Martinez, haven’t they?” he laughs. “Maybe I’ll have to fight him out for the title in years to come? I’m still battling to be Villa’s No.1!”
Barring a Boxing Day match in 1979 and the final 81 minutes in Rotterdam, the latter jersey was his from 1977 up until December 1982. Arsenal, whom he had joined from United in 1974, were willing to offer a five-year contract but Rimmer had become distracted by the interest shown in him by Villa boss Ron Saunders.
He explains: “Alex Cropley, who had played for Villa, lived three doors down from me and he was always saying: ‘Ron’s been asking about you’.
“He got me tickets for one game and I went in the players’ lounge and liked what I saw.
“Arsenal had offered me five years. Bobby Robson wanted me at Ipswich too. They were a top team.
“Villa weren’t at that time but I saw young players who were going to be brilliant. And they believed in themselves. That is the biggest thing.
“We had some super players – not superstars – super players, who wanted to play for each other. I just looked at Villa and believed they were going to come good. How Dennis Mortimer never got an England cap, I’ll never know.”
One of the older members of the squad, Rimmer had a closer relationship than most with Saunders.
“We played golf. Ron was what you might call a bandit. His real handicap was about seven but he would always play off at least double!” he says. “He could be a man of few words but we did talk. There were times when he would ask for my opinion on things. We had a team of battlers who would never give up, just like Ron.”
Such qualities were critical to Villa’s success during a European Cup run in which they conceded only twice.
“You can’t lose many games if you only concede two goals all the way through a cup run,” says Rimmer.
“But that is the way we were. It was all about the team. We had injuries and we brought kids in, the likes of Brendan Ormsby and Gary Williams. They did a tremendous job.
“We were probably the most unfancied team to win the European Cup. But we’d won the league.
“It is a damned hard year to win a league. People kept saying Ipswich this, Ipswich that. No, we were good.
“The only disappointment is if we had stayed together we would have won more.
“I can’t say we would have won the European Cup again. But I think we would have been near the top of the league all the time and won the League Cup and FA Cup.
“That is how much I believed in Aston Villa, the club and the players.”