The swashbuckling England captain's career appeared to be over in 1965 when, at the age of 30, he suffered a horrific leg injury in a freak accident after his car ran out of petrol.
The following year, he released his autobiography, Ted Dexter Declares, which became a staple of every cricket fan's bookshelves.
A brief return to Test cricket during the 1968 Ashes series was followed by a successful career as a sports writer and commentator, and then in 1989 he began a four-year stint as supremo of the England cricket committee.
Throw in an attempt to unseat future prime minister Jim Callaghan towards the end of his playing career, and his role developing the use of statistics in the development of the game, and it is fair to say he has enjoyed quite a second innings.
Now, some 54 years after his first autobiography, Dexter has just released his second, 85 Not Out, a reference to the fact he is still going strong at the age of 85.
So why now? For the first time, after half an hour chatting cheerily in the neat garden of his home in Wolverhampton about his glittering career and colourful life, his mood darkens slightly.
"I wanted to set the record straight, after that last autobiography, which was unauthorised," he says.
Shortly after retiring as chairman of the MCC England committee, he was approached by Alan Lee, a sports journalist at The Times, about a biography he was writing, but Dexter declined an interview.
"I knew what he was doing, he was trying to diminish my role with the England cricket committee and the changes I made," he says.
When the book, Lord Ted, The Dexter Enigma, was published in 1995, he was somewhat irritated to find himself mentioned in the acknowledgements section.
"He had the cheek to thank me," he says, bristling with indignation. He initially pledged never to read the book, but when an article appeared in an MCC magazine making claims about his brief career in the City of London, he was incandescent.
"It came up with some eyewash about me being given an insurance job on the strength of my reputation, and that I had been spending all my time in the bookies," he says.
"I wondered where it had all come from, and of course it was from this book." The MCC magazine printed a retraction, but the episode clearly still rankles.
Perhaps surprisingly, when asked about the proudest moment of his eventful life, he does not refer to his stunning six-hour, 180-run stand against Australia at Edgbaston in 1961, or any of his other acts of derring-do on the cricket pitch. Instead, he cites his sometimes difficult four years as the chairman of the England committee.
Back in 1989, English cricket was in a bad way. The previous year England had endured the humiliating 'Summer of Four Captains', where Mike Gatting, John Embury and Chris Cowdrey were all dismissed from their roles during a five-match series in the West Indies, with Graham Gooch acting as skipper in the final defeat of the series at The Oval.
"Not so much a shambles, more a chalice of hemlock," he observes, wryly.
But he emphatically denies that plans for a radical shake-up of the way the team was managed had been in place for some months before he was invited to replace Peter May as chairman of selectors.
Indeed, he says he was totally taken aback when he was offered the job, and his initial reaction was to turn it down flat.
After further consideration, he laid down his own specific conditions for taking on the role as chairman of an England committee as opposed to the more traditional chairman of selectors.
There would be no more county delegates on the selection committee, with match-by-match selections and tactics becoming the exclusive responsibilities of the team manager and captain. His own role would be to oversee the long-term future of the game.
He didn't get off to the easiest of starts. Along with team manager Micky Stewart, he chose Mike Gatting to be captain, but the appointment was vetoed by Test and County Cricket Board representative Ossie Wheatley, forcing Stewart and Dexter to opt for their second-choice, David Gower.
Gower's first Test series saw England lose the Ashes on home soil for the first time since 1964 in a 4-0 drubbing, and Gower was dropped from the winter tour of the Caribbean.
But while he was in for a rough ride, and had a somewhat uneasy relationship with the Press, he did press ahead with a raft of reforms which have long been accepted as laying the foundations of the England team's own renaissance in the years that have followed.
He successfully lobbied for three-day county games to be increased to four, to better prepare players for the rigours of test cricket. He introduced the now-routine practice of overseas tours during the winter, to give players experience of playing in different environments.
And he gave England players fixed-term contracts after noticing how performances were affected by the pressure to secure a place on a winter tour, which was often their only way of earning a decent living.
"The batsmen particularly were playing more for their place on the tour than for the immediate needs of winning a match, and this undoubtedly added a tension to their play which didn't help," he says.
By this time, he had already transformed the way the game used computerised statistics to develop the player rankings system which is now used across the world.
In his book, Dexter reveals how much of the idea came from a magazine article written by a young man about his friend's new computer game, which used player data to create what would today be described as 'virtual' cricket games.
"It was clear to me that ranking players simply by their averages, both batsmen and bowlers, didn't tell even half the story because it didn't take into account all the other factors that might have surrounded a particular innings or bowling performance," he says.
Working with the two young men, he developed a mathematical formula which would weight scores depending on whether they had been achieved in a high- or low-scoring match, set against a rolling average since Test matches began. For batsmen, the strength of bowling would be a factor and for bowlers, the strength of the batting opposition.
Having lived for 12 years in Nice, he admits that one or two eyebrows were raised when he told them he was swapping the Cote D'Azur for the West Midlands.
He lives in Tettenhall, not far from the Shropshire and South Staffordshire borders. And despite the autumnal nip in the air as we chat in his garden, he has no regrets.
"I don't miss the heat, I like being able to experience different weather types," he says.
It was the death of two close friends in Nice that bought Dexter, and his wife Susan, a former model, back to the UK.
"We were surprised and alarmed by the complexities of dealing with these things from a distance and were sure that we didn't want to expose our own children to the same experience," he says.
His son Tom, daughter-in-law Catherine and two grandchildren had been living in the city for some years 20 years, with Ted and Susan regularly spending the summer months at their large Victorian house.
"We didn't want to live in London, nor could we afford the property prices, so we felt we would be more use as grandparents in Wolverhampton," he says.
The Dexters had no problem at all settling into their new city, with Ted being able to indulge in another one of his passions at South Staffordshire Golf Club, while Susan has been able to fulfil her love of art.
"In contrast to the lack of amenable painting companions in Nice, in the far less obvious milieu of Wolverhampton, Susan found just the group she'd been missing there," says Ted.
"A bonus has been hearing her new mentor telling the others that her work is what they should be aspiring to."
Ted adds that through his time in Nice he made a good friend who lives in Much Wenlock, helping him to widen his circle of friends in the West Midlands. He has even spent time as a volunteer teaching children to read at Wolverhampton's Graiseley Primary School, and still produces a regular blog giving his thoughts on cricket, golf and other matters.
And what does he think of cricketers today? He says the standard is pretty much the same as in his day, believing fast bowler Jimmy Anderson can be considered alongside 'Fiery' Freddie Trueman.
But he's not so sure about the batsmen wearing helmets.
"Those batsmen who have played all their lives in a helmet have never had to learn the skills that we bare headed batsmen were forced to acquire from day one," he says.
"It was drilled into us to always keep our eye on the ball. Ducking the head was the preserve of tail-enders only. I can recall very few nasty blows to the head in pre-helmet days. Nowadays it is a rare day of Test cricket which passes without the ball cannoning off helmets as batsmen take their eyes off the ball and duck."
He says with today's 24-hour media, it is not surprising that modern-day cricketers get caught out. But are today's well-paid stars living their gilded lives, less disciplined, and less well-behaved than in his day? There is brief pause, and then he replies with a glint in his eye: "I shouldn't think so."
Ted Dexter is 85 not out. And you wouldn't bet against him reaching his century.
Ted Dexter: 85 Not Out, by Quiller Publishing, is on sale now priced £20, with all proceeds going towards the MCC charity.