His teacher had told him to learn his times tables, and as a rebellious young man, he naturally ignored this instruction. So he was given the cane.
"I got punished for it and learned them in two days after that," said the Black Country retail magnate in a 2005 interview. "The experience completely changed me."
Mr Smith, best-known as the co-founder of Poundland with his son Steve, has died two days before his 80th birthday.
Steve said his father died at Ludstone Hall, Claverley, last Friday shortly after being diagnosed with lung cancer.
Steve said his father died in peace with his family by his side, and the family had helped him celebrate his birthday early.
Keith grew up in Willenhall, where he attended Central School. He said he had no interest at all in education until his experience of the cane.
After leaving school he studied at Wednesbury College, and at the age of 16 he followed in the footsteps of his parents by landing an apprenticeship with Willenhall engineering firm Wellman, Smith and Owen.
He didn't stay there long. While wife Maureen was expecting son Steve, he bought a gross of pens to sell to workmates and neighbours. So successful was the enterprise, that he decided there was more money to be made as a trader on Bilston Market.
"It turned out I was right," he said.
After 15 years on the market, he realised there might be more money to be made supplying the retail trade. Starting out from a garage in his mother's garden, he expanded to take on a 2,000 sq ft unit at Five Ways in Tipton. He soon ran out of space, and after moving to the Coneygre Industrial Estate and then a former Vono factory, he set up the famous Hooty’s cash and carry store – named after his nickname – on the Longacres Estate in Willenhall.
Dad started selling to other traders and opened up a small warehouse in tipton called hootys after his nick name
Son Steve, who had been attending Bilston Market with his father from the age of two, followed in the family business, before they sold the business and planned to retire.
"Customers used to come from all over the country and Ireland to buy from us," said Steve.
"When we moved to the Vono warehouse in Tipton, this was the biggest cash and carry warehouse in Europe, we had Lewis Collins from The Professionals come on an open day."
Keith and wife Maureen moved to Majorca, and the plan was for Steve and his wife Tracy to join them 12 months later, having agreed to stay behind for the handover of the business.
But at the last minute Tracy broke down in tears and revealed she couldn't bear to be apart from her mother, so the couple visited Keith and Maureen to explain they would be staying the in UK.
Sitting round the pool in Spain, Keith recalled a trader on Bilston Market who had a box in which everything cost a shilling.
"Out of everything he sold, it was this box that made him all his money," recalled Keith. "So I thought we could do the same with a pound."
Keith gave Steve £50,000 to open their first shop in Burton-on-Trent just before Christmas 1990, which took £100,000 in the first 10 days. The business headquarters was a £30-a-week office in Dudley Street, Sedgley, equipped with a second-hand desk and fax machine.
As Poundland became a staple of the high street, the company needed more storage space. And in an amazing twist of fate, the company opened a new warehouse on the site of the Wellman, Smith and Owen engineering works in Willenhall, where Keith had done his apprenticeship.
By this time he was living at Ludstone Hall, which he bought for £2 million in 1997, and regularly opened to the public to raise money for local charities.
Keith and Steve sold their business for £50 million in 2002, turning the pair into multi-millionaires.
In 2014, Keith opened a Poundland museum in the coach house at Ludstone Hall, chronicling the family's rags-to-riches story. Among the items are Keith's first business computer, complete with an 8in black-and-white screen and floppy disc drive.
"We had a client list of 19,000 stored on those discs," recalled Keith at the opening of the museum.
"It was all about getting return business so the computer was a godsend. Computers were a completely new thing back then but we always tried to be up with the times."
All of Keith's business enterprises are documented through a series of photographs and newspaper articles decorating the walls of the museum.
They included a 1974 cutting from the Express & Star, documenting complaints about delivery wagons blocking the roads around his 'fancy goods and popular toys retailer' in Poplar Avenue, Tipton.
"That was my first warehouse and we were ridiculously busy," he recalled in 2014.
"I always had the view that where business is concerned no news is bad news, so I never minded being in the paper.
"Some of the residents got fed up for a few weeks but it all died down eventually."
But for Keith, the most significant photograph in his museum depicted the huge interior of the Poundland distribution depot in Wellmans Road, which, by a strange twist of fate, was built at the site of the Wellman, Smith & Owen factory he started out at in 1960.
"You could say I came full circle," he said.
"Both myself and my dad worked at that factory and my nan had a house on the same piece of land.
"Then years later we ended up knocking it down. It's funny how life goes sometimes."
He leaves a widow Maureen, 79, and sons Steve, 60, Sean 51, and Liam, 33, and daughter Mandy, 57, as well as 10 grandchildren.