The MP for the Wrekin says that if Boris Johnson is removed it could put pressure on parliament for an early election. He says his colleagues could play into Labour's hands.
Anger over allegations that Downing Street hosted leaving parties and 'bring your own booze' Fridays during lockdown has led to some of the Tory MPs writing to the influential Conservative 1922 committee calling for a vote of no confidence in the Prime Minister.
There would have to be letters from more than 50 of the Conservative MPs, to set such a vote in motion.
Mr Pritchard spoke in the Commons lobby shortly after Bury South MP, Christian Wakeford defected to Labour and long-standing MP David Davis told the Prime Minister it was time for him to go.
But Mr Pritchard said he is firmly against a change at the top of his party.
He said: "Any colleague who believes replacing PM will help their electoral chances need to be careful what they wish for.
"The Labour plan is to 'Remove Boris, the best election campaigner in a generation; create disunity and distraction within Tory ranks; Put pressure on for an 'early election'."
While Mr Pritchard joined a steady stream of backbenchers giving media interviews backing Mr Johnson, other Tory MPs in Shropshire remained tight-lipped. Ludlow MP Philip Dunne said: "Sorry I don't comment on such issues."
After Wednesday's Prime Minister's Questions Mr Pritchard said it had been a very good PMQs for the Prime Minister.
He said that Mr Johnson's position had been strengthened as a result and described the so-called "pork pie plot" of rebel backbenchers as a "miniature pork pie, well past its sell-by date".
He said fellow MP Mr Davies, who had had supported over many years was mistaken and a little ill-judged.
But he maintained that the mood in Parliament was much better than it had been for some weeks.
"People are saying lets build back better, focus on the economy. While the Sue Gray inquiry is important we are seeing a lot more support for the Prime Minister. The mood has changed and is much better.
He added: "Despite his best efforts Sir Kier Starmer fails to land any political punches. The PM is showing fight and grit. Any plots will falter and fail."
Across the Shropshire border in Wolverhampton, new red wall MP Stuart Anderson was among those also backing the Prime Minister.
He said he had “not been invited” to any pork pie plot meeting and added: “Fighting amongst ourselves will not get us anywhere.”
Mr Anderson urged his fellow MPs to wait for a forthcoming inquiry into Downing Street parties to be published.
He said: “I’m very focused on supporting the Prime Minister and moving forward.
“There’s never going to be any good time for us with a vote of no confidence. I don’t agree with it and I don’t think it’s the right way forward.”