Mr Dunne said he was proud to have served in Government but disappointed to be have lost the health post.
The 59-year-old Conservative was one of the casualties of Prime Minister Theresa May’s reshuffle.
His dismissal brings to an end a seven-and-a-half year period in Government, having previously held a post in the Ministry of Defence.
Mr Dunne was today replaced by Caroline Dinenage as minister for health and social care.
He said this afternoon: “I have had nine-and-a-half years on the front bench, seven-and-a-half of those in Government, over five-and-a-half as minister of state in two of the largest spending departments in Government, which has been a fantastic experience and I am very grateful for the opportunity to serve and try to do my best to improve circumstances for the public at large through the health service and keeping the country safe through defence.”
He added: “It is always disappointing to leave something you really enjoy but I am not surprised as I have had a significant career in Government and all these things come to an end at some stage.”
The Health Secretary Jeremy Hunt, who remained in his post despite speculation he would be moved to another department, paid tribute to Mr Dunne on Twitter.
Mr Dunne’s dismissal comes the day after he was criticised for a response to a Labour MP during an urgent question in the Houses of Parliament, where he said people in hospital who were without beds could use seats instead.
He said: “There are seats available in most hospitals where beds are not available and I can’t comment individually what happened in her case but I agree with her it’s not acceptable.”
Shorty after he issued a statement in which he apologised for instances where NHS care fell below acceptable standards.
He said: “I responded to a question about Pinderfields Hospital in Yorkshire, which had earlier indicated chairs were available for those patients who were pictured on the floor.
“As the Prime Minister and the Secretary of State have done, I too have apologised for any cases where care has fallen below the high standards we expect.”
Garnier loses job as trade minister
Conservative MP Mark Garnier has also lost his job as trade minister just weeks after being cleared by an investigation into allegations of inappropriate behaviour towards a female member of staff.
The Wyre Forest MP said in a tweet that he was "very sad" to have lost his job at the Department for International Trade as Theresa May embarked on the second day of her Government reshuffle.
Last month a Cabinet Office probe in allegations Mr Garnier used derogatory language to his secretary and asked her to buy sex toys found that he did not break the ministerial code.
His dismissal today came as Mrs May's Cabinet met for the first time since a misfired reshuffle of top jobs on Monday, which saw Justine Greening walk out as education secretary rather than accept a move to work and pensions, while Jeremy Hunt turned down the PM's offer of the business brief, insisting instead on an expanded health and social care role.
Downing Street has yet to provide official confirmation of the latest round of changes.
Despite widespread criticism of the Cabinet shake-up, newly-appointed Tory chairman Brandon Lewis insisted the party is "not quite" in a mess.
Mr Lewis told BBC Radio 4's Today programme: "What we have seen yesterday is a real influx of new talent, not just my position itself, obviously."
Pressed on whether the party is in a mess, Mr Lewis replied "not quite" but admitted there was a "job of work" to be done.
After the conclusion of a Cabinet Office investigation into the allegations, Mrs May said that "a line should be drawn under the issue".
In a tweet on Tuesday morning, Mr Garnier said: "Very sad to have lost my job at @tradegovuk but looking forward to supporting @theresa_may Government from the backbenches."
Mrs May was facing calls to be more courageous in bringing about change and diversity to refresh the look of the Government.
After a day of little movement in the top ranks, and many social media mistakes, veteran Tory grandee Sir Nicholas Soames tweeted: "I don't mean to be rude or to be seen to be disloyal but there needs to be a major improvement to the Reshuffle tomorrow #doitwell."
Former Tory chairman Grant Shapps, who was accused of trying to oust Mrs May after last June's disastrous election for the Conservatives, told BBC Newsnight: "Clearly, to be blunt, it wasn't a brilliantly executed performance with the reshuffle today."
There were few new faces around the Cabinet table on Tuesday, as the "big four" of Chancellor Philip Hammond, Foreign Secretary Boris Johnson, Home Secretary Amber Rudd and Brexit Secretary David Davis all remained in place.
So too did Transport Secretary Chris Grayling, Business Secretary Greg Clark and Commons Leader Andrea Leadsom, despite widespread speculation that Mrs May would demote them.
Ms Greening, who could now become a backbench Brexit thorn in the Prime Minister's side, was succeeded as Education Secretary by Damian Hinds.
The job Ms Greening turned down, Work and Pensions Secretary, was given instead to Esther McVey, who triggered controversy when she was a junior minister in the department under David Cameron.
The big winner of the shake-up was former justice secretary David Lidington, who replaced Damian Green as Minister for the Cabinet Office, but was not awarded the title of First Secretary of State enjoyed by his predecessor.
However, Mr Lidington will fill in for Mrs May at Prime Minister's Questions and take on some of the responsibilities for chairing influential Cabinet committees, including some relating to Brexit.