She said a review had found members of the Windrush generation had "suffered terrible injustices".
They included Wolverhampton grandmother Paulette Wilson, who grew up in Telford after moving there from Jamaica aged 10.
She was held in a detention centre and came close to being deported, despite having lived in the UK for 50 years.
Ms Wilson is now among thousands seeking compensation for their ordeals.
The independent inquiry found the Home Office demonstrated “institutional ignorance and thoughtlessness towards the issue of race” and some ministers still “do not accept the full extent of the injustice".
The report’s author, Wendy Williams, an inspector of constabulary, said at its launch: “The Windrush generations has been poorly served by this country, a country to which they contributed so much and in which they had every right to make their lives. The many stories of injustice and hardships are heartbreaking, with jobs lost, lives uprooted and untold damage done to so many individuals and families."
Making a statement to MPs on the Windrush lessons learned review, the Home Secretary said: "As this review makes clear, some members of this generation suffered terrible injustices spurred by institutional failings spanning successive governments over several decades - including ignorance and thoughtlessness towards the race and history of the Windrush generation."
Ms Patel said there was an "ongoing mission" to put this right, adding: "Lives were ruined and families were torn apart, and now an independent review has suggested that the Home Office's institutional ignorance and thoughtlessness to the issue of race and the history of the Windrush generation contributed to this. This is simply unacceptable.
Ms Patel went on: "Despite the diverse and open nature of our country, too many people still feel they may be treated differently because of who they are or where their parents came from.
"Today's report, which suggests in the Home Office there was an institutional ignorance and thoughtlessness to the issue of race and the history of the Windrush generation, is worrying for us all."
Ms Patel said: "We must all look to ourselves, we must all do better at walking in other people's shoes. We must all take responsibility for the failings that led to the unimaginable suffering of this generation.
"And let me be clear, Mr Speaker, there is nothing that I can say today which will undo the pain, the suffering and the misery inflicted upon the Windrush generation.
"What I can do is say that on behalf of this and successive governments, I am truly sorry for the actions that spanned decades and I'm sorry that people's trust has been betrayed.
"We will continue to do everything possible to ensure that the Home Office protects, supports and listens to every single part of the community it serves."
Desmond Jaddoo, chairman of the Windrush Movement UK, who has been helping victims with compensation claims, including Ms Wilson, said the review "confirmed the ignorance of the Home Office".
He added: "This has caused stress and some people have become ill. People have lost their livelihoods and families have broken up over this.
"The Home Office played with the lives of individuals and it's not right."