Last week, households in the area were sent letters from Shropshire Council outlining the authority's plans to convert Coton Hill House in Berwick Road from a retirement home into "temporary accommodation for individuals who are homeless".
The plans will see the 45 rooms of the former Coverage Care retirement home turned into 25 self-contained units as well as communal and staffing facilities to help give rough sleepers a new start.
The council first revealed its plans in a consultation in June last year and asked residents for their thoughts on the proposals.
Follow a raft of concerns raised by residents, Shropshire Council said it had "listened to the feedback and taken concerns and ideas onboard".
However, the council says it now intends to seek planning permission for the project, but residents in the area say they have safety fears over the proposals.
One householder in Coton Hill, who didn't want to be named, said: "Given the ongoing and worsening issues in Shrewsbury town centre over the last few years, many residents of Coton Hill have little faith that the council and its partners are capable of delivering a successful project.
"There are also serious questions on the viability and efficacy of this project in Coton Hill, and legitimate concerns around safety and quality of life in the area."
For those wishing to attend the consultation into Coton House, Shropshire Council are holding at at The Gateway Centre in Chester Street on Wednesday afternoon between 3pm and 6pm.
Shropshire Council has been contacted for comment.