The Twentieth Century Society has included Shropshire Council's Shrewsbury headquarters on a list of ten 'at risk' buildings for 2021.
The building was opened by the Queen in 1967 and designed by County Architect Ralph Crowe, but the council has already voted to abandon it and move into Shrewsbury's town centre.
It has cited the prohibitive costs of refurbishing the site, and instead plans to sell it for redevelopment.
Local campaigners from the Save Our Shirehall Group (SOS) failed earlier this year with a bid to block the council's application for immunity from listing.
Had the building been listed it would have placed serious hurdles in the place of any potential sale for redevelopment.
The Twentieth Century Society's Director Catherine Croft said they hoped that any developers would look to work with the building, rather than demolish it.
She said: "We feel it is a really elegant set of buildings and one that is crying out for re-use, both to preserve the architectural and cultural history or the site."
Ms Croft said that it would be "incredibly wasteful and short sited" to allow the building to be demolished.
She added: "We feel very strongly that it should be a listed building and that it is a real pity it has not been recognised in that way, and that reflects an under-appreciation of buildings of that period and particularly an under-appreciation of post-war buildings that are not aggressively avant garde and are civilised and well constructed.
"They are not being confrontational, they are trying, in this instance, to provide a really pleasant working environment."
Ms Croft said that if the redevelopment does not go ahead over the next five years then they would look at asking for the listed status to be reassessed.
In January the council voted to leave Shirehall, with a plan to relocate to the Pride Hill Shopping Centre, in Shrewsbury's town centre by March 31, 2023 – at a cost of £12.5 million.
It says the project will be paid for by the sale of Shirehall and a £5m grant from the Marches Local Enterprise Partnership.
The council already owns the shopping centre having purchased the site as part of a much criticised £51m deal in 2018.
Announcing the decision earlier this year Steve Charmley, Shropshire Council’s cabinet member for assets, economic growth and regeneration, said the move would provide a boost to the town centre.
He said: "This is a very important milestone for the council and the future of Shrewsbury town centre. It shows how we can create a new civic centre in a way that would be entirely self-funding.
“More importantly this can provide a huge boost for the regeneration of Shrewsbury town centre, bring a major employer right into the heart of the town centre and create the right conditions to help the town thrive as a place that’s great to be in – whether for shopping, leisure, business or as a resident.
“A new civic centre included in the wider development plans for the town centre will be used by the council and partners who seek similar benefits in terms of location, cost, carbon efficiency and more modern working environments. This in turn generates additional footfall and business for Shrewsbury town centre, helping to boost the economy of Shrewsbury and Shropshire.
“As we recover from the pandemic this will be more vital than ever.”
The SOS group has however called on the council to rethink its plans, arguing the building should be "treasured".
A statement from the group said: "We appreciate that modern architecture is 'Marmite' to some tastes but the Shirehall, besides being Shropshire's civic centre, is a very fine example of post war modern architecture and should be protected by listing by Historic England."
The statement added: "Architectural styles come and go and in future years the Shirehall could be seen as a much-treasured example of late twentieth century architecture."