Travellers at Long Lane in Craven Arms have said they have faced "dirty looks" over the last week or so following the outbreak, in which 29 cases of Covid-19 have so far been confirmed.
All who were staying there were asked to self-isolate in the beginning, but now only five are still being asked to remain in quarantine by Shropshire Council.
It was a blow to the community following an announcement that police will be able to seize caravans and arrest travellers who set up camp on private or public land.
James, who did not wish to give his surname, said the move needlessly victimised travellers, and that it's time they united and made a stand similar to the Black Lives Matter movement.
"Black people have done something about discrimination towards them. Travellers still haven't made a stand. Black people have protests, but we haven't done anything like that," he said.
"People need to realise, we're not rubbish. We're all humans. We work hard. That's how it is."
The Black Lives Matter movement gathered huge support following the death of George Floyd in the US, where a police officer was filmed kneeling on his neck. Several protests have taken place across the UK, including in Shrewsbury, Telford and Ludlow.
Last week, Shropshire Council said it had been made aware of people leaving the caravan site who should have been self-isolating and has warned West Mercia Police is now "considering taking appropriate enforcement action".
For the week to Wednesday, Shropshire's infection rate was 6.6 cases per 100,000 people after 21 new cases were confirmed - almost entirely in Craven Arms. This rate is four times higher than the previous week but remains extremely low compared to places such as Blackburn and Leicester where the rate was 83.9 and 58.1 cases per 100,000 people respectively.
In Telford & Wrekin the rate was 2.8 cases per 100,000 people after five positive tests compared to 7.9 the week before when 14 cases were confirmed.
Residents in Craven Arms are being encouraged to get tested whether they show symptoms or not at the testing centre in Craven Arms Business Park, open from 10.30am to 3.30pm.
Appointments can be booked via NHS Test and Trace online or by phoning 119.