The Gingerbread Men are eight points adrift of safety at the foot of the Midland Premier after a 16th defeat from 17 games, by way of a 3-0 reverse at Lichfield on Saturday.
Brown has lost all nine of his league games in charge but said upon his appointment that he needed time to arrest the slide at Greenfields after a flurry of change in recent years.
“Wolverhampton Casuals against Worcester was postponed so it would’ve been nice to get something,” Brown said of two other teams battling against the two drop zone places with Drayton and Uttoxeter.
“The teams above the four are starting to pull away which is concerning for the hierarchy.
“But when you bring someone in to come and change things with half the budget you’ve got to give time.
“If they want to get rid of me who do they bring in? Who is going to do it?
“If there was a change the chances are they would change all 20 players again because they don’t know them and bring their own in, so there would be an influx again.”
He added: “I would understand if we are down there with no chance but I’m looking on the positive side of it.”
Brown, whose team welcome second-placed Darlaston on Saturday, added former Walsall junior Ade Francis, a versatile forward, to his ranks before the Lichfield game but the 19-year-old spurned a glorious early chance to mark his debut in style.
Drayton were then caught a few minutes before the break and conceded two in the second period.
AFC Bridgnorth head to the East Midlands for a crunch Midland League clash on Saturday.
The Crown Meadow men face fellow Division One strugglers Ingles looking to snap a four-game losing streak – three of which have come in the league.
Ingles sit one place outside the relegation zone, level on points with Bridgnorth who are one spot higher but have played one more game.
Bridgnorth put in a spirited display on home soil last Saturday against second-placed Ashby Ivanhoe. They had player/boss Jason Pike sent-off midway through the first half but more than matched their high-flying opponents before being undone by a stunning strike late in the game.
That displayed followed an encouraging showing the week before in a 2-1 loss to leaders Dudley Town and secretary Steve Groome believes the side can take heart from their recent displays.
“It was another good performance on Saturday,” said Groome. “At times during the second half you would have thought we had the extra player.
“We were unlucky not get anything from the game but we have to take the positives out of that display and the one against Dudley into Saturday’s game.”
A much-changed Bridgnorth side lost 4-0 to OJM Black Country in the JW Hunt Cup on Tuesday.