Teenagers were seen using 'Henry', who has been there since May, as a punchbag.
Bridgnorth Town Council says police have been told about the criminal damage and descriptions of the youths have been passed on.
Castle Hall and grounds caretaker, Clive Leadbetter, said he created the scarecrow after this year's annual scarecrow event did not materialise in the town.
"People were saying how sad it was not to have scarecrows so I built Henry," he said.
"There were so many positive comments that the deputy mayor then organised a scarecrow festival herself, with dozens being built in the community."
Henry stood proudly in the castle grounds.
"Then on Tuesday night someone saw two lads in football strips punching his head into pieces," said Mr Leadbetter. He said the witness challenged them and they swore at him.
Mr Leadbetter said the youths were about 14 or 15 years old, one with blond hair the other mousey brown.
"Bridgnorth Town Council has informed the police," he added
The vandalism has been met with anger on social media with about 200 people decrying the behaviour.