People aren't annoyed that a character is being played by a black actor, they're annoyed specifically that snape is being played by a black actor.
The character of snape is immediately deemed suspicious to harry and his friends, and he was bullied by James and sirius.. Now both of those plotpoints will have racist undertones to them.
James and sirius and popular characters, and nobody wants to seem them made into racist rich schoolboys
I mean, people wouldn't have cared if sirius black was made black, but they had to do it with the one character who is mistrusted and bullied by the other white characters