May, who is currently on loan with the Essex side, scored the winner with just two minutes remaining of extra time to cause the biggest upset of the round.
The result now places extraordinary pressure on Baggies manager Gary Megson, who has yet to see his side win this season.