I see little point in yelling/giving the finger because even though that is a rightful reaction it won't accomplish much with people that do that, so if they are really a nuisance then moving on is best. Having said that, if he is following you around like that, then I don't know, there isn't really a good solution. Your best bet in terms of providing the best outcome is still probably to try to calmly reason with him, although I would be pretty furious in your situation.