Unless you've got professional training, I doubt anyone is going to jump up like Rambo and start defending themselves (Like they think they would).. You'll never know until you're faced with that situation, but to think anyone that's only poked holes in pop cans has any sort of tactical ability is silly. Look at police officers, how many hours of hands on training do they get in real life scenarios and THEY still freeze/screwup/whatever when faced with the real thing.
Personally if the only way I could feel safe was with a gun at my side, I'd move. I can't imagine living life like that to be honest.