Sounds like a crazy person. Firstly tell your friends and family about a crazy stalker one nite stand you had. Or ex boyfriend or whatever. They should believe you over a crazy stranger or they arent friends.
As far as getting him to go away, once i had warned my family i would ignore him. If that doesnt work hint but dont outright say that you are connected and thus protected. Who cares if you lie and say mob backing, or giant ganster pimp protection, or hell even that you recently turned stooly for LE. Choose whichever seems scariest to you. Even if you have to say the protection is new to you because he wont stop scaring you. Good luck.