Work with a reputable agency or whatever and you won't have these problems.

I have only seen providers at home, I like the atmosphere, makes it feel more like a date, etc.  I don't see many providers and only go with highly rated ones so, at least so far, I haven't had any problems.

That said, don't expect providers to be willing to park around the block or meet you somewhere else where you can pick them up.  At least not until they have met you.  They are at least as concerned about their safety as you are about yours.  In my experience most providers would view you wanting to meet them somewhere else with suspicion and probably not agree to it.

I also resent it when a provider is using a driver but not up front about it.  I to live in a quiet, affluent, 24-hour patrolled, neighborhood.  Any foreign cars in the area will get the attention of security and I don't need that.  I understand that in the minds of the provider the driver affords them some protection, but I want our time together to be our little secret and the fact a driver knows about it makes me uncomfortable.  So I guess my point is find out up front if the provider uses the services of a driver and make your decision accordingly.

to deal with. Not everyone is so lucky.
In some gated monolithic neoNazi communities, stereotypical suburbanite neighbors sue neighbors over such nonesense as backporch furniture color. Only in the mentally emptier part of America, called suburbs.
Many of my pals are terrorized by bosses who think they own them 24 hours a day.
My only concern is robbery or burglarly by the provider or any acquaintance of hers. My neighbors have real lives and could care less. They have complimented the visuals of visiting vixens from time to time. I do have pepper spray handy.
A provider boyfriend once was trying to break in when I was out of town. My large building contractor happened by, messed up his face, and knee-capped him with a large brick. What are large friends for. LOL