...and I'm assuming you're not visiting a lady at 4AM.  Hotels have people going in and out all the time so I doubt anyone would ask if the hour is before 11 PM, at the earliest.  If asked, you're meeting a guest of the hotel.  If the doorman has to announce you, tell him the name and room number.  After all, she IS a registered guest, right?

As the previous poster pointed out, if you are not visiting an an extremely late hour or a very early one, you should not get noticed if you behave as you know where you are going.
    There are some hotels that will not allow guests to access the rooms without permission from the guest that is to be seen but providers probaly would avoid such hotels, if they are aware of the policy.
    If you use common sense (i.e. have a sense of direction and do not get lost and start wondering around), you should not have any problems.

Some hotels look all the doors, including the front door after 11:00 pm. Your best bet to meet the provider in a bar and have her give you a room key so you can get in the back door.

Better yet you get the hotel room.

Well you got some dam good advice from the guys that responded, I suggest you follow it. I travel all over the world and have never had that problem.