This has happened to me, on occasion. While I enjoy chatting and exchanges between visits, and have actually made some very close friends in the process, I understand. There are 2 possibilities:
1. There are select gentlemen who require a certain chemistry and connection with their companion. They like to engage before meeting, and enjoy a more complete experience. That is a god thing.
But...he could be:
2. He's a time waster who is looking for attention. While there are some who prefer to connect prior to meeting, which is perfectly understandable and can add to the experience, there are certain guys who enjoy simply messing with you. They have the idea that you're supposed to accommodate them constantly, and never schedule or follow through with arrangements. I've even heard from some who claim, "I had a girlfriend who was an escort, so I know the business," or "I've often helped escorts out of a jam," and so on.. It's an ego boost and a ploy for attention.
If he's not genuine and you're certain he's the 1st option, I remedy this with responding, "What date and time would you like?" If he's a chronic cancellation, you know how to handle that one! You handled this perfectly. He responded that you are "too much about business," because he realized you're not falling for his apparent ploy.
It's fun to connect with people. But if he's a waste of your time, cut him off - as you did.