There have been posts before by guys who said they wouldn't see a lady who sees TS clients, or even a lady who has seen clients who see TS providers. There are some extreme fetishes which some guys would be completely turned off by finding out if you participate in with other clients (ie 'scat')
However, many guys wouldn't care what you do with others, just what you do with them. Also for every guy you scare away, you probably pick up one on the other side. One guy's turn off is another guy's turn on.