The soap you mentioned isn't gonna do squat, in the long run. The problem is ingrown hairs, not zits. Body hair is finer than facial hair, and body skin is more tender than facial skin, generally, unless you've been a full-time nudist your entire life. When you shave body hair close to the skin, it can curl "in", and if the skin is delicate, the prospensity toward ingrown hairs in increased. That's what you've got.
Arms and legs, which are much more exposed, don't bump as easily. Butts and pubes, which are covered most of the time, unless, of course, your Faye Desiree, it seems,
, are more sensitive and easily irritated.
Benzoil peroxide will clear it up, eventually, only because it will eventually dry out the skin, which is then tough enough and allows the hair to grow out. But you're not gonna solve the problem, becuase next time you shave you'll get bumps, and eventualy the skin will get used to the BZ, and the law of diminishing returns applies. So what you're doing really isn't making it better, it's just shortening the time to clear, and temporarily.
How do I know this? I am one hairy MOFO, and when I started weight training, I began to shave down to accentuaute my new body cuts. But, I got serious bumps on my fanny, which tended to get even more irritated because I regularly freeride mountain bike, and bicycle seats are bad enough. I went through the same issues as you, tried some stuff until I met, through work, a former gay porn entertainer turned counselor/psycologist to gay adult industry workers. Somehow the conversation turned mainstream gay beauty images that prefer "hairless" and since most all men I know are hairy, but many gay men are not, I asked the dumb question -- "Are gay men just hairless by nature." No, no more than the general male population, but many gay men shave down, and that's where I learned about Tend Skin and Visine. I've heard about Bactine and Cotizone cream as well.
In short, either talk to a cut, hairless gay porn star (a rudundancy?) or listen to the women. And get over your psuedo-macho fear of visiting the cosmetics section. Listen, just by participating on this board, you are no longer in Kansas.
Ciao