It's a bit of both... but I think for us it's more the fact he's black. The taboo, hot nature of it excites us as we both come from very white areas. And in the bedroom... the skin contrast with my girlfriend, his stamina, and ability to thoroughly outperform me in bed drive us both wild.
Yes, those factors are extremely important!