Does it mean Your being Racist Agianst your own Kind?

When a Dark Skin Man only wants to be with light skin women, Can you jump to the conclusion that he has a problem with his own skin tone or is racist against his own kind?