Is it not racist to not want to live around blacks?

Nope, it's smart if you want to keep your valuables, not have your daughter raped and your car to remain in the driveway.