I'd suggest that there's more than one way in which racism manifests. One is the specific racist who explicitly hates, as it might be, Jews (or Blacks, or Muslims, or whoever) and says so directly. But another is the case where racist attitudes have so saturated a particular society and culture that members of that world simply don't question them, endorse and replicate them without thought. This latter state of affairs is, surely, worse than the former: when a loudmouth racist is sounding off, we can disagree and isolate him. But when a large tranche of society believes (as it might be) Black people are below-par intelligence, Jews are dishonest money-grubbers, Tutsi are evil etc, then there's a genuine danger of societal-level action and violence. Carroll, it seems to me, belongs to this latter world.