Because words are contextual and relative to the speaker and the listener. For instance, if I call a black friend of mine 'my nigga' is he going to be offended? No, it is between friends and understood as much. The word is different when used in different social scenarios, with different people, different inflections, and different intent. On the other hand if I said 'You're a useless fucking nigger'; even to my friend it is obviously completely different.
This wasn't a racist incident; it did contain a slur but the slur wasn't used with racist intent. As you know words have several meanings; the word 'yid' has two, a derogatory term for Jews and term the Tottenham supporters refer to themselves as. 'Alonso notched a brace against the Yids' (context: football discussion) is a different statement that 'gas the yids' (context: violent racial slur; holocaust implications).
Mate, you're a gweilo. Don't know what that means? Well, it is actually a Cantonese slur for white people. Are you offended? If you are, that is only because I told you it is a slur; otherwise the word has no context for you and is rendered meaningless because you don't have anything basis to form offence from. If we are to use the Post-Modern style of deconstruction, you would realise that the only reason we are offended by words is because we are taught to be offended by them.