AIO- My bf thinks I’m “bipolar” and the problem because I told him I’m black and I can say the N word.

For some context: My bf wants to go back to being friends with his racist friend who has disrespect me for YEARS now (and has also disrespect him) because I told him that I’m black and that I can say the N word but it’s wrong when his white friends say it (Also I barely even say the word). Now he’s saying I’m controlling him and that I have unrealistic expectations just because I don’t want him to be friends with someone like that. This has also happened before in the past and I’ve let him go back to being friends with him but I think this just crosses the boundary.