The phrase “I’m just joking/having fun” does not excuse racism, misogyny, sexism, or ableism.
It just shows that you know that you’re saying something really shitty and offensive and you don’t want to suffer real repercussions because of your shitty as fuck behavior.