I’m sure you’ve all had the experience of meeting someone for the first time and not liking the person because they say or do something which annoys you. In that situation, you can use today’s idiom: to “rub someone the wrong way”. This is used when talking about people that we don’t like. When giving the reason we don’t like them, we can use this expression. For example:
I don’t like my new co-worker. I don’t know why, but he just rubs me the wrong way.
There’s a new student in my class, and the others don’t like him very much. He really rubbed them the wrong way when he started asking them really personal questions.
Tracy has a very aggressive personality, and she can really rub people the wrong way. I’ve tried to explain that to her, but she doesn’t listen to me.
Ian is always talking about sex, and he can really rub some people the wrong way because of it.
So, with this expression, we can only use it about ourselves or other people, but we don’t use it when speaking directly to someone. Therefore, we DON’T say, “You rub me the wrong way.” It would be very rude to say that, and it also sounds a little unnatural.