I understand your point: one day, I was jobless and a company called to offered me a job that was too far from my home (in France). So I refused.
But the company didn’t like my answer, and told Pôle Emploi (french employment center) that i was “cold and rude” on the phone.
They were wrong: I was just shy and not interested. But I almost lost my unemployment benefit because of that (you can loose them in France if you refuse 2 or 3 “fair” job offers).
So yeah, I would be angry too. People confuse being over-enthusiastic with “being polite”.
This is annoying.