If I get a 'not today', (very rare) I just say 'OK no problem' and will see them next time. I almost never get the same thing from the same customer twice in a row. If I do, then I would ask them if they would like to let me know when their circumstances change.
I only dump customers who continually mess me around over payment. When I finally have had enough, I just walk away without telling them I won't be back. The way I see it is that I have done what I was paid to do and the payment wasn't forthcoming. I don't feel the need to explain myself for that.
I don't believe it undermines our reputation. If word did get about, it would come out that it is the customer's reputation which would be more damaged, especially if their neighbours got to know about it. I have never had any negative reaction from customers whose neighbour I have dropped.
John