whenever i get to a house and there are builders there i knock on and enquire whether they want me to go ahead and clean or just miss them out.
sounds to me like the OP just turned up and started cleaning without seeing the customer first or didnt use common sense and skip it this time.
some of my customers will phone me up prior to clean to tell me to skip but most dont as they have other things on their mind than the window cleaner esp if your house resembles a bombed out building!
if builders had cement mixer going and all the crap that goes with building then there is no way id start cleaning their windows!