This often happens when your brush is dirty.
If you've done a first clean, or a particularly dirty building, it's necessary to wash your brush to get rid of all the dirt. Also needed when finished doing a spider web house (we all know them!)
My preferred method is to simply run it under flowing water (eg the port on your van) while brushing the bristles through with a washing up brush like this:
Use the bristles that point outward at the front to slide in between the bristles of the brush.
You don't need to spend ages at it, 2 minutes will make it as good as new.