I have had bees occupy a trap only to leave it a short time later. At times they have stayed as long as a couple of days. This caused me a lot of frustration last year as you can see from this video. It was mainly due to a lack of patience and a healthy dose of ignorance. It forced me to start classifying HITs and catches.
While making the rounds checking traps I always get excited when I see bees coming and going. However there are differences that are observable. Up until the point that I see pollen entering I refer to an occupied box as a HIT. This means that there are bees present, but I don’t start counting any chickens yet.
It has served me well to wait until I see pollen entering the trap to count it as a catch. At that time there is a pretty good chance that there is brood or will bee soon enough that the bees feel the need to gather it. You won’t have brood without a queen. I have only been doing this for two years, but I have not had a laying worker in a swarm trap yet. Beekeeping for All by Abbe Emile Warre teaches the importance of external observation in beekeeping. I happened to be reading from that book about the time the video was shot last year.
Pollen entering your trap is also an important factor in deciding when to move the trap. Remove it before there is brood and the bees may abscond. They have little reason to stay. They are way less likely to leave if there are babies on board. If you leave the trap in the tree too long it will be so heavy that you may have trouble handling it.
So don’t jump the gun, but don’t forget about the thing either.