There is a 2min period after a lock action (incl. manually unlocking with key) during which the Smart Lock ignores Auto Unlocks and does not perform them (Error 0x03 entries in the activity log). This is to prevent cases you described.
If there is a longer delay the Auto Unlock will be executed like normal, because neither the App nor the Smart Lock can identify whether a person/device is inside or outside of the door.