that sounds like a good explanation of what's happening. However I would expect this happening with UFS and not ZFS. UFS partitions are mounted during system startup according to /etc/fstab. It's very much possible that the installer puts da1 in there and it should be da0 if the USB drive is not attached.
ZFS works per pool and it should not make any difference on which drive a pool exists since the system will look on all available devices for importable pools.
I once had an error like the one that you have, though. However in my case it was a mixed system with a ZFS pool and a gmirror and the error was some side-effect of me wanting to fix a problem with the latter. I was a bit under pressure and did not take notes. So unfortunately I don't remember what the exact error message was nor what I did in the end.
Since I don't have any hardware matching your scenario I cannot recreate it. But there's quite a bit of material about problems like that on the net. If you manage to solve it, please report back!