When in a party. and someone q's for a map / door. and the You're party is waiting for you happens.. If the person walks out of the circle everyone is instantly taken in. Even if they're not in the circle.
I'm in the Circle in this photo. as soon as I walked out Matilda and I Instantly were pulled inside.. We tested it 4 more times. It happened 3 of the 4x. We also had this happen with a Avernus Weekly quest.. 3 party, 2 of us were not in circle.. 1 left the circle and we all pulled in.