Sam had left for a moment, but only because he needed to use the bathroom as well. Which meant he left Dean on the door, though he's back before Castiel opens the door to the bathroom.

When the door opens, Sam turns and smiles, offering his arm to Castiel. "All right. Come on, let's get you to bed. You could probably use some sleep."