When the sun was standing low over the hills I wondered: Maybe I always start so late, because I know there's no way to bail out?

Is there a difference between touring and racing? Of course. But the difference is actually a lot smaller than I thought.

Why are we so attached to the idea, that winning by being the fastest is everything it could ever possibly be about?

Going back to where you started doesn't always mean to literally cycle to a certain point. It's a state of mind, not a place on the road.