It is a challenge. Most importantly you need to make sure to minimise time wasted around whichever ferry trip you choose, not getting there too early etc. There's no 'spare' time on the ride. You can sleep on the ferry. I always advise doing London 3am ish.
