It sounds like you plan to do pretty big legs. In a 40' yacht I would be tempted to leave a little before high tide and head straight down to Dale. Even at 40' you would struggle to make it to Oxwich in one tide, so I would just put up with it and carry on. Obviously a lot depends on the wind and how much you would like to motor.
Unless you want to stay a while in Milford, Dale is the best place to stop in my opinion. Either at anchor or on the pontoon in the bay.
From Dale it's around St Davids with the tide and up into your home patch.
If you need any crew give me a shout, it sounds like fun.
Chausey is now sold, I'm off for a sail!