Secrets to Robust Offline First