All you have to do is make it clear that you don't want to do the push at the start of the raid, if that's what the others want to do leave and find a team that don't want to. I don't care when in a raid fight or push it can take just the same amount of time sometimes, it really isn't that big of a deal