I personally would not kick down a Sand Castle whether nobody is around or not and I also would not want to wait until the tide reaches it.
I would admire it and continue on with my day. :/
-
*Looks left* *Looks right* Kicks in castle *runs away*