In order for the two guards to leave the post, you must have La Tim's lease and the crystal skull. The guards will appear in the bar at Holy village and you need to talk to them, before you go to Revival Island.