The island is one hour by boat from the nearest land and experts said it was too risky to house the refugees on the island as it is prone to flooding during storm surges