The site was first fortified by the Arabs in the 9th or 10th centuries. By the second Byzantine period, a tower known as Castellum Comunis stood on the site. In 1303, the tower was destroyed in an earthquake but was repaired. In 1462, the Venetian Senate approved a programme to improve the fortifications of Candia. Eventually, the Byzantine tower was demolished in 1523, and the Castello a Mare began to be built instead. The fortress was completed in 1540. In 1630, the fort was armed with 18 cannons on the ground floor, and 25 cannons on the pathway leading to the roof. During the 21-year long Siege of Candia, Ottoman batteries easily neutralized the fort's firepower. The Ottomans eventually took the fort in 1669, after the Venetians surrendered the entire city. They did not make any major alterations to the fort, except for the additions of some battlements and embrasures. They built a small fort known as Little Koules on the landward side, but this was demolished in 1936.