Having captured and demolished the previous Norman earth and timber fortress, The Lord Rhys moved his court here in 1171. He rebuilt the castle in stone, making it the first stone castle ever built by a Welshman. He then celebrated his achievement by hosting the first ever Eisteddfod in 1176. After falling into serious disrepair, the castle, and the Georgian mansion built within the castle walls in 1808, were restored and reopened to the public in 2015.