Clare Castle was built between the River Stour and the Chilton Stream and took the form of a motte and bailey design, with two baileys rather than the more common one. The motte is 850 feet (259 m) wide at the base and 100 feet (30 m) tall, with its flat summit 63 feet (19 m) across. The two baileys, stretching alongside from the motte along the north and east, were protected by deep ditches and steep palisades, with either a causeway or a drawbridge linking the inner to the outer bailey; the inner bailey was also sheltered by the curve of the Chilton.

The castle was built on the site of a former Anglo-Saxon manor house, probably reflective of the wider Norman effort to demonstrate that their authority had replaced those of the previous lords. Castles such as this were termed a caput. The castle was surrounded by three parks, including the Great Park at Hundon, established by 1090. Like many other major castles, Clare was twinned with a local religious house when Richard de Clare founded Clare Priory in 1249, close to the castle, and grew to contain 29 friars.

A new keep was built, probably in the 13th century. This took the form of a polygonal shell keep, with fourteen triangular buttresses supporting six foot (1.8 m) thick walls. The inner bailey was strengthened with new stone walls, 20 to 30 feet (6 to 9 m) tall on top of the earlier earth banks, the walls and keep being built of flint and rubble. The castle was manned during this period by a castle-guard system, in which lands were given out to local lords in return for their providing knights and soldiers to serve at the castle.

The de Clares continued to own estates across England, but in Suffolk their possessions were concentrated in estates clustered around Clare Castle. The castle was passed through the de Clare line until Gilbert de Clare died at the battle of Bannockburn in 1314, when the estate passed onto his sisters. Elizabeth de Clare, whose husband John de Burgh had died the previous year, acquired the castle. She used the castle as her main residence between 1322 and 1360.

The castle was well developed by this time, and was reached through three gates positioned across the wider estate, called Nethergate, Redgate and Dernegate. The castle itself had four stone towers protecting the entrance to the inner bailey and the keep, called Auditorstower, Maidenstower, Constabletower and Oxfordtower. Elizabeth built a chamber for her own use at the castle between 1346-7. A substantial water-garden, forming part of the moat to the east, existed at the castle during this period; it may have included a fountain and probably had a geometric form, possibly similar to the garden at nearby St Benet's Abbey in Norfolk. Vineyards and orchards surrounded the property. The castle's three parks continued in active use, and, as part of the local rearing programme, the local deer were moved between them as the animals grew older.

After Elizabeth de Clare's death, Clare Castle passed by marriage into the Mortimers of Wigmore. When Sir Edmund Mortimer acquired the castle in 1405, contemporaries reported it "in good repair and stocked". The subsequent years saw the Mortimers heavily involved in the Wars of the Roses; after Edmund's death in 1425, the castle passed to Richard of York and in turn, via his son Edward IV, to the Crown.

Edward VI gave the castle to Sir John Checke, until it reverted to Mary . After Mary, the castle was acquired by Sir Gervase Elwes, whose family retained it until the 19th century. At some point after 1720, the surviving east and south sides of the inner bailey walls were destroyed.

In 1867 the Cambridge and Colchester branch line of the Great Eastern Railway was built through the castle, cutting across and largely destroying the inner bailey in order to make room for a new station. The railway line was later closed in 1967 as part of the Beeching Axe.

The castle now consists of a motte, on which a part of the keep still stands and the outer bailey earthworks; fragments of the inner bailey stone wall can also still be seen. The disused station, goods yard and the castle grounds have been developed into a landscaped country park, interlaced with water in the old moats, called Clare Castle Country Park, which is crossed by the Stour Valley Path. The park was opened in June 1972, after 15 acres (6.1 ha) were donated by Anthony de Fontblanque and 4.5 acres (1.8 ha) of railway land bought for £10,000.

In 2014, the keep and curtain wall were extensively consolidated with the assistance of English Heritage. In March 2015, the stewardship of the park transferred from Suffolk County Council to the Clare Town Council; a trust manages the park with the support of local volunteers. The castle is protected under UK law as a scheduled monument and a grade 2* listed building.