For 250 years after the Norman conquest the population grew and the villagers expanded the cultivated area westward beyond Rainster Rocks and eastward up the slopes below Carsington Pasture. It was during this period of prosperity and expansion that the village church was built, in the late 12th century. Until the 19th century Brassington was a part of Bradbourne parish and the church, dedicated to St James, was a chapelry of the mother church at Bradbourne, serviced by the Bradbourne vicar’s curate.

In the 12th century half of the manor was granted to a De Ferrers heiress and this half, whose ground was scattered among the lands held by the old manor, was run as a freehold estate. The remaining feudal manor was taken from the De Ferrers family in the 13th century and given to a son of the king, the Earl of Lancaster. His successor in the following century, by then Duke of Lancaster, became king as Henry IV in 1399 and the manor remained in royal hands until it was sold by Charles I in 1632. The records of the Duchy manor court from 1300 until 1632 are preserved in the Public Record Office. From the date of the sale in 1632 until the formal abolition in 1925 of the feudal system of land tenure known as copyhold, the manor was held by Derbyshire gentlemen and its records are now in the Derbyshire Record Office.

During the 14th century a succession of calamities reversed the progress made since the Conquest. There was a major climate change to wetter and colder weather, a ruinous cattle disease and finally, in 1358, the Black Death. The effect on the village can be seen in the manor court records, which ceased altogether for several years, and which resumed with most of the family names of the years before the plague gone. The cultivated fields on the west reverted to wasteland and the shortage of labour to cultivate the land caused the Duchy of Lancaster to abandon the rigours of the feudal system. The manor court still ran things but the villagers now held their land by money payment and owed no other dues to the lord. When, under James I, the Duchy tried to reassert its ancient rights, the villagers established that they owed “ne works nor boones nor other duties” to the lord.