The Tea Rooms  occupy a section of the grade II listed building, which is shared with the Heritage Centre, housed in previously private sections of the house. It is thought the building was the house of a wealthy merchant in the 16th century, with the only remaining Medieval elements of the building being the thick stone walls and the half-timbered dividing wall in the Heritage Centre. The building was later modified to create a second storey and refinements such as the fireplace and elaborate ceiling features demonstrate the merchant’s wealth. Remnants of the fireplace in the tea rooms can still be seen, with the sides and lintel made from a robust stone. This was later crudely cut through to create access to the room behind, and further damaged to create space for Colyton’s first phone exchange around 1930.