The abbey was founded in the 7th century by saint Germer, lord of Wardes. A rich man, who decided to withdraw from king Clotaire's court. He started the construction of a monastery and Benedictine monks joined him. After a first reconstruction in the 9th century, the Normans invaded the city and destroyed the abbey. This one was rebuilt in 1036 by bishop of Beauvais, Drogon.
Then, Pierre-Guillaume de Vessencourt put up the Sainte-Chapelle in 1259 (which owns its name from the Sainte-Chapelle in Paris, because it was as like as two peas). Duke of Burgundy besieged the abbey then abbot Jean de Silly restored it in the 14th century. But Saint-Germer started to decline...
In 1644, they tried to found a school for poor people, in the church, in vain... The monastery was sold during the Revolution and demolished, except the Gothic church (12th century) and the chapel.