Originally built as a convictoria (a type of Catholic theology school) for young noblemen by Bishop Kristóf Migazzi who named it Theresianum in the honor of Maria Theresa. From 1808 it housed the Ludovika Academy and it has been used as a prison since 1855. The old building was later renovated and expanded, and a Neo-Gothic prison chapel was built in the courtyard.

Interestingly, the name of the original Catholic school (convictus) and the word convict (as for someone sent to prison) shares the same etymological roots.