It will offer a more solid build long term, and does away with the insulation which might end up having damp problems, as well as potential vermin, plus its extremely easy to work with.
I've been doing some calculations to compare both materials, using values obtained via the interweb, which might not be accurate at all, but still give a rough impression.
Situation one, Plasterboard.
|Thermal Conductivity (K)||mm||Thermal Resistance (R)|
|Mortar/Concrete Wall||1.3||W/mK||400||0.307692||M² K/W|
|Thermal Transmittance (U)||0.227737||W/M²K|
I think those values are very acceptable for an old building's walls, especially if you compare it against total price.
Oh yes, the polystyrene is for the outside wall, sometime during the summer.