Tuned Mass Dampers (TMDs) have been widely used in the passive vibration control of engineering structures. By properly tuning these systems, TMDs can significantly reduce the seismic response of the primary structure, whose dynamics should be controlled. However, when large earthquakes occur, the structure can experience inelastic behaviour, thus implying a detrimental de-tuning effect on the primary system. In order to in-depth investigate this issue, in this paper, non-conventional TMDs with large mass ratios, e.g. Inter-storey Isolation Systems (IISs) are considered and the influence of the inelastic behaviour of primary system on the overall structure is analysed. In detail, IIS is employed for retrofitting existing masonry buildings, through a vertical extension, isolated at its base and built up on the roof of the existing structure. In previous papers, the authors have demonstrated that, by carefully designing the upper structure and the isolation level, the inter-storey isolation system can reduce the seismic response of the existing structure. This result has been achieved by assuming a linear behaviour for both the existing masonry structure and the new upper structure, while a non-linear behaviour was adopted only for the isolation system. However, when severe ground motions occur, the hysteretic behaviour of the masonry should be assumed, by removing the hypothesis on the linear elastic behaviour adopted for the initial design phase. For this aim, pivot-type hysteretic models are used to faithfully grasp the mechanical behaviour of the masonry structure. Based on a case-study building, time-history analyses on 3D FE models are implemented. The results show a reduction of the displacements at the top of the lower structure and a significant decrease of the hysteretic energy dissipated by the masonry piers subjected to a selected set of accelerograms, thus confirming that, although a de-tuning effect can occur, the global ductility demand is reduced by comparing the controlled structure with the as-is configuration.