In a timely coincidence — following yesterday’s discussion — someone just asked that very question to Msgr. Charles Pope, who offered this answer in OSV:
The prayer to St. Michael is not part of the Mass. It is recited after Mass has finished, though usually before the priest leaves the sanctuary. Nevertheless, the Mass is concluded and there is no requirement for anyone to stay once the priest or deacon has given the dismissal. Having certain prayers and devotions after Mass, such as the Rosary, is common in many parishes. However, in those cases, the prayers begin after the priest leaves the sanctuary. But the priest himself often leads the prayer to St. Michael before leaving, and hence your question has significant merit.
Technically, the Roman rite makes no provision for the recitation of prayers after the dismissal. The General Instruction of the Roman Missal states: “The concluding rites consist of: a) Brief announcements, if they are necessary; b) The priest’s greeting and blessing, which on certain days and occasions is enriched and expressed in the prayer over the People or another more solemn formula; c) The dismissal of the people by the deacon or the priest, so that each may go out to do good works, praising and blessing God; d) The kissing of the altar by the priest and the deacon, followed by a profound bow to the altar by the priest, the deacon, and the other ministers” (No. 90).
While it is arguably problematic that the prayer to St. Michael is added after the words of dismissal, one should note that there is also no mention of a hymn that is to be sung as the priest and others leave the sanctuary. But this is a very widespread custom in parishes, especially on Sundays. No recessional hymn has ever been prescribed in the Roman rite. It is merely a custom about which the General Instruction is silent. It is neither forbidden nor prescribed.
For a liturgical purist, however, additions or accretions are to be avoided.
And I’ll just add what my friend Deacon Bill Ditewig suggested:
Back in the early days of the post-conciliar reforms, the Leonine prayers were perceived — rightly or wrongly — as part of the Mass. They were removed simply to help people understand that they were not. What I would recommend is this: after Mass, change out of Mass vestments. and then go back out and pray whatever prayers you like, including to St Michael.