"Under the apostles there was great simplicity in administering the Lord’s Supper. Their immediate successors made some additions to the dignity of the ordinance, which are not to be disapproved. Afterwards came foolish imitators, who, by ever and anon patching various fragments together, have left us those sacerdotal vestments which we see in the mass, those altar ornaments, those gesticulations, and whole farrago of useless observances." – John Calvin
Christ and His Apostles did not wear any sort of special garments in the discharge of their ministerial duties, neither did the Elders and Deacons of the early church. For a long time after the church began the shift towards Episcopacy, all evidence indicates that the Christian clergy simply wore the normal attire of the populace. As even the Catholic Encyclopedia acknowledges: "In that period the priestly dress did not yet differ from the secular costume in form and ornament. The dress of daily life was worn at the offices of the Church".The period when this began to change was around the time of Constantine (324 AD). At that time, for a number of reasons distinctive liturgical garments began to be adopted.
There is no biblical precept for a minister to wear special garments in the discharge of his office. The Reformers in clearing away non-biblical accretions in the worship of God also eliminated the wearing of special vestments from worship.
The only rule that governed their and our attire was later encapsulated in WCF 1.6 "there are some circumstances concerning the worship of God, and government of the church, common to human actions and societies, which are to be ordered by the light of nature, and Christian prudence, according to the general rules of the Word, which are always to be observed." In other words, if you are to conduct the worship of the church let all things be done decently and in order, and this includes the attire of the ministers. Therefore, we believe that Ministers of the Gospel should dress in good and decent, but ordinary attire, for that was the practice of the Apostolic church, and should be ours as well.
In answer to your original question. Calvin eschewed all the ceremonial vestments of the Roman Clergy and chose instead to wear what were for him his normal "work clothes" into the pulpit. Calvin was a teacher of theology, and therefore he wore the robes of that occupation into the pulpit, as did most of the Reformers. Although those robes are largely ceremonial for us, we need to remember that at one time that was the ordinary garb of teacher and indicated his trade as clearly as a blacksmith's garb would reveal his trade. The very act of wearing one's ordinary clothes into the pulpit was a shock for many of the people, but was in keeping with the Reformers committment to removing all the ceremonial accretions and restoring the biblical simplicity of the gospel ministry.
As John Owen put it regarding the duties of ministers: "Herein, then, alone, and not in bowing, cringing, and vestments, lies the glory and beauty of these administrations, namely, that they are compliant with and andexpressive of the institution of Christ nor is any thing done in them but in express obedience unto his authority. “I have received of the Lord that which also I delivered unto you,” saith the apostle in this case, 1 Corinthians 11:23."
It all depends on the reason for wearing special garments. If the reasons are based on the Old Testament priesthood, then there is, within that setup, a denial of the priesthood of all believers. The garments become the means for separating the "clergy" and the "laity" which is not biblical and not honouring to God. It usually also includes a salvation by works or by sacraments, which is also wrong.
Obviously, Luther had great difficulty with shaking off that which he had become used to. When you habitually wear a robe or special vestments, you do not cast them off so easily. But it can be done.
But the wearing of special garments by ministers while leading the worship services is totally irrelevant in terms of salvation and gaining entrance into Heaven. And the kind of worship service that is conducted whether it is liturgical or non liturgical is totally irrelevant in terms of salvation and gaining entrance into Heaven.
Luther kept the use of special garments for Lutheran ministers to wear while leading worship services in the Lutheran Church and Luther had received Christ as his Savior and yet Luther retained in the Lutheran Church the use of special garments for minister to wear and Luther retained the basic outline of the Mass in the Lutheran Church and Luther retained a lot of RCC doctrines in the Lutheran Church. There is nothing unscriptural about ministers wearing special garments to lead the worship service. The wearing of special garments by ministers leading the worship service is totally irrelevant in terms of salvation and gaining entrance into Heaven.