[2412.08147] Variational Mannequin Merging for Pareto Entrance Estimation in Multitask Finetuning

View a PDF of the paper titled Variational Mannequin Merging for Pareto Entrance Estimation in Multitask Finetuning, by Hugo Monz’on Maldonado and 4 different authors

View PDF
HTML (experimental)

Summary:Pareto fronts are helpful to search out good task-mixing methods for multitask finetuning, however they’re additionally expensive to compute. To scale back prices, latest works have used current mannequin merging strategies to assist prepare low-cost surrogate fashions to estimate the Pareto fronts. Nevertheless, no work has but thought-about designing new model-merging strategies to instantly, and provably, enhance the standard of Pareto fronts. Right here, we fill this hole by proposing a brand new Bayesian strategy known as Variational Mannequin Merging. On this strategy, current model-merging strategies are obtained as particular circumstances of “posterior-merging” when Gaussian posteriors are used and new model-merging methods might be derived through the use of non-Gaussian posteriors. Our most important theoretical result’s to point out that extra versatile posteriors essentially yield higher estimates of Pareto fronts. As an illustration, a Pareto entrance estimate obtained by merging full-Gaussian posteriors is anticipated to be higher than that obtained through the use of isotropic Gaussian posteriors. We validate the speculation via in depth empirical outcomes on imaginative and prescient and language transformers the place higher Gaussian households persistently yields higher or comparable Pareto fronts. Our work is a uncommon occasion the place Bayesian concepts are used to enhance Pareto evaluation.