Title: Federated Learning in the Generative AI Era — Seminar
Theme framing (subheader): Presented as part of TDAI’s Yearlong Theme: The Geometrization of AI
Date & Time: Friday, November 21, 2025 · 10:30–11:30 AM (ET)
Location: Pomerene Hall Room 350 (Project Zone)
Audience/Format: Research seminar · Open to all · No registration required
Dr. Gauri Joshi (Carnegie Mellon University) discusses how federated learning can fine-tune large language models using data on edge devices, highlighting adaptive multi-head LoRA and model-merging strategies for privacy-preserving collaboration.
Abstract
Large language models (LLMs) have not yet effectively leveraged the vast amounts of data available on edge devices. Federated learning (FL) offers a promising way to collaboratively fine-tune LLMs without transferring private edge data to the cloud. To work within the computation and communication constraints of edge devices, recent research on federated fine-tuning of LLMs uses low-rank adaptation (LoRA) and similar parameter-efficient methods. LoRA-based methods suffer from accuracy loss in FL settings, primarily due to data and computational differences across clients. In this talk, I will first discuss an adaptive multi-head LoRA method that balances parameter efficiency and model expressivity by reparameterizing weight updates as the sum of multiple LoRA heads. Another way to leverage edge data is to train models locally at each client and merge them using a one-shot transfer to the aggregating cloud server. In the second part of my talk, I will discuss model merging from a federated learning perspective.
Speaker
Gauri Joshi is an associate professor in the ECE department at Carnegie Mellon University. She completed her Ph.D. from MIT EECS and received her B.Tech and M.Tech from IIT Bombay. Her awards include the MIT Technology Review 35 under 35 Award, ONR Young Investigator and NSF CAREER Award, Best Paper awards at MobiHoc 2022 and SIGMETRICS 2020, and the Institute Gold Medal of IIT Bombay (2010).