Date Apr 3, 2025, 3:00 pm – 4:00 pm Location Online Event Audience Princeton students, graduate students, researchers, faculty, and staff Related link More details in My PrincetonU Details Princeton Hackathon 2024 Event Description This workshop is an overview of how pre-trained LLMs can be customized and applied to specific domains using domain adaptation techniques: custom tokenization and domain adapted continued pre-training with curated domain specific data, and supervised fine tuning with domain specific instructions. NVIDIA will walk through how these techniques were applied to ChipNeMo - an LLM customized for industrial chip design which was then used for building a code generation assistant, a bug summarization and analysis assistant, and an engineering chatbot assistant. Results show that these domain adaptation techniques enable significant LLM performance improvements over general-purpose base models in domain-related downstream tasks without degradation in generic capabilities. References: https://research.nvidia.com/index.php/publication/2023-10_chipnemo-domain-adapted-llms-chip-design https://arxiv.org/abs/2311.00176 Format: Presentation with open Q&A See the full PICSciE/RC spring training program or subscribe to the PICSciE/RC mailing list.
