Recent advancements in natural language processing (NLP) have been driven by pre-trained large language models (LLMs). These LLMs serve as the foundation for the most cutting-edge model across a wide array of NLP tasks. With the availability of vast textual data and improvements in GPU parallelization, these models can now generate text fluently and follow instructions to accomplish a diverse range of tasks.

This course aims to teach the fundamentals of large language models, allowing the attendees to gain hands-on understanding and implementation of them.. The course will begin with the definition of language models and gradually increase in complexity, emphasizing adaptation techniques (e.g., in-context learning, few-shot learning, instruction learning) and methods to align models with human preferences. Additionally, advanced training techniques such as parallelism, selective architectures, and scaling laws will be covered. The course will also address bias and ethical concerns related to these models.

The course is part of the NLP master hosted by the Ixa NLP research group at the HiTZ research center of the University of the Basque Country (UPV/EHU).

Student profile

Addressed to professionals, researchers and students who want to understand and apply deep learning techniques to text. The practical part requires basic programming experience, a university-level course in computer science and experience in Python. Basic math skills (algebra or pre-calculus) are also needed.

Contents