Introduction to Large Language Models

AIM

This course gives a full introduction to Large Language Models (LLMs). It covers the whole process, from basic ideas to real applications and deployment. Students will learn how transformer models work, how pre-training and fine-tuning are done, and how to build and deploy applications using LLMs. The course mixes theory with practical exercises, so students can get both understanding and hands-on skills. By the end, students will be prepared to work with and understand the fast-growing field of generative AI. As a semester project, students will either design, implement, and deploy an LLM-based application that addresses a real-world problem or conduct a research study on a current topic related to Large Language Models.

CONTENT

This course contains; Introduction to NLP and Language Models,Tokenization, Word Embeddings and Representational Learning,LLMs Basics,Attention Mechanism in Transformers,The Transformer Architecture,Transformer-Based Architectures and Models,LLM Training,Midterm Week,LLM Inferencing, Prompting, ICL, RAG, FT, Prompt Engineering,Information Retrieval and Retrieval Augmented Generation - RAG,Parameter Efficient Fine Tuning, LMM Model Compression,Agentic AI and Agentic AI Frameworks, Model Context Protocol (MCP),Multi-Modal LLMs,Review and Presentations.

1. 1

   By the end of the course, students will be able to: 1. Explain the architecture and training pipeline of modern LLMs. 2. Implement core components of Transformer-based language models.3. Fine-tune and evaluate pretrained LLMs for downstream tasks.4. Understand alignment, instruction tuning, and human-feedback pipelines.5. Analyze efficiency, scaling, and deployment constraints.5. Students may develop and deploy LLM-based applications, critically review and reproduce recent research on Large Language Models (LLMs), or conduct original research on novel LLM architectures, applications, prompting techniques, fine-tuning strategies, evaluation methods, safety, or efficiency improvements.

WEEKLY PLAN

1. WEEK 1

   Introduction to NLP and Language Models

2. WEEK 2

   Tokenization, Word Embeddings and Representational Learning

3. WEEK 3

   LLMs Basics

4. WEEK 4

   Attention Mechanism in Transformers

5. WEEK 5

   The Transformer Architecture

6. WEEK 6

   Transformer-Based Architectures and Models

7. WEEK 7

   LLM Training

8. WEEK 8

   Midterm Week

9. WEEK 9

   LLM Inferencing, Prompting, ICL, RAG, FT, Prompt Engineering

10. WEEK 10

    Information Retrieval and Retrieval Augmented Generation - RAG

11. WEEK 11

    Parameter Efficient Fine Tuning, LMM Model Compression

12. WEEK 12

    Agentic AI and Agentic AI Frameworks

13. WEEK 13

    Model Context Protocol (MCP)

14. WEEK 14

    Multi-Modal LLMs

15. WEEK 15

    Review and Presentations

ASSESSMENT

• Rate of Midterm Exam to Success30%
• Rate of Final Exam to Success70%

READING

• - Jurafsky, D., & Martin, J. H. (2025). Speech and Language Processing (3rd ed., draft). Stanford University. Available at: http://web.stanford.edu/~jurafsky/slp3/ - Hands-On Large Language Models by Jay Alammar, Maarten Grootendorst (2024) ( https://github.com/HandsOnLLM/Hands-On-Large-Language-Models ) - LLM Engineer's Handbook by Paul Iusztin, Maxime Labonne (2024) ( https://github.com/PacktPublishing/LLM-Engineers-Handbook ) - Build a Large Language Model (From Scratch) ( https://github.com/rasbt/LLMs-from-scratch ) - Hands-On Machine Learning with Scikit-Learn and TensorFlow (https://github.com/ageron/handson-ml3) - Deep Learning for Coders with fastai and PyTorch (https://github.com/fastai/fastbook) - HuggingFace NLP Course (html) - Practical Natural Language Processing (2020) (https://github.com/practical-nlp) - Natural Language Processing with Transformers, Revised Edition (2022) (https://github.com/nlp-with-transformers )

TEACHING STAFF

• Prof.Dr. Selim AKYOKUŞCOORDINATOR
• Prof.Dr. Selim AKYOKUŞ