ADVANCED TOPICS in NATURAL LANGUAGE PROCESSING

AIM

The objective of this course is to explore recent research areas within natural language processing with sufficient depth. By the end of the course, participants will be equipped to actively contribute to research within their chosen subjects. This course is aimed for graduate students in computer science/engineering. The course assumes that students have a foundational knowledge of machine learning and prior experience or coursework in natural language processing. Topics covered encompass natural language understanding, representation learning, contextual representations, multitask learning, learning from multiple modalities, deep generative models, reinforcement learning, generative adversarial learning, NLP methods and metrics. The specific list of topics for the current year will be dependent on the instructor and prevailing trends in natural language processing research, with details announced during the course.

CONTENT

This course contains; Natural language understanding,Representation learning,Contextual representation models,Semantic and syntactic parsing,Question answering,Machine translation,Exam week,Multitask learning, Learning from multiple modalities,Language generation,Deep generative models,Large language models,Reinforcement learning,Generative adversarial learning ,Project/research presentations.

1. 1

   1 - Acquire knowledge about the selected advanced topics in natural language processing with a focus on design of learning algorithms and evaluation of learning algorithms

   Taught by: Project Based Learning Model · Assessed by: Homework

2. 2

   2 - Develop the ability to read and understand recent scientific literature in the field of natural language processing, apply the knowledge obtained by reading scientific papers, discuss and compare methods and assess their potentials and shortcomings

   Taught by: Question - Answer Technique, Project Based Learning Model · Assessed by: Oral Exam, Project Task

3. 3

   3 - Gain a comprehensive understanding of advanced methods, and apply this knowledge to solutions of practical problems

   Taught by: Problem Solving Method, Self Study Method, Simulation Technique, Experiential Learning, Lecture Method · Assessed by: Traditional Written Exam, Oral Exam, Quiz

4. 4

   4 - Carry out research projects in a chosen area of interest within natural language processing.

   Taught by: Self Study Method, Project Based Learning Model · Assessed by: Project Task

WEEKLY PLAN

1. WEEK 1

   Natural language understanding

2. WEEK 2

   Representation learning

3. WEEK 3

   Contextual representation models

4. WEEK 4

   Semantic and syntactic parsing

5. WEEK 5

   Question answering

6. WEEK 6

   Machine translation

7. WEEK 7

   Exam week

8. WEEK 8

   Multitask learning, Learning from multiple modalities

9. WEEK 9

   Language generation

10. WEEK 10

    Deep generative models

11. WEEK 11

    Large language models

12. WEEK 12

    Reinforcement learning

13. WEEK 13

    Generative adversarial learning

14. WEEK 14

    Project/research presentations

ASSESSMENT

• Rate of Midterm Exam to Success50%
• Rate of Final Exam to Success50%

WORKLOAD

READING

• - Eisenstein (2019), Introduction to Natural Language Processing. - Jurafsky and Martin (~2021), Speech and Language Processing. - Manning and Schütze, Foundations of Statistical NLP. - Murphy, Machine Learning: a Probabilistic Perspective - Goodfellow, Bengio and Courville (2016), Deep Learning. - Bird et al, NLP with Python, a.k.a. the NLTK book. - Lewis Tunstall, Leandro von Werra, and Thomas Wolf. Natural Language Processing with Transformers - Selected Papers.

TEACHING STAFF

• Prof.Dr. Reda ALHAJJCOORDINATOR
• Prof.Dr. Reda ALHAJJ