NATURAL LANGUAGE UNDERSTANDING

AIM

The objective of this course is to teach theoretical concepts, methods and algorithms from linguistics, natural language processing (NLU), and machine learning methods to develop systems and algorithms to understand natural languages efficiently and reliably. Topics include lexical semantics, distributed representations of meaning, contextual language representation, large language models, information retrieval, and advanced evaluations of NLU models, relation extraction, semantic parsing, sentiment analysis, and dialogue agents. Students are expected to develop a project in natural language understanding with a focus on following best practices in the field.

CONTENT

This course contains; Introduction to Natural Language Understanding (NLU),Matrix designs for representations of text, weighting methods, distances, and vector comparisons.,Dimensionality reduction and representation learning.,Distributed word representations,Supervised sentiment analysis,Deep Learning and Transformers,Contextual Representation Models,Exam Week,Large Language Models,Fine-tuning large language models,NLU and Information Retrieval (IR) ,Grounded language understanding,Model evaluation methods and metrics ,Project presentations.

1. 1

   2. Comprehend semantic and syntactic relationships among words using contextual word representation models like transformers, BERT, ELECTRA, and GPT.

   Taught by: Question - Answer Technique, Project Based Learning Model · Assessed by: Oral Exam, Project Task

2. 2

   3. Construct neural information retrieval systems and retrieve specific information from texts employing both classical and neural information retrieval techniques.

   Taught by: Problem Solving Method, Self Study Method, Simulation Technique, Experiential Learning, Lecture Method · Assessed by: Traditional Written Exam, Oral Exam, Quiz

3. 3

   4. Design/implement a Natural Language Understanding (NLU) research project based on your preferences.

   Taught by: Self Study Method, Project Based Learning Model · Assessed by: Project Task

4. 4

   1. Develop robust language models, machine learning systems and algorithms to understand human language effectively.

   Taught by: Project Based Learning Model · Assessed by: Homework

WEEKLY PLAN

1. WEEK 1

   Introduction to Natural Language Understanding (NLU)

2. WEEK 2

   Matrix designs for representations of text, weighting methods, distances, and vector comparisons.

3. WEEK 3

   Dimensionality reduction and representation learning.

4. WEEK 4

   Distributed word representations

5. WEEK 5

   Supervised sentiment analysis

6. WEEK 6

   Deep Learning and Transformers

7. WEEK 7

   Contextual Representation Models

8. WEEK 8

   Exam Week

9. WEEK 9

   Large Language Models

10. WEEK 10

    Fine-tuning large language models

11. WEEK 11

    NLU and Information Retrieval (IR)

12. WEEK 12

    Grounded language understanding

13. WEEK 13

    Model evaluation methods and metrics

14. WEEK 14

    Project presentations

ASSESSMENT

• Rate of Midterm Exam to Success50%
• Rate of Final Exam to Success50%

WORKLOAD

READING

• - Dan Jurafsky and James H. Martin. Speech and Language Processing - Jacob Eisenstein. Natural Language Processing - Yoav Goldberg. A Primer on Neural Network Models for Natural Language Processing - Delip Rao and Brian McMahan. Natural Language Processing with PyTorch - Lewis Tunstall, Leandro von Werra, and Thomas Wolf. Natural Language Processing with Transformers - Selected Papers - State of art software resources in NLU
• Natural Language Processing with Python, Steven Bird, Ewan Klein, and Edward Loper at http://www.nltk.org/book/

TEACHING STAFF

• Prof.Dr. Selim AKYOKUŞCOORDINATOR
• Prof.Dr. Selim AKYOKUŞ