DEEP LEARNING for NATURAL LANGUAGE PROCESSING

AIM

Natural language processing (NLP) stands out as a pivotal technology in the information age. Its applications influences various aspects of our lives, given that human communication encompasses a wide array of activities: from web searches, advertising, and emails to customer service, language translation, virtual agents, medical reports, and political discourse. Over the past decade, deep learning, employing neural network approaches, has demonstrated remarkable efficacy in numerous NLP tasks. This involves the use of singular end-to-end neural models that eliminate the need for traditional, task-specific feature engineering. This course offers students a comprehensive introduction to the latest advancements in Deep Learning for NLP. Through a combination of lectures, assignments, and a final project, participants will acquire the essential skills to conceptualize, implement, and comprehend different neural network models.

CONTENT

This course contains; Introduction to NLP and Deep Learning,Foundations of NLP, Machine Learning and Deep Learning,Vector Semantics and Embeddings,Language Models,Neural Networks and Neural Language Models,Recurrent Neural Networks and Language Models,Seq2Seq, Machine Translation, Subword Models,Exam Week,Transformers and Pretrained Language Models ,Transformers,Fine-tuning and Masked Language Models ,Prompting and Instruct Tuning,Question Answering, Chatbots and Dialogue Systems, Automatic Speech Recognition and Text-to-Speech Conversion ,Project presentations.

1. 1

   Recognizes the natural language processing and adopt advanced natural language processing techniques to solve real-world problems.

   Taught by: Project Based Learning Model · Assessed by: Homework

2. 2

   Analyze state-of-the-art deep learning architectures for NLP.

   Taught by: Question - Answer Technique, Project Based Learning Model · Assessed by: Oral Exam, Project Task

3. 3

   Implement the common deep neural network models for NLP.

   Taught by: Problem Solving Method, Self Study Method, Simulation Technique, Experiential Learning, Lecture Method · Assessed by: Traditional Written Exam, Oral Exam, Quiz

4. 4

   Evaluate the research literature on the application of deep learning to natural language processing, and prepare a research project with deep learning architectures for natural language processing and summarize its contents through an oral presentation.

   Taught by: Self Study Method, Project Based Learning Model · Assessed by: Project Task

WEEKLY PLAN

1. WEEK 1

   Introduction to NLP and Deep Learning

2. WEEK 2

   Foundations of NLP, Machine Learning and Deep Learning

3. WEEK 3

   Vector Semantics and Embeddings

4. WEEK 4

   Language Models

5. WEEK 5

   Neural Networks and Neural Language Models

6. WEEK 6

   Recurrent Neural Networks and Language Models

7. WEEK 7

   Seq2Seq, Machine Translation, Subword Models

8. WEEK 8

   Exam Week

9. WEEK 9

   Transformers and Pretrained Language Models

10. WEEK 10

    Transformers

11. WEEK 11

    Fine-tuning and Masked Language Models

12. WEEK 12

    Prompting and Instruct Tuning

13. WEEK 13

    Question Answering, Chatbots and Dialogue Systems, Automatic Speech Recognition and Text-to-Speech Conversion

14. WEEK 14

    Project presentations

ASSESSMENT

• Rate of Midterm Exam to Success50%
• Rate of Final Exam to Success50%

WORKLOAD

READING

• - Dan Jurafsky and James H. Martin. Speech and Language Processing - Jacob Eisenstein. Natural Language Processing - Yoav Goldberg. A Primer on Neural Network Models for Natural Language Processing - Delip Rao and Brian McMahan. Natural Language Processing with PyTorch - Lewis Tunstall, Leandro von Werra, and Thomas Wolf. Natural Language Processing with Transformers
• Natural Language Processing with Python, Steven Bird, Ewan Klein, and Edward Loper at http://www.nltk.org/book/

TEACHING STAFF

• Prof.Dr. Selim AKYOKUŞCOORDINATOR
• Prof.Dr. Selim AKYOKUŞ