INTRODUCTION to NATURAL LANGUAGE PROCESSING

AIM

Natural language processing (NLP) is a crucial technology in the era of information age. Exciting advancements in natural language processing (NLP) have recently emerged, enabling systems that can perform tasks such as text translation, question answering, and spoken conversations. This course aims to provide students with a foundational understanding of NLP, including standard frameworks, algorithms, and techniques used to solve various NLP problems. The curriculum will cover topics like language modeling, representation learning, text classification, sequence tagging, syntactic parsing, machine translation, and question answering, with a particular focus on recent deep learning approaches. Through this course, students will receive a comprehensive introduction to NLP concepts, methods, algorithms, applications and state-of-the-art methods research in deep learning for NLP.

CONTENT

This course contains; Introduction to Natural Language Processing (NLP),Lingustic Essentials, Regular Exp., Text Normalization, Edit Distance,N-gram Models,Machine Learning Basics, Text Classification, Naive Bayes and Logistic Regression,Vector Semantics and Dense Word Embeddings,Neural Networks and Neural Language Models,Sequence Labeling for Parts of Speech and Named Entities,Exam Week,RNNs and LSTMs,Transformers and Pretrained Language Models, Fine Tuning and Masked Language Models,Machine Translation, Question Answering and Information Retrieval,Chatbots and Dialogue Systems, Automatic Speech Recognition and Text-to-Speech,Context-Free Grammars, Constituency Parsing, Dependency Parsing, Logical Representations of Sentence Meaning,Review and Project Presentations.

1. 1

   Decompose a real-world problem into subproblems in NLP, use existing natural language processing tools to conduct basic NLP, and identify potential solutions.

   Taught by: Demonstration Method · Assessed by: Traditional Written Exam, Project Task

2. 2

   Learn about the main uses of machine learning techniques and deep learning models in NLP.

   Assessed by: Traditional Written Exam, Project Task, Quiz

3. 3

   Explain state-of-the-art methods to tackle NLP sub-problems, such as text representation, representation learning techniques, text mining, language modeling, and similarity detection, and gain a an understanding about the methods and metrics for various natural language processing tasks and applications.

   Assessed by: Traditional Written Exam, Project Task, Quiz

4. 4

   Extract information from text automatically using concepts and methods from natural language processing (NLP) including stemming, n-grams, POS tagging, and parsing.

   Assessed by: Traditional Written Exam, Homework, Project Task

5. 5

   Get familiarized with the terminology, a breadth of concepts and tasks in NLP.

   Assessed by: Traditional Written Exam, Quiz

WEEKLY PLAN

1. WEEK 1

   Introduction to Natural Language Processing (NLP)

   Preparation: Ch 1

2. WEEK 2

   Lingustic Essentials, Regular Exp., Text Normalization, Edit Distance

   Preparation: Ch 2

3. WEEK 3

   N-gram Models

   Preparation: Ch 3

4. WEEK 4

   Machine Learning Basics, Text Classification, Naive Bayes and Logistic Regression

   Preparation: Ch 4, 5

5. WEEK 5

   Vector Semantics and Dense Word Embeddings

   Preparation: Ch 6

6. WEEK 6

   Neural Networks and Neural Language Models

   Preparation: Ch 7

7. WEEK 7

   Sequence Labeling for Parts of Speech and Named Entities

   Preparation: Ch 8

8. WEEK 8

   Exam Week

   Preparation: Ch 1-8

9. WEEK 9

   RNNs and LSTMs

   Preparation: Ch 9

10. WEEK 10

    Transformers and Pretrained Language Models, Fine Tuning and Masked Language Models

    Preparation: Ch 10, 11

11. WEEK 11

    Machine Translation, Question Answering and Information Retrieval

    Preparation: Ch 13, 14

12. WEEK 12

    Chatbots and Dialogue Systems, Automatic Speech Recognition and Text-to-Speech

    Preparation: Sohbet Robotları ve Diyalog Sistemleri, Otomatik Konuşma Tanıma ve Metinden Konuşmaya

13. WEEK 13

    Context-Free Grammars, Constituency Parsing, Dependency Parsing, Logical Representations of Sentence Meaning

    Preparation: Ch 17, 18, 19

14. WEEK 14

    Review and Project Presentations

ASSESSMENT

• Rate of Midterm Exam to Success30%
• Rate of Final Exam to Success70%

WORKLOAD

READING

• - Speech and Language Processing, D.Jurafsky, J.H.Martin, 3rd Edition, Pearson-Prentice Hall. - Foundations of Statistical Natural Language Processing, C.D.Manning, H.Schütze, MIT Press, 2002. - Jacob Eisenstein, Introduction to Natural Language Processing, 2019.
• - Yoav Goldberg. A Primer on Neural Network Models for Natural Language Processing - Ian Goodfellow, Yoshua Bengio, and Aaron Courville. Deep Learning - Delip Rao and Brian McMahan. Natural Language Processing with PyTorch - Lewis Tunstall, Leandro von Werra, and Thomas Wolf. Natural Language Processing with Transformers

TEACHING STAFF

• Prof.Dr. Selim AKYOKUŞCOORDINATOR
• Prof.Dr. Selim AKYOKUŞ