Course Review CS 725

CS 725 - Foundations of Machine Learning

Year: 2023-24 Autumn Semester
Instructor: Prof. Sunita Sarawagi

Motivation

In the dynamic realm of Machine Learning and Data Science, CS 725 - Foundations of Machine Learning stands as a crucial gateway. Delving into the intricacies of mathematical principles behind neural networks and optimization problems, this course equips you with indispensable tools to navigate and innovate in the rapidly evolving landscape of technology. Embrace CS 725 as a pathway to unlocking the potential of machine learning and making meaningful contributions to the field.

Course Content

The official course content can be found here. The updated syllabus is listed below:

• Linear regression, Binary classification, Multiclass logistic regression.
• Feed forward neural network, Forward and backward propagation, Gradient descent algorithm, SGD.
• Probabilistic classifiers, MLE, Bayesian estimation, Naive Bayes classifier, Conditional classifiers.
• KNN classifiers, Siamese network, Kernel regression, Classification and Regression tree, Ensemble methods - Bagging, Random forest, Gradient boosting.
• Support vector machine - Hard margin SVM, Soft margin SVM, Kernel trick, Lagrangian primal and dual formulations of SVM
• Convolutional neural network - Cross correlations, Pooling, ResNet.
• Recurrent neural network - Seq2Seq models, Encoder, Decoder, Backpropagation through time, attention in RNN.
• Transformers - Cross attention, Self attention mechanisms.
• Advanced topics - GPT, BERT, CLIP model, PCA

Feedback on Lectures

• Teaching Style: On the positive side, the instructor exhibited excellent communication skills, delivering complex concepts with clarity, and showcasing a profound knowledge of the subject matter. The incorporation of hands-on visual demonstrations in class added a practical dimension to theoretical concepts, enhancing the overall learning experience. However, the lectures were fast-paced, covering a substantial amount of content in a short time, which, while comprehensive, might have made it challenging for some students to fully absorb the material. Additionally, there was an assumption of a certain level of mathematical understanding, potentially leaving some students at a disadvantage. The limited time dedicated to in-depth problem-solving during class sessions could be improved to provide students with more opportunities to apply theoretical knowledge to practical scenarios.
• Attendence: Not taken.

Feedback on Assignments and Exams

• Weightage: Assignments - 12%, Scibes - 3%, Project - 15%, SAFE quizzes - 10%, Midsem - 25%, Endsem - 35%
• Pattern: The homework assignments were generally considered manageable, with a level of difficulty deemed easy. Proficiency in Python was emphasized as a prerequisite for solving these assignments, adding a practical programming component to the coursework. The SAFE quizzes, while also regarded as easy, required diligent revision of class material for successful completion. On the contrary, the exams were perceived as exceptionally challenging, featuring a plethora of “what if” type questions. Success in these exams demanded a very high level of analytical thinking, reflecting a rigorous assessment of the students’ understanding and application of the course content.

Difficulty Level

The overall difficulty level of the course was deemed moderate by students. While the exams presented a challenge, with advanced topics like transformers, GPT, and BERT requiring additional effort, the core course content was generally considered manageable. The course content was labeled as easy, provided students had a comprehensive understanding of every little detail. Notably, the course demanded a significant time investment for completing homework and preparing for exams, making it unsuitable for last-minute preparation. The syllabus, though extensive, was open-notes during exams, providing students with the opportunity to reference their materials.

Prerequisites

While CS 725 doesn’t have formal prerequisites, a strong foundation in key areas can significantly enhance your learning experience. Proficiency in Python is essential, as the coursework involves practical implementation, particularly in homework assignments. Familiarity with basic linear algebra, multi-variable calculus and probability is beneficial, providing a solid groundwork for understanding machine learning concepts.

Grading Stats

Reference Books

• Aston Zhang, Dive into Deep Learning
• Kevin Murphy, Probabilistic Machine Learning
• Jerome Friedman, The Elements of Statistical Learning

Reviewed by

Soumen Mondal (Email: 23m2157@iitb.ac.in)