Files/Machine-Learning-Course/README.md

# Machine Learning Course Homeworks 
A short description of each question is included below.

## HW1 - Bayes Classifier & Parametric Density Estimation

### Theoretical Section
* Q1: Bayes error and cauchy distribution
* Q2: Bayesian minimum risk classifier
* Q3: Bayes classifier decision boundary
* Q4: Bayes classifier and normal distribution
* Q5: Parameter estimation using maximum likelihood (MLE) and applying Bayes classifier
* Q6:  Parameter estimation using MLE and maximum a posteriori (MAP)

### Programming Section
 * Q7: Implementing naive Bayes classifier from scratch
 * Q8: Implementing a simple pixel classification

## HW2 - Non-parametric Density Esimation

### Theoretical Section
* Q1: Parzen window variance
* Q2: Parzen window mean
* Q3: Linear regression with L1/L2 regularization
* Q4: Decision boundary using nearest-neighbor rule 
* Q5: Nearest-neighbor classifier error

### Programming Section
* Q6: Implementing parzen window density estimation from scratch
* Q7: Classifying using the Parzen Window
* Q8: Implementing Logistic Regression and K-Nearest Neighbors (KNN) classifiers from scratch, and use them to classify the `seeds.csv` dataset
* Q9: Implementing Linear Regression from scratch, and using it to classify the `marketing_campaign.csv` dataset

## HW3 - Decision Tree & AdaBoost

### Theoretical Section
* Q1: AdaBoost concept
* Q2: AdaBoost classifier error
* Q5: Decision tree and information gain

### Programming Section
* Q3: classifying the `credit_scoring_sample.csv` dataset using a Random Forest and Bagging classifiers. Additionally, utilizing Bootstrap sampling to estimate the mean of customers' age
* Q4: Implementing AdaBoost classifier from scratch, and using it to classify the iris dataset
* Q6: Implementing Decision Tree from scratch using ID3 algorithm, and use it to classify `prison_dataset.csv` dataset

## HW4 - Kernel Methods & Neural Network

### Theoretical Section
* Q1: Multi-Layer Perceptron (MLP) and activation function
* Q2: Forward and backward propagation in neural networks
* Q5: Kernel methods and meaning of data in transferred space

### Programming Section
* Q3: Comparing MLP and CNN concerning translational invariance feature
* Q4: 
    - Classifying a 4-sample dataset in a 2D space with hard SVM
    - Finding a mapping to transfer a dataset to a new space where it Is linearly separable
* Q6: Classifying MNIST dataset with kernel SVM. Linear, RBF and polynomial kernels will be tested.

## HW5 - Clustering & Expectation-Maximization

### Theoretical Section
* Q1: Calculate within-class and between-class scatter matrices
* Q2: Model selection concepts
* Q3: Expectation-Maximization (EM) method for exponential mixture model 
* Q4: Estimate Gaussian Mixture Model (GMM) using neural network 

### Programming Section
* Q5: Implementing Principal Component Analysis (PCA) from scratch, and using it to reduce the dimensionality of the fashion-MNIST dataset
* Q6: GMM density estimation for the MNIST dataset
* Q7: Clustering the `customers_dataset.csv` dataset using the k-means algorithm. Also find the optimal value of k using different methods and score functions, such as:
    - K-means Distortion and Elbow Method
    - Silhouette Score
    - Davies-Bouldin Index
    - Calinski-Harabasz Index
    - Dunn Index
## How to Run
Make a folder named `assets` in the root of each homework, download the necessary datasets that has been ignored, and run the code.

---
For more information, please refer to the report of desired homework.