CUET MSc CSE Admission · Add-on Note · Machine Learning

Machine Learning — The Complete Add-On Note

A gap-filler note for the topic your DSA/DBMS/OS/Networks book doesn't cover: undergraduate-level Machine Learning — what a "model" is, how classification works, image processing, image augmentation, neural networks, the key equations, and the classic worked examples (like the watermelon decision-tree example) that examiners like to ask in viva. Written beginner-first: every term is defined in plain language before it is used.

CoversModels · training · classification · image processing · neural networks · CNN · equations

FormatWritten definitions + viva-style Q&A, example-driven

How to useRead chapters 1→4 first (core ideas), then 5-6 (neural nets/CNN), then revise ch.9 rapid-fire the night before

Chapter 01

What is Machine Learning?

Foundation — every viva starts here

Definition — Machine Learning (ML) A field of computer science where a program learns patterns from data instead of being explicitly programmed with fixed rules. Example: instead of writing hundreds of if-else rules to detect spam email, you show the program thousands of labelled emails (spam / not-spam) and it learns the pattern itself.

Definition — Model A model is the mathematical object that has learned the pattern — think of it as a function f(x) = y whose internal numbers (called parameters or weights) were tuned using data. "Running a model" means feeding it a new input x and reading the output y (this is called inference or prediction). "Training a model" means the earlier step of tuning those weights using known examples.

Everyday analogy A model is like a student. Training = studying past exam questions with answers (data). Parameters/weights = what the student remembers (their understanding). Inference/running the model = sitting the real exam and answering a question they've never seen before, using what they learned.

1.1 Three types of learning

Type	What the data looks like	Goal	Example
Supervised learning	Input and correct output (label) given	Learn input→output mapping	Given house size → predict price; given image → predict "cat" or "dog"
Unsupervised learning	Only input, no label	Find hidden structure/groups	Group customers into clusters by buying behaviour
Reinforcement learning	No fixed dataset — an agent acts and gets reward/penalty	Learn a policy that maximises reward	A game-playing agent learning to win chess

1.2 Supervised learning splits into two jobs

Definition — Classification Predicting a category/class label (discrete output). Example: is this email spam or not-spam? Is this tumour benign or malignant? Is this image a cat, dog, or bird?

Definition — Regression Predicting a continuous number. Example: predicting house price, predicting tomorrow's temperature.

Quick way to tell them apart in an MCQ Ask: "is the answer a label/category or a number?" Category → classification. Number → regression. "Spam or not" → classification. "House price in taka" → regression.

1.3 AI vs ML vs Deep Learning (very common viva opener)

Definition — the hierarchy Artificial Intelligence (AI) — the broadest field: any technique that makes machines act "intelligently" (includes hand-coded rule systems, search algorithms, ML, etc). Machine Learning (ML) — a subset of AI where the system learns from data rather than fixed rules. Deep Learning (DL) — a subset of ML that uses neural networks with many (deep) layers.

Picture it as nested circles

AI  ⊃  Machine Learning  ⊃  Deep Learning

Example: a chess program using hand-written if-else rules is AI but not ML. A spam filter trained from labelled emails is ML. A CNN that recognises faces from raw pixels is ML and also DL.

1.4 Structured vs Unstructured data

Definition — Structured data Data that fits neatly into rows and columns (a table/spreadsheet), each column a clear feature. Example: a spreadsheet of students with columns age, CGPA, department.

Definition — Unstructured data Data with no fixed table format. Example: images, audio, free-text documents. Deep learning (especially CNNs and RNNs) is what made ML good at handling this kind of data.

1.5 The general ML pipeline (memorize this order)

1. Collect data — gather raw examples (images, text, numbers).

2. Preprocess — clean, normalize, handle missing values, resize images.

3. Split data — into training set and test set (sometimes validation set too).

4. Choose a model — e.g. Decision Tree, Neural Network.

5. Train — feed training data, let the model adjust its weights to reduce error.

6. Evaluate — test on unseen data, measure accuracy/error.

7. Tune / repeat — adjust settings (hyperparameters), retrain if needed.

8. Deploy / run — use the trained model on real new inputs.

Chapter 02

Data, Features & the ML Workflow

Vocabulary that every later chapter assumes you know

Definition — Feature A measurable input variable used to describe an example. Example: to predict house price, features could be: size in sq. ft, number of rooms, location. In an image, a "feature" can literally be a pixel value, or a higher-level pattern like an edge.

Definition — Label / Target The correct answer attached to a training example, that the model tries to predict. Example: in a spam dataset, the label is "spam" or "not-spam".

Definition — Dataset split: Training / Validation / Test Training set — data the model learns from (usually ~70-80%). Validation set — used during development to tune settings, not for final judgement. Test set — completely unseen data, used only once at the end to report real-world performance.

Trap — never test on training data If you evaluate accuracy using the same data you trained on, the score looks great but is misleading — the model may have just memorised the answers rather than learned the pattern. Always report accuracy on the held-out test set.

2.1 Feature scaling / normalization

Definition — Normalization Rescaling feature values to a common range (often 0 to 1) so that features with big numbers (like "salary in taka") don't unfairly dominate features with small numbers (like "age"). Common formula:

Min-Max normalization equation

x_new = (x - min(x)) / (max(x) - min(x))

Example: if ages in a dataset range from 10 to 90, an age of 30 becomes (30-10)/(90-10) = 0.25.

2.2 Parameters vs Hyperparameters

Parameter	Hyperparameter
Learned automatically from data during training (e.g. weights, bias)	Set by the human before training starts (e.g. learning rate, number of layers, K in KNN)

Viva Q: What's the difference between training and testing? Training is when the model looks at labelled examples and adjusts its internal weights to reduce mistakes. Testing is when the already-trained (frozen) model is given brand-new inputs it has never seen, purely to check how well it generalises.

Chapter 03

Classification Algorithms

Very likely to be asked — know at least 3 of these with an example

3.1 Linear Regression (the simplest starting point)

Definition — Linear Regression Fits a straight line (or plane, in higher dimensions) through the data to predict a continuous number.

Equation

y = w1*x1 + w2*x2 + ... + wn*xn + b

where w's are weights, b is the bias/intercept. Training = finding the w's and b that minimize the average squared difference between predicted y and actual y (called Mean Squared Error).

3.2 Logistic Regression (despite the name, this is classification)

Definition — Logistic Regression Takes the linear regression output and squashes it through the sigmoid function to get a probability between 0 and 1, used for binary classification (two classes).

Sigmoid equation

sigma(z) = 1 / (1 + e^(-z))     where z = w.x + b

If sigma(z) > 0.5 → predict class 1, else class 0. Example: predicting pass/fail from hours studied.

3.3 K-Nearest Neighbours (KNN)

Definition — KNN A "lazy" learner with no real training step — to classify a new point, it looks at the K closest points in the training data (by distance, usually Euclidean) and takes a majority vote of their labels.

Example K=5, new fruit's 5 nearest neighbours in the dataset are: apple, apple, apple, orange, orange → majority is apple → predict apple.

Trap — choosing K Small K (like 1) → sensitive to noise/outliers. Large K → smoother but may blur class boundaries. K is usually chosen as an odd number to avoid ties in binary classification.

3.4 Naive Bayes

Definition — Naive Bayes A probabilistic classifier based on Bayes' theorem, that "naively" assumes all features are independent of each other.

Bayes' theorem

P(class | features) = P(features | class) * P(class) / P(features)

Classic use: spam filtering — probability an email is spam given the words it contains.

3.5 Decision Tree — the "Watermelon" style example

Definition — Decision Tree A flow-chart-like model that asks a series of yes/no or multi-way questions about features, splitting the data at each node, until it reaches a leaf that gives the final class. Built using measures like Entropy and Information Gain (algorithms: ID3, C4.5, CART).

Worked example — is this watermelon good or bad? (classic textbook dataset) Features used in the textbook example: colour (green/black/light), root shape (curled/slightly curled/straight), sound when tapped (crisp/dull/muffled) → label: good melon or bad melon.

Root node: "Sound when tapped?"
 ├── Crisp      → mostly "bad melon" (leaf)
 ├── Dull       → check next feature: "Root shape?"
 │      ├── Curled → "good melon"
 │      └── Straight → "bad melon"
 └── Muffled    → "bad melon" (leaf)

The tree picks the most informative feature first (highest Information Gain) as the root, exactly like "sound" is the strongest clue for melon ripeness in this dataset.

Definition — Entropy A measure of impurity/disorder in a set of labels. Pure set (all same class) → entropy = 0. 50/50 mixed set → entropy is at its maximum (1, for 2 classes).

Entropy equation

Entropy(S) = - sum( p_i * log2(p_i) )

where p_i is the proportion of class i in set S. Information Gain = Entropy(parent) − weighted average Entropy(children); the feature with the highest Information Gain is chosen to split on.

3.6 Support Vector Machine (SVM)

Definition — SVM Finds the best separating boundary (hyperplane) between two classes — "best" meaning it maximizes the margin (gap) between the boundary and the closest points of each class, called support vectors. For data that isn't separable by a straight line, SVM can use a kernel trick to project data into higher dimensions where it becomes separable.

Algorithm	One-line idea	Good for
Linear Regression	Best-fit straight line	Continuous number prediction
Logistic Regression	Sigmoid-squashed line → probability	Binary classification, simple & fast
KNN	Vote among nearest neighbours	Small datasets, simple boundaries
Naive Bayes	Bayes' theorem + independence assumption	Text/spam classification
Decision Tree	Series of feature questions	Interpretable rules, mixed data types
SVM	Max-margin separating boundary	High-dimensional, clear-margin data

Chapter 04

Evaluating a Model

Guaranteed short-written / definition question

4.1 Overfitting vs Underfitting

Definition — Overfitting The model learns the training data too well — including its noise/quirks — so it performs great on training data but poorly on new/test data. Like a student who memorised last year's exact exam questions but can't solve a slightly different question.

Definition — Underfitting The model is too simple to capture the real pattern, so it performs poorly on both training and test data. Like a student who barely studied.

Fixes Overfitting → get more data, simplify the model, use regularization, use dropout (in neural nets), early stopping. Underfitting → use a more complex model, add more/better features, train longer.

Definition — Bias-Variance Tradeoff Bias = error from a model being too simple/rigid — it "assumes" too much and misses real patterns (leads to underfitting). Variance = error from a model being too sensitive to the exact training data — it changes wildly if the training data changes slightly (leads to overfitting). Good models balance the two: low bias and low variance together, which is why the tradeoff is often drawn as a U-shaped total-error curve as model complexity increases.

Example A straight-line model fit to clearly curved data → high bias (underfits). A wiggly line passing through every single training point → high variance (overfits). The right-degree curve in between generalises best.

4.2 Confusion Matrix (for classification)

	Predicted Positive	Predicted Negative
Actual Positive	True Positive (TP)	False Negative (FN)
Actual Negative	False Positive (FP)	True Negative (TN)

Key metric equations

Accuracy  = (TP+TN) / (TP+TN+FP+FN)
Precision = TP / (TP+FP)     -- of predicted positives, how many were right
Recall    = TP / (TP+FN)     -- of actual positives, how many did we catch
F1-score  = 2 * (Precision*Recall) / (Precision+Recall)

Worked example Out of 100 emails: 40 actually spam, 60 actually not-spam. Model predicts 35 of the spam correctly (TP=35, FN=5), and wrongly flags 10 good emails as spam (FP=10, TN=50). Accuracy = (35+50)/100 = 0.85. Precision = 35/45 ≈ 0.78. Recall = 35/40 = 0.875.

4.3 Cross-validation

Definition — K-fold cross-validation Split data into K equal parts (folds); train on K−1 folds and test on the remaining 1 fold, repeat K times so every fold is used once as test data, then average the results. Gives a more reliable performance estimate than a single train/test split.

4.4 ROC Curve & AUC

Definition — ROC Curve A plot of True Positive Rate (Recall) against False Positive Rate at every possible classification threshold, showing the tradeoff between catching more positives and raising more false alarms as the decision threshold changes.

Definition — AUC (Area Under the Curve) A single number (0 to 1) summarizing the ROC curve. AUC = 1 → perfect classifier. AUC = 0.5 → no better than random guessing. Higher AUC = better overall separation between classes, regardless of which threshold you pick.

Chapter 05

Neural Networks

High weight — equations are commonly asked

5.1 The single neuron (Perceptron)

Definition — Perceptron / Artificial Neuron The basic unit of a neural network. It takes inputs, multiplies each by a weight, sums them with a bias, then passes the result through an activation function to produce an output.

Neuron equation

z = w1*x1 + w2*x2 + ... + wn*xn + b
a = activation(z)

This is exactly the same weighted-sum idea as linear/logistic regression — a neuron IS basically a tiny logistic regression unit.

5.2 Activation functions

Function	Equation	Output range	Typical use
Sigmoid	`1/(1+e^-z)`	0 to 1	Binary output layer, probabilities
Tanh	`(e^z - e^-z)/(e^z + e^-z)`	-1 to 1	Hidden layers (older nets)
ReLU	`max(0, z)`	0 to ∞	Hidden layers — modern default, fast, avoids vanishing gradient
Softmax	turns a vector of scores into probabilities that sum to 1	0 to 1 (each), sum=1	Multi-class output layer (e.g. 10 digit classes)

Why do we need activation functions at all? Without a non-linear activation, stacking many layers would collapse into just one big linear equation — the network couldn't learn curved/complex decision boundaries (like telling apart handwritten digits). Non-linearity is what gives deep networks their power.

5.3 Network structure

Definition — Layers Input layer — receives raw features (e.g. pixel values). Hidden layer(s) — intermediate computation; "deep learning" means many hidden layers. Output layer — produces the final prediction (e.g. class probabilities).

5.4 Training a neural network

Definition — Loss function A number measuring how wrong the model's prediction is compared to the true label. Training tries to minimize this number. Common ones: Mean Squared Error (regression), Cross-Entropy Loss (classification).

Definition — Gradient Descent An optimization algorithm that repeatedly nudges each weight in the direction that reduces the loss, using the loss function's slope (gradient).

Weight update equation

w_new = w_old - learning_rate * (dLoss/dw)

Learning rate controls step size — too high → overshoots and never settles; too low → training is very slow.

Definition — Backpropagation The algorithm that efficiently computes the gradient of the loss with respect to every weight in the network, by applying the chain rule backward from the output layer to the input layer, layer by layer.

Viva Q: In one sentence, how does a neural network "learn"? It makes a prediction (forward pass), measures the error with a loss function, then uses backpropagation + gradient descent to adjust every weight slightly so the error gets smaller next time — repeated over many examples and many passes (epochs).

Definition — Epoch / Batch One epoch = one full pass through the entire training dataset. Data is usually split into small batches (mini-batches) so weights are updated many times per epoch rather than once.

5.5 Three flavours of Gradient Descent

Type	Updates weights using	Tradeoff
Batch Gradient Descent	The whole training set at once, per update	Stable but slow, memory-heavy
Stochastic GD (SGD)	Just 1 example at a time	Fast, noisy updates, can escape bad local minima
Mini-batch GD	A small batch (e.g. 32, 64 examples)	Practical middle ground — used almost everywhere in practice

5.6 Regularization & Dropout (the actual overfitting fixes for neural nets)

Definition — Regularization (L1/L2) Adding a penalty term to the loss function that discourages weights from becoming too large, which keeps the model simpler and less likely to overfit.

Equations

L2 (Ridge):  Loss_new = Loss + lambda * sum(w^2)
L1 (Lasso):  Loss_new = Loss + lambda * sum(|w|)

lambda controls how strongly weights are penalised. L1 can push some weights to exactly 0 (automatic feature selection); L2 just shrinks them smoothly.

Definition — Dropout During training, randomly "turn off" a fraction of neurons (e.g. 20-50%) in a layer at each training step, forcing the network to not rely too heavily on any single neuron. At test time, all neurons are used again. This reduces overfitting by preventing neurons from co-adapting too closely to each other.

5.7 One-Hot Encoding

Definition — One-Hot Encoding A way to turn a categorical label into numbers a model can use — each category becomes a vector of 0s with a single 1 marking that category.

Example Classes {cat, dog, bird}: cat = [1,0,0], dog = [0,1,0], bird = [0,0,1]. This avoids implying a false order/size relationship that plain numbers (cat=1, dog=2, bird=3) would wrongly suggest.

5.8 Vanishing & Exploding Gradient

Definition — Vanishing Gradient In deep networks, gradients can shrink toward 0 as they're propagated backward through many layers (especially with sigmoid/tanh), so early layers barely update — training stalls. Exploding Gradient is the opposite problem — gradients grow huge and weight updates overshoot wildly, destabilising training. ReLU activations, careful weight initialization, and architectures like batch normalization are common fixes.

5.9 RNN — for sequences

Definition — Recurrent Neural Network (RNN) A neural network designed for sequential data (text, speech, time-series) — it processes one element at a time while keeping a hidden state that carries information from earlier elements in the sequence forward, so the output at each step depends on both the current input and what came before.

Example Predicting the next word in "I love machine ___" — an RNN reading the sentence word by word retains context from "I love machine" to predict "learning" is likely next. (LSTM and GRU are improved RNN variants that handle longer sequences better.)

5.10 Transfer Learning

Definition — Transfer Learning Taking a model already trained on a large dataset (e.g. a CNN trained on millions of general images) and reusing/fine-tuning it for a new, smaller, related task, instead of training a new model from scratch. Saves time and works well even with limited data.

Example Using a CNN pretrained on ImageNet (general objects) and fine-tuning just its last few layers to classify medical X-ray images, instead of training an entire new CNN from zero.

Chapter 06

Image Processing Basics

Needed before CNNs make sense

Definition — Digital Image A grid (matrix) of numbers. A grayscale image is a 2-D matrix where each cell (pixel) holds an intensity value, usually 0 (black) to 255 (white). A colour image is 3 stacked matrices — Red, Green, Blue (RGB channels) — so its shape is written as height × width × 3.

6.1 Common preprocessing operations

Resizing: making all images the same fixed dimensions so they can be fed to a model in batches.

Grayscaling: converting 3-channel RGB into 1-channel intensity, reduces data size.

Normalization: scaling pixel values from [0,255] to [0,1] (divide by 255), helps training converge faster.

Cropping: cutting out a region of interest.

Filtering/blurring: smoothing an image, e.g. Gaussian blur, to reduce noise.

Edge detection: highlighting boundaries between regions, e.g. Sobel filter.

6.2 Convolution — the core image operation

Definition — Convolution (image processing sense) Sliding a small matrix called a kernel/filter over the image, and at each position computing a weighted sum of the pixels it covers, to produce one output pixel. Different kernels detect different patterns (edges, blur, sharpen).

Tiny worked example A 3×3 edge-detecting kernel slid over a 5×5 image produces a smaller output map where bright values mark where an edge (sudden change in pixel intensity) was found.

Definition — Stride & Padding Stride = how many pixels the kernel moves each step (stride 1 = move one pixel at a time, slower but more detail; stride 2 = skip a pixel, faster, smaller output). Padding = adding a border of zeros around the image so the output size can be kept the same as the input, and edge pixels get processed properly.

Chapter 07

CNNs & Image Augmentation

Most likely image-processing-meets-ML question

7.1 Convolutional Neural Network (CNN)

Definition — CNN A neural network specialised for grid-like data (images), built mainly from convolutional layers (learn filters automatically instead of hand-designing them), pooling layers (shrink the data, keep the important info), and finally regular (fully-connected) layers for the final classification.

Typical CNN pipeline

Input image → [Conv layer → ReLU → Pooling] × several times → Flatten → Fully-connected layer(s) → Softmax → class probabilities

Definition — Pooling (e.g. Max Pooling) Reduces the size of the feature map by summarizing small regions — Max Pooling keeps only the largest value in each small window (e.g. 2×2), which keeps the strongest signal while cutting computation and adding some tolerance to small shifts in the image.

Why CNN and not a plain neural network for images? A plain (fully-connected) network would need a separate weight for every single pixel-to-neuron connection — huge and ignores that nearby pixels are related. CNNs reuse the same small filter across the whole image (parameter sharing), drastically cutting the number of weights while naturally detecting local patterns like edges, corners, and eventually whole shapes in deeper layers.

7.2 Image Augmentation

Definition — Image Augmentation Artificially creating more training images by applying random, label-preserving transformations to existing images — used when you don't have enough real data, and to make the model more robust (generalise better, reduce overfitting).

Augmentation	What it does
Rotation	Rotates the image by a random angle
Flip (horizontal/vertical)	Mirrors the image
Zoom / Crop	Zooms in or crops a random region
Shift/translate	Moves the image content slightly within the frame
Brightness/contrast change	Simulates different lighting conditions
Adding noise	Adds random pixel noise to simulate camera imperfections

Example Training a cat/dog classifier with only 500 real cat photos: flipping, rotating by ±15°, and slightly zooming each photo can turn 500 images into an effectively much larger and more varied training set — the model learns "cat-ness" rather than "cats that happen to face left."

Viva Q: Does image augmentation change the label? No — augmentation must be label-preserving. A rotated cat is still a cat. This is the whole point: more variety, same ground truth.

Chapter 08

Unsupervised Learning & Ensembles

Often one MCQ / short-answer question

8.1 K-Means Clustering

Definition — K-Means An unsupervised algorithm that groups data into K clusters. Steps: (1) randomly place K "centroids"; (2) assign each point to its nearest centroid; (3) move each centroid to the average position of its assigned points; (4) repeat steps 2-3 until centroids stop moving.

Example Grouping customers by (age, spending score) into 3 clusters — e.g. "young big spenders", "older moderate spenders", "young low spenders" — with no labels given in advance.

8.2 PCA (Principal Component Analysis)

Definition — PCA An unsupervised technique for dimensionality reduction — it finds new axes (principal components) that capture the most variance in the data, letting you represent data with fewer numbers while losing as little information as possible. Useful before visualizing or speeding up training on high-dimensional data.

8.3 Ensemble methods

Definition — Ensemble learning Combining multiple models to get better performance than any single model alone.

Method	Idea	Example
Bagging	Train many models in parallel on random subsets of data, then average/vote	Random Forest — many decision trees, majority vote
Boosting	Train models one after another, each new one focusing on the previous one's mistakes	AdaBoost, Gradient Boosting, XGBoost

Chapter 09

Rapid-Fire Fact Bank + Key Equations

Last-night revision — read this twice before the exam

Model: learned function f(x)=y; weights = learned numbers.

Training vs Inference: tuning weights vs using frozen weights on new input.

Classification: predicts a category. Regression: predicts a number.

Supervised: needs labels. Unsupervised: no labels, finds structure.

Overfitting: great on train, bad on test (too complex/memorised).

Underfitting: bad on both (too simple).

Sigmoid: 1/(1+e^-z), squashes to (0,1), used for binary output.

Softmax: converts scores to probabilities summing to 1, multi-class output.

ReLU: max(0,z), most common hidden-layer activation today.

Loss function: measures prediction error; training minimizes it.

Gradient Descent: w_new = w_old − lr × gradient.

Backpropagation: computes gradients layer-by-layer via chain rule.

Epoch: one full pass over the whole training set.

Entropy: impurity measure; 0 = pure set, max = perfectly mixed.

Information Gain: entropy drop after a split; picks the best splitting feature.

KNN: classify by majority vote of K nearest points, no real training step.

Naive Bayes: uses Bayes' theorem + assumes feature independence.

SVM: finds max-margin boundary; closest points = support vectors.

Confusion matrix corners: TP, FP, FN, TN.

Accuracy: (TP+TN)/total. Precision: TP/(TP+FP). Recall: TP/(TP+FN).

Convolution: sliding kernel over image, weighted sum → feature map.

Pooling: shrinks feature map, Max Pooling keeps strongest signal.

Stride: kernel step size. Padding: zero-border to preserve output size.

CNN advantage: parameter sharing — same filter reused across the image.

Image augmentation: label-preserving transforms (flip/rotate/zoom/etc.) to enlarge/diversify training data.

K-Means: unsupervised clustering by repeatedly updating centroids.

PCA: reduces dimensions, keeps most variance.

Bagging vs Boosting: parallel independent models vs sequential error-correcting models.

Hyperparameter vs Parameter: set by human before training vs learned during training.

Chapter 10

Written & Viva Practice

Practice explaining these out loud, not just reading them

Q1. What is the difference between a parameter and a hyperparameter? Give one example of each. Parameters are learned automatically during training (e.g. the weights in a neuron). Hyperparameters are chosen by the developer before training starts and control how training happens (e.g. learning rate, or K in KNN).

Q2. Explain, with an example, why a decision tree picks one feature to split on before another. It computes the Information Gain of splitting on each candidate feature — the drop in entropy (impurity) that split would cause — and picks the feature with the highest gain as the root/next node. In the watermelon example, "sound when tapped" gave the biggest drop in impurity, so it became the root question.

Q3. What is overfitting, and name two ways to reduce it. Overfitting is when a model fits the training data (including its noise) so closely that it fails to generalise to new data — high training accuracy, low test accuracy. Two fixes: gather more training data, and use regularization / a simpler model / dropout / early stopping.

Q4. Compare classification and regression with one example each. Classification predicts a discrete category — e.g. classifying an email as spam or not-spam. Regression predicts a continuous number — e.g. predicting a house's price in taka.

Q5. What role does an activation function play in a neural network? It introduces non-linearity. Without it, no matter how many layers are stacked, the whole network would mathematically collapse into one linear equation and could never learn curved/complex decision boundaries.

Q6. Why do we augment images before training a CNN? To artificially increase the amount and variety of training data using label-preserving transforms (flips, rotations, zooms, brightness changes), which helps the model generalise better and reduces overfitting, especially when the real dataset is small.

Q7. Explain the steps of K-Means clustering. Pick K initial centroids, assign every data point to its nearest centroid, recompute each centroid as the mean of its assigned points, and repeat the assign-and-recompute steps until the centroids stop changing.

Q8. What is the vanishing-gradient problem and how does ReLU help? In deep networks using sigmoid/tanh, gradients can shrink to nearly zero as they are propagated back through many layers, making early layers learn extremely slowly. ReLU's gradient is either 0 or 1 (not squashed into a tiny range) for positive inputs, which keeps gradients healthier through many layers.

Tip for the viva: examiners usually care less about reciting a definition word-for-word, and more about whether you can immediately follow it with a concrete example (like the ones above). Practice saying each definition + example out loud in under 30 seconds.

Chapter 11

Extra Topics — the gaps filled in

Added on request — read alongside Ch.1, 4 and 5 where each term is fully explained

These terms are explained in full inside their natural chapters (AI/ML/DL and structured data → Ch.1; bias-variance and ROC/AUC → Ch.4; regularization, dropout, gradient descent types, one-hot encoding, vanishing gradient, RNN, transfer learning → Ch.5). This chapter is just the compressed revision list so you can check nothing is missing.

AI ⊃ ML ⊃ DL: AI is broadest, ML learns from data, DL = ML with deep neural nets.

Structured data: table/rows-columns. Unstructured: images/text/audio.

Bias: too-simple-model error (underfitting). Variance: too-sensitive-model error (overfitting).

ROC curve: TPR vs FPR across thresholds. AUC: 0.5=random, 1=perfect.

Batch GD: whole dataset per update. SGD: one example per update. Mini-batch: small group per update.

L1/L2 regularization: penalize large weights in the loss to reduce overfitting.

Dropout: randomly disable neurons during training only.

One-hot encoding: categories → vectors of 0s with a single 1.

Vanishing gradient: gradients shrink to ~0 in deep nets, early layers stop learning.

Exploding gradient: gradients grow huge, training destabilises.

RNN: neural net with memory (hidden state) for sequences/text/time-series.

Transfer learning: reuse/fine-tune a model pretrained on a big dataset for a new task.

Q9. Is Deep Learning a type of Machine Learning, or a separate field? Deep Learning is a subset of Machine Learning — specifically ML using neural networks with many layers. All DL is ML, but not all ML is DL (e.g. Decision Trees and KNN are ML but not DL).

Q10. What's the difference between bias and variance, and how does that connect to underfitting/overfitting? High bias means the model is too simple and makes strong wrong assumptions, causing underfitting. High variance means the model is too sensitive to the specific training data and captures noise as if it were signal, causing overfitting. A good model finds a balance between the two.

Q11. Why can't we just feed category names like "cat"/"dog" directly as numbers into a model? Assigning arbitrary numbers (cat=1, dog=2) implies a false ordering or distance between categories that doesn't exist. One-hot encoding avoids this by giving each category its own independent 0/1 vector.

Q12. What problem does transfer learning solve? Training a deep network from scratch needs huge amounts of data and compute. Transfer learning reuses the general features already learned by a model trained on a large dataset, and only fine-tunes it for the new, usually smaller, task — saving time and working well with limited data.