LLM 25-Day Course - Day 1: Essential AI Terminology

Day 1: Essential AI Terminology

Before studying AI, let’s organize the terms you absolutely need to know. Without understanding these terms, you won’t be able to comprehend a single line of any paper or blog post.

AI / ML / DL Relationship

Term	Meaning	Scope
AI (Artificial Intelligence)	All technology that mimics human intelligence	Broadest concept
ML (Machine Learning)	A subfield of AI that learns patterns from data	Subset of AI
DL (Deep Learning)	Neural network-based machine learning	Subset of ML

Core Learning Terminology

Term	Description	Analogy
Parameter	Weight values that the model learns	Synaptic connection strength in the brain
Epoch	One complete pass through the entire dataset	One full reading of a textbook
Batch Size	Number of data samples processed at once	Number of problems solved at once
Learning Rate	Size of weight updates	Step size when walking
Loss Function	Measures the difference between prediction and actual value	Exam error rate
Optimizer	Strategy to reduce loss	Study methodology
Overfitting	Model performs well only on training data	Memorizing answers without understanding
Generalization	Model performs well on new, unseen data	Being able to solve application problems

Core Concepts in Python

import numpy as np

# Parameters: values that the model learns
weights = np.random.randn(3)  # 3 parameters
bias = 0.0                     # Bias is also a parameter

# Simple prediction function
def predict(x, weights, bias):
    return np.dot(x, weights) + bias

# Loss function: difference between prediction and actual (MSE)
def loss_function(predicted, actual):
    return np.mean((predicted - actual) ** 2)

# Basic structure of a training loop
learning_rate = 0.01  # Learning rate
epochs = 100          # Number of epochs
batch_size = 32       # Batch size

for epoch in range(epochs):
    for batch in get_batches(data, batch_size):
        prediction = predict(batch, weights, bias)
        loss = loss_function(prediction, labels)
        gradient = compute_gradient(loss)
        weights -= learning_rate * gradient  # Update weights
    print(f"Epoch {epoch}: Loss = {loss:.4f}")

Detecting Overfitting vs Generalization

# Detecting overfitting: when training loss is low but validation loss is high
train_loss = 0.01   # Loss on training data
val_loss = 0.85     # Loss on validation data

if val_loss > train_loss * 5:
    print("Overfitting suspected! Regularization or data augmentation needed")
else:
    print("Looking good: generalization performance is acceptable")

These terms will keep appearing throughout the next 30 days. You don’t need to memorize them perfectly, but bookmark this page and come back whenever you need a refresher.

Today’s Exercises

1. Draw a Venn diagram of the AI, ML, and DL relationship, and list 2 real-world services for each. (e.g., chess engine, spam filter, ChatGPT)
2. Explain what problems arise when the learning rate is too large versus too small. Experiment with learning_rate = 10.0 and learning_rate = 0.000001.
3. Does increasing the number of epochs always improve performance? Explain in relation to overfitting, and research how to find the optimal number of epochs (Early Stopping).