BAB 04

04 - Deep Learning with PyTorch: CNNs, RNNs, Training Loops

Estimasi Waktu: 60 menit Level: Intermediate

Kenapa PyTorch?

PyTorch adalah framework deep learning paling dominan di research dan production (2025).

FrameworkMarket ShareUse Case
PyTorch~85%Research, production, academia
TensorFlow/Keras~10%Legacy production, some enterprise
JAX~5%Cutting-edge research (Google)

PyTorch Fundamentals

Tensor: The Fundamental Data Structure

Tensor = multidimensional array. Sama kayak NumPy array, tapi bisa jalan di GPU.

import torch

# Create tensors
x = torch.tensor([1, 2, 3])
y = torch.randn(3, 4)  # random normal
z = torch.zeros(2, 3, 4)  # 3D tensor

# GPU
device = torch.device("cuda" if torch.cuda.is_available() else "cpu")
x = x.to(device)

# Operations
a = torch.randn(100, 50)
b = torch.randn(50, 200)
c = a @ b  # matrix multiplication
print(c.shape)  # torch.Size([100, 200])

Autograd: Automatic Differentiation

Ini adalah magic di balik deep learning. PyTorch otomatis menghitung gradients.

x = torch.tensor(2.0, requires_grad=True)
y = x ** 3 + 2 * x ** 2 + 1
y.backward()
print(x.grad)  # dy/dx = 3x² + 4x = 3*4 + 4*2 = 20

Building Neural Networks

The Basic Recipe:

import torch.nn as nn
import torch.optim as optim

class SimpleNN(nn.Module):
    def __init__(self, input_dim, hidden_dim, output_dim):
        super().__init__()
        self.layers = nn.Sequential(
            nn.Linear(input_dim, hidden_dim),
            nn.ReLU(),
            nn.Dropout(0.2),
            nn.Linear(hidden_dim, hidden_dim // 2),
            nn.ReLU(),
            nn.Linear(hidden_dim // 2, output_dim),
        )

    def forward(self, x):
        return self.layers(x)

model = SimpleNN(784, 256, 10)
criterion = nn.CrossEntropyLoss()
optimizer = optim.Adam(model.parameters(), lr=0.001)

The Training Loop:

def train_epoch(model, dataloader, criterion, optimizer, device):
    model.train()
    total_loss = 0

    for batch_idx, (data, target) in enumerate(dataloader):
        data, target = data.to(device), target.to(device)

        # Forward pass
        output = model(data)
        loss = criterion(output, target)

        # Backward pass
        optimizer.zero_grad()
        loss.backward()
        optimizer.step()

        total_loss += loss.item()

    return total_loss / len(dataloader)

The Evaluation Loop:

@torch.no_grad()
def evaluate(model, dataloader, criterion, device):
    model.eval()
    total_loss = 0
    correct = 0

    for data, target in dataloader:
        data, target = data.to(device), target.to(device)
        output = model(data)
        total_loss += criterion(output, target).item()
        pred = output.argmax(dim=1)
        correct += pred.eq(target).sum().item()

    accuracy = correct / len(dataloader.dataset)
    return total_loss / len(dataloader), accuracy

CNN (Convolutional Neural Networks)

CNNs adalah arsitektur untuk image processing.

Key Concepts:

class SimpleCNN(nn.Module):
    def __init__(self, num_classes=10):
        super().__init__()
        self.conv_layers = nn.Sequential(
            nn.Conv2d(3, 32, kernel_size=3, padding=1),
            nn.ReLU(),
            nn.MaxPool2d(2),
            nn.Conv2d(32, 64, kernel_size=3, padding=1),
            nn.ReLU(),
            nn.MaxPool2d(2),
            nn.Conv2d(64, 128, kernel_size=3, padding=1),
            nn.ReLU(),
            nn.AdaptiveAvgPool2d((1, 1)),
        )
        self.classifier = nn.Linear(128, num_classes)

    def forward(self, x):
        x = self.conv_layers(x)
        x = x.view(x.size(0), -1)
        return self.classifier(x)

RNNs & LSTMs (Untuk Sequence Data)

Sebelum Transformers, RNNs adalah standard untuk text/time-series.

class SimpleLSTM(nn.Module):
    def __init__(self, vocab_size, embed_dim, hidden_dim, num_classes):
        super().__init__()
        self.embedding = nn.Embedding(vocab_size, embed_dim)
        self.lstm = nn.LSTM(embed_dim, hidden_dim, batch_first=True)
        self.classifier = nn.Linear(hidden_dim, num_classes)

    def forward(self, x):
        x = self.embedding(x)
        _, (hidden, _) = self.lstm(x)
        return self.classifier(hidden[-1])

Kenapa Transformers Menggantikan RNNs:

Data Loading with PyTorch

from torch.utils.data import Dataset, DataLoader

class CustomDataset(Dataset):
    def __init__(self, data, labels, transform=None):
        self.data = data
        self.labels = labels
        self.transform = transform

    def __len__(self):
        return len(self.data)

    def __getitem__(self, idx):
        x = self.data[idx]
        y = self.labels[idx]
        if self.transform:
            x = self.transform(x)
        return x, y

# DataLoader handles batching, shuffling, multiprocessing
dataloader = DataLoader(
    dataset,
    batch_size=32,
    shuffle=True,
    num_workers=4,
    pin_memory=True,  # faster GPU transfer
)

Best Practices

1. Start Simple

2. Monitor Everything

from torch.utils.tensorboard import SummaryWriter

writer = SummaryWriter()
writer.add_scalar("Loss/train", train_loss, epoch)
writer.add_scalar("Loss/val", val_loss, epoch)
writer.add_histogram("weights/layer1", model.layers[0].weight, epoch)

3. Use Mixed Precision

scaler = torch.cuda.amp.GradScaler()

with torch.cuda.amp.autocast():
    output = model(data)
    loss = criterion(output, target)

scaler.scale(loss).backward()
scaler.step(optimizer)
scaler.update()

4. Save Checkpoints

torch.save({
    "epoch": epoch,
    "model_state_dict": model.state_dict(),
    "optimizer_state_dict": optimizer.state_dict(),
    "loss": loss,
}, "checkpoint.pt")

Latihan

  1. Train CNN untuk MNIST/Fashion-MNIST classification
  2. Train LSTM untuk text classification (sentiment analysis)
  3. Implement training loop dari scratch (tanpa high-level trainer)
  4. Experiment: ganti optimizer (Adam vs SGD), learning rate, batch size
  5. Visualisasikan training curves dengan TensorBoard

Target: 2 model trained (CNN + LSTM), paham training loop, bisa debug training issues.