CSC701 · DEEP LEARNING · MODULE 6

Recent Trends and Applications

Generative Adversarial Networks (GANs): Architecture, Training, and Applications

in Image Generation and DeepFake Technology

6.1 GAN Architecture

6.2 Image Generation

6.2 DeepFake

4 Hours · Based on the CSC701 Deep Learning Syllabus

Module Outline

As per the CSC701 Deep Learning syllabus

6.1 · GENERATIVE ADVERSARIAL NETWORK (GAN)

Architecture

The two-network adversarial system: a Generator that creates synthetic data and a Discriminator that judges it as real or fake, trained together in competition.

6.2 · APPLICATIONS

Image Generation

Using GANs to synthesize entirely new, realistic images from random noise — faces, objects, scenes, and artwork that never existed before.

DeepFake

Using GAN-based encoder-decoder systems to swap or synthesize faces in video and imagery, with major implications for media, security, and ethics.

CSC701 Deep Learning · Module 6

2

What Is a Generative Adversarial Network?

From discriminative models to generative models

DISCRIMINATIVE vs GENERATIVE

THE ADVERSARIAL IDEA

Introduced by Ian Goodfellow et al. in 2014, a GAN pits two neural networks against each other in a zero-sum game:

• A Generator that tries to produce fake data realistic enough to pass as real.
• A Discriminator that tries to correctly tell real data from the Generator's fakes.

As training proceeds, the Generator gets better at fooling the Discriminator, and the Discriminator gets better at catching fakes — each pushes the other to improve, much like a forger and an art detective locked in an ongoing contest.

6.1 GAN Architecture

3

GAN Architecture

The Generator and Discriminator working as a system

SYSTEM DIAGRAM

z

Random Noise

G

Generator

D

Discriminator

R/F

Real or Fake?

Real training data x is also fed into the Discriminator alongside the Generator's fake output G(z).

KEY COMPONENTS

• Latent space (z): a vector of random numbers — the Generator's raw material and source of variety.
• Generator network: upsamples/transforms z into a full synthetic sample (e.g. an image).
• Discriminator network: a binary classifier outputting the probability that its input is real.

6.1 GAN Architecture

4

GAN Architecture

The Generator and Discriminator working as a system

SYSTEM DIAGRAM

6.1 GAN Architecture

4

The Generator Network

Turning random noise into realistic synthetic data

ROLE & GOAL

• Input: a random noise vector z, sampled from a simple distribution (e.g. Gaussian or uniform).
• Output: a synthetic sample G(z) in the same format as real data (e.g. a 64x64 image).
• Goal: maximize the chance the Discriminator classifies G(z) as real.
• The Generator never sees real data directly — it only learns from the Discriminator's feedback about how convincing its fakes are.

TYPICAL STRUCTURE

• A stack of transposed convolution (or fully-connected) layers that progressively upsample the noise vector into larger, higher-resolution feature maps.
• Uses ReLU or Leaky ReLU activations in hidden layers.
• Uses a Tanh (or Sigmoid) activation on the output layer, matching the normalized pixel range of real training images.

Generator's objective (informal)

The Generator is trained to minimize log(1 - D(G(z))) — equivalently, to maximize D(G(z)), i.e. to make the Discriminator's output for its fakes as close to "real" (1) as possible. In practice, the non-saturating variant — maximizing log(D(G(z))) directly — is preferred because it provides much stronger gradients early in training, when the Generator's samples are still easy for the Discriminator to reject.

6.1 GAN Architecture

5

The Discriminator Network

Learning to tell real data from generated fakes

ROLE & GOAL

• Input: either a real training sample x, or a fake sample G(z) from the Generator.
• Output: a single probability D(.) in [0, 1] — the estimated chance the input is real.
• Goal: correctly assign high probability to real data and low probability to generated data.
• It is simply a standard binary classifier — the adversarial part comes from what it is trained against.

TYPICAL STRUCTURE

• A stack of convolutional layers that progressively downsample the input image into compact feature maps.
• Uses Leaky ReLU activations in hidden layers (helps gradient flow for negative inputs).
• Uses a Sigmoid activation on the single output neuron to produce a valid probability.

Discriminator's objective (informal)

The Discriminator is trained to maximize log(D(x)) + log(1 - D(G(z))) — i.e. to push its output toward 1 for real samples and toward 0 for fake samples. It is updated using standard binary cross-entropy loss and ordinary backpropagation; the only thing unusual about GAN training is that its "negative" examples are supplied by a Generator that keeps changing.

6.1 GAN Architecture

6

GAN Training Process

Alternating updates in a minimax game

MINIMAX OBJECTIVE

min_G max_D V(D,G) = E_x~p_data[log D(x)] + E_z~p_z[log(1 - D(G(z)))]

THE TRAINING LOOP (REPEATED EACH ITERATION)

Step 1 — Train the Discriminator:

Sample a batch of real data and a batch of fake data G(z). Update D's weights to better separate the two, holding G fixed.

Step 2 — Train the Generator:

Sample a new batch of noise z. Update G's weights to increase D(G(z)), holding D fixed.

Step 3 — Repeat:

Alternate these two steps for many epochs until D can no longer reliably distinguish real from generated data (ideally D(x) approx 0.5 for everything).

TRAINING CHALLENGES

Mode collapse — Generator produces limited variety instead of the full data distribution. Training instability — the two networks can oscillate rather than converge. Vanishing gradients — if D becomes too strong too quickly, G receives little useful signal to learn from.

6.1 GAN Architecture

7

Notable GAN Variants

The base architecture has many specialized descendants

DCGAN

Deep Convolutional GAN — established the convolutional architecture patterns most image GANs still use today.

Conditional GAN (cGAN)

Both G and D are conditioned on extra information (e.g. a class label), allowing controlled, targeted generation.

CycleGAN

Learns image-to-image translation between two domains (e.g. photo <-> painting) without paired training examples.

StyleGAN

Injects style information at multiple resolutions, giving fine-grained control over generated image attributes — widely used for photorealistic faces.

Pix2Pix

A conditional GAN for paired image-to-image translation (e.g. sketch -> photo, day -> night).

Wasserstein GAN (WGAN)

Replaces the original loss with the Wasserstein distance, improving training stability and reducing mode collapse.

6.1 GAN Architecture

8

Application · Image Generation

Synthesizing entirely new, realistic visual content

HOW IT WORKS

• A trained Generator maps points in a low-dimensional latent space to full images.
• Sampling a new random noise vector z and passing it through G produces a brand-new image that never existed in the training set.
• Nearby points in latent space tend to produce visually similar images, enabling smooth interpolation and controlled attribute editing.

REAL-WORLD USE CASES

• Synthetic training data — generating extra labeled examples for domains where real data is scarce or expensive.
• Art and design — tools like StyleGAN power AI-assisted art, character design, and fashion prototyping.
• Super-resolution and photo restoration — GANs can plausibly fill in or upscale missing image detail.
• Game and film asset generation — rapidly producing textures, characters, and concept art.

Why GANs (and not just any generative model)

• Compared to earlier generative approaches (e.g. plain autoencoders), the adversarial training signal from the Discriminator pushes the Generator toward sharp, high-fidelity outputs rather than blurry averages.
• GANs need no explicit likelihood function for the data — they only need a way to sample from it — which makes them applicable to complex, high-dimensional domains like natural images where an explicit density is intractable.

6.2 Applications · Image Generation

9

Application · DeepFake

GAN-based synthesis and manipulation of faces in video and imagery

HOW IT WORKS

• A common approach uses a shared Encoder that extracts identity-agnostic features (expression, pose, lighting) from a face.
• Separate, identity-specific Decoders reconstruct a particular person's face from those features.
• Feeding Person A's encoded features into Person B's Decoder produces "B" performing "A's" expressions and movements — the essence of a face swap.

LEGITIMATE USES

• Film and VFX — de-aging actors, dubbing dialogue with matching lip movement across languages.
• Accessibility — synthesizing personalized avatars and sign-language presenters.
• Privacy protection — anonymizing faces in sensitive footage while preserving natural motion.

Risks and Ethical Concerns

• Misinformation and fraud — fabricated video/audio of public figures used to deceive or manipulate opinion.
• Non-consensual content — using someone's likeness without permission, causing real harm.
• Detection is an active research area — artifacts in lighting, blinking patterns, and boundary blending (e.g. inconsistent SSIM/structural similarity) are commonly used to flag manipulated media.

6.2 Applications · DeepFake

10

Module Summary

Recent Trends and Applications

1

GANs are adversarial by design

A Generator and Discriminator train against each other, each improving only because the other is competing with it.

2

Generator creates, Discriminator judges

G maps random noise to synthetic data; D is a binary classifier trained to catch G's fakes.

3

Training is a minimax game

D maximizes its ability to separate real from fake; G minimizes D's ability to do so — an equilibrium is the ideal end state.

4

The architecture has many variants

DCGAN, cGAN, CycleGAN, StyleGAN, Pix2Pix, and WGAN each adapt the core idea for specific goals or stability improvements.

5

Image generation is a flagship application

GANs synthesize new, realistic images for art, synthetic data, super-resolution, and creative industries.

6

DeepFakes are dual-use technology

The same encoder-decoder mechanics enable legitimate VFX and accessibility tools, and also raise serious ethical and security concerns.

CSC701 Deep Learning · Module 6: Recent Trends and Applications

11