UNIT 4

• Generation of Painting, Music, and Play:
• Variants of GAN,
• Types of GAN, Cyclic GAN,
• Using Cyclic GAN to Generate Paintings,
• Neural Style Transfer,
• Style Transfer,
• Music Generating RNN,
• Muse GAN, Autonomous agents,
• Deep Q Algorithm,
• Actor-critic Network.

​

Generation of Painting, Music, and Play

• Generative AI is a branch of Artificial Intelligence that learns patterns from existing data and generates new content.
• It can create different forms of creative content such as paintings, music, and games/plays.
• Modern models use technologies such as Generative Adversarial Networks (GANs), Variational Autoencoders (VAEs), Transformers, and Diffusion Models.

1. Generation of Painting

Generative AI can automatically create digital paintings and artwork from text descriptions, images, or sketches.

Example

​

Prompt:�"A beautiful village surrounded by mountains during sunset in water colour style."

The AI generates a new painting according to the given description.

Applications

• Digital artwork
• Advertising
• Animation and movies
• Concept art
• Game design
• Graphic design

2. Generation of Music 🎵

Generative AI can create new music compositions by learning patterns from existing musical data.

It can generate:

• Melody
• Rhythm
• Harmony
• Lyrics
• Instrumental sounds
• Complete songs

Example

Prompt:�"Generate relaxing piano music for meditation."

The model generates a suitable melody, rhythm, and musical arrangement.

Applications

• Film and background music
• Music composition
• Video games
• Advertising
• Personalized music
• Music education

3. Generation of Play / Games 🎮

​

• Generative AI can create content for interactive games and plays.
• It can generate stories, characters, dialogues, environments, and game levels.

Example

Prompt:�"Create a fantasy adventure game where a player explores a magical forest."

AI can generate the story, characters, dialogues, challenges, and environment.

Applications

• Video game development
• Interactive storytelling
• Virtual worlds
• Character creation
• Game-level generation
• Educational games

A GAN consists of two neural networks:

• Generator (G): Creates fake/synthetic data.
• Discriminator (D): Distinguishes real data from generated (fake) data.

Different GAN variants have been developed to improve image quality, training stability, controllability, and generation performance.

Variants of GAN

​

Generative Adversarial Networks (GANs) have several types, each designed for a specific purpose such as image generation, image translation, stability, or controlled generation.

​

Types of GAN

​

1. Vanilla GAN :Basic GAN architecture with Generator and Discriminator.
2. DCGAN (Deep Convolutional GAN) :Uses CNNs for better image generation.
3. CGAN (Conditional GAN) :Generates images based on a given condition or class.
4. WGAN (Wasserstein GAN) : Improves training stability using Wasserstein distance.
5. WGAN-GP :Adds Gradient Penalty to improve WGAN training.
6. LSGAN (Least Squares GAN) : Uses least-squares loss to improve stability and image quality.
7. CycleGAN : Performs image-to-image translation without paired training images.
8. Pix2Pix :Performs image-to-image translation using paired images.
9. StyleGAN :Provides control over different styles and features of generated images.
10. InfoGAN :Learns meaningful and interpretable latent representations.
11. SRGAN (Super-Resolution GAN) :Converts low-resolution images into high-resolution images.
12. ProGAN (Progressive GAN) :Progressively increases image resolution during training.

Types of GAN

Cycle GAN

​

This GAN is made for Image-to-Image translations, meaning one image to be mapped with another image. For example , if summer and winter are made to undergo the process of Image-Image translation we find a mapping function that could convert summer images into that of winter images and vice versa by adding or removing features according to the mapping function, such that the predicted output and actual output have minimized loss.�

1.Seasonal Transfer: CycleGAN can transform images between different seasons, such as converting winter scenes into summer landscapes and vice versa.

2.Collection Style Transfer:  CycleGAN can learn styles from entire artwork collections such as Van Gogh, Monet and Cezanne, allowing it to generate diverse artistic image styles

3.Object Transformation: CycleGAN can transform objects between different classes, such as apples to oranges or zebras to horses, making it useful for image editing and content generation.

Applications

​

• 🐴 Horse ↔ Zebra transformation
• 🌞 Summer ↔ Winter images
• 🌅 Day ↔ Night conversion
• 🎨 Photo ↔ Painting style transfer
• 🏙️ Synthetic ↔ Real images
• 🖼️ Image enhancement and domain adaptation

Using Cyclic GAN to Generate Paintings

• CycleGAN (Cycle-Consistent Generative Adversarial Network) is a type of Generative AI model used to transform images from one artistic style into another without requiring paired training images.

​

• For example, CycleGAN can convert an ordinary photograph into a painting that looks like the work of an impressionist artist, or transform a painting into a photograph-like image.

​

Applications

​

• Converting photographs into oil paintings or watercolor art.
• Creating artistic landscapes and portraits.
• Digital art and animation.
• Photo restoration and artistic style transfer.
• Creative content generation for games and media.

• Neural Style Transfer (NST) is a deep learning technique that combines the content of one image with the artistic style of another image to generate a new, stylized image.
• It uses a Convolutional Neural Network (CNN), such as VGG-19, to extract content and style features from images.

Neural Style Transfer

Working Procedure

​

1. Content Image: Select an original photograph, such as a mountain, person, or landscape.
2. Style Image: Select an artistic painting, such as a Van Gogh painting.
3. Feature Extraction: A pretrained CNN extracts content and style features from both images.
4. Loss Calculation: The model calculates content loss and style loss.
5. Optimization: Gradient descent updates a generated image to reduce the combined loss.
6. Output Image: The final image preserves the original content while adopting the artistic style.

Style Transfer

Definition

​

Style Transfer is a technique in Artificial Intelligence (AI) that changes the visual appearance of an image by applying the style of another image while preserving the original image's main content.

​

For example, we can convert an ordinary photograph into a painting that looks like a Van Gogh artwork.

Music Generating RNN

• A Recurrent Neural Network (RNN) is a type of deep learning model used to generate music by learning patterns from existing musical sequences.

​

• It predicts the next musical note based on the previously learned notes and generates a new sequence of music.

​

• RNNs are suitable for music generation because music contains sequential patterns, rhythm, melody, and relationships between notes over time.

Working Principle of Music-Generating RNN

• Input: The RNN receives musical notes from existing songs.
• Learning: It learns the sequence of notes, timing, and rhythm.
• Prediction: It predicts the next note based on previous notes.
• Generation: The predicted note is added to the sequence and used to predict another note.
• Output: The complete sequence is converted into a melody using a MIDI synthesizer or audio-generation system.

5. Applications

• Automatic composition of new songs and melodies.
• Background music generation for movies and games.
• AI-generated piano and instrumental music.
• Music composition assistance for musicians.
• Personalized music creation.

MuseGAN is a deep learning model based on a Generative Adversarial Network (GAN) that automatically generates multi-track music. It learns musical patterns from existing music and creates new compositions containing multiple instruments, such as piano, bass, drums, and strings.

MuseGAN

Architecture of MuseGAN

The architecture consists of two main neural networks:

1. Generator (G): Generates new multi-track music from random noise.
2. Discriminator (D): Evaluates whether the music is real (from the training dataset) or generated by the Generator.

Working Principle of MuseGAN

Explanation of the steps

​

1. Input: MuseGAN learns from existing musical sequences, commonly represented using MIDI or piano-roll data.
2. Generation: The Generator converts random latent vectors into musical patterns.
3. Multi-track creation: It produces separate tracks for instruments such as piano, bass, and drums.
4. Evaluation: The Discriminator learns to distinguish real musical patterns from generated patterns.
5. Training: Both networks improve through adversarial learning until the Generator produces more realistic music.
6. Output: The generated musical tracks can be converted into MIDI and played using suitable synthesizer software.