MUSIC-DRIVEN SYSTEM FOR GROUP DANCE FORMATION GENERATION

​

25-26J-358

“SyncForm”

MEMBERS

Supervisor

Ms. Jenny Krishara

​

​

Co .Supervisor

Ms. Dinuka Wijendra

​

​

External Supervisor

Mr. Shakitha Kanchana

​

25-26J-358

7/9/2025

02

MEMBERS

Team Leader

Dissanayake D.M.S.A.B

IT22595294

Uthpalani A.K.M.

IT22327444

Attanayake J.S

IT22911230

Samarakoon K.K

IT22346940

Overall system Diagram

7/9/2025

03

MUSIC FEATURE EXTRACTION & SEGMENTATION

Proven Gap / Creative Solution

​

​

​

​

• Gap: Current systems sync dancer moves with music but ignore group formations.

​

• Solution: Use music features + beat detection → generate cues for both moves and formations.

​

What Gap Are We Addressing

FEATURE COMPARISON

KEY PILLARS & TECHNOLOGIES

Key Pillars

Technologies

• Music Information Retrieval(MIR): Extract rhythm, beats,

song parts (verses, choruses)

​

• Deep Audio Embeddings

Capture music mood, energy,

and rhythm context

​

• Machine Learning

• Conformer & OpenL3

Capture rhythm + long-term

music structure

​

• Better than Librosa-only

approaches

OVERVIEW OF THE COMPONENT

HOW TO USE IT IN A REAL-WORLD SCENARIO

Real-World Workflow

​

• Choreographer uploads a song
• System auto-generates a segmented timeline
• Used for formation design

​

J.S. ATTANAYAKE

IT22911230

BSC (HONS) DEGREE IN INFORMATION TECHNOLOGY (SPECIALIZATION IN INFORMATION TECHNOLOGY)

FORMATION LAYOUT PLANNING

Proven Gap / Creative Solution

​

​

​

​

​

• Gap: No system auto-generates formations from music; manual tools are slow.
• Solution: Use music-conditioned generative models to create 2D formations, cutting choreographer effort.

​

What Gap Are We Addressing

FEATURE COMPARISON

KEY PILLARS & TECHNOLOGIES

Key Pillars

Technologies

• Machine Learning : Create novel, musically consistent formations

​

• Deep Learning : Map music embeddings to stage formations

​

• Conditional VAE : Generates formations conditioned on audio embeddings

​

• Audio Embeddings (from M1) : Capture style, rhythm, and phrase segmentation

​

HOW WE MEASURE SUCCESS

How to Evaluate / Prove Success

Data Availability

• Expert Validation : Choreographers rate formations (balance, spacing, visual appeal)
• Quantitative Checks : Symmetry, center-bias, spacing variance

​

• Converted AIST++ : Extract pelvis joint (2D coordinates) for dancer positions
• Custom Dataset : Formation references from public performances

​

OVERVIEW OF THE COMPONENT

HOW TO USE IT IN A REAL-WORLD SCENARIO

​

• A choreographer uploads a song.
• System automatically suggests multiple formation layouts for different sections of the music.
• Choreographer reviews the top-ranked options (already filtered for feasibility), saving hours of manual arrangement.

​

​

FORMATION TRANSITION PLANNING & INTERPOLATION

Proven Gap / Creative Solution

Even if formations look valid, transitions can cause collisions, long travel, or poor timing with music.

• Collisions between dancers
• Distances too far for one beat
• Timing mismatches

We use multi-agent path planning to make transitions smooth and realistic.

What Gap Are We Addressing

FEATURE COMPARISON

KEY PILLARS & TECHNOLOGIES

Key Pillars

Technologies

Transitions are planned using structured principles.

• Multi-Agent Systems → each dancer as an agent with rules
• Reinforcement Learning → adaptive and smooth strategies

​

We use algorithms to ensure safe and synchronized transitions.

• Hungarian Assignment → maps dancers to positions
• Conflict-Based Search (CBS) → plans collision - free paths

​

HOW WE MEASURE SUCCESS

We check both technical measures and expert feedback.

• Collision rate → aim for ~100% collision-free transitions
• Beat alignment
• Expert validation → choreographers confirm realism and comfort

We rely on existing datasets and custom annotations.

• AIST++ dataset
• Custom dataset → transitions labeled with choreographer input

​

How to Evaluate / Prove Success

Data Availability

OVERVIEW OF THE COMPONENT

HOW TO USE IT IN A REAL-WORLD SCENARIO

The system generates a complete transition plan for dancers.

• Waypoints and timestamps for each dancer
• Simulation for choreographer review
• Export to animation or visualization tools
• Usable as a blueprint in real rehearsals

​

FORMATION STYLE CONTROL & EVALUATION

Knowledge Gap / Problem Definition

Dance formation evaluation is subjective, leading to inconsistency and lack of fairness.

• No standardized evaluation framework.
• Relies only on choreographer judgment.

​

Problem: No objective system exists to evaluate formations consistently.

Proven Gap / Creative Solution

Creative Solution - Formation Quality Index (FQI)

• Rule-based metrics for symmetry, spacing, balance.
• ML classifier → detects hip-hop styles (popping, locking, House).

What Gap Are We Addressing

FEATURE COMPARISON

KEY PILLARS & TECHNOLOGIES

Key Pillars

Technologies

Our component combines measurable rules with intelligent style recognition.

​

• Rule-Based Evaluation → Use measurable metrics to judge formation quality.
• Machine Learning → classifies hip-hop formation styles (popping, locking, breaking, krumping)

​

Convolutional Neural Network (CNN)

Learns to recognize hip-hop formation styles from 2D spatial layouts.

• CNN analyzes dancer positions and structures.
• Detects styles like popping, locking, breaking, krumping.

Rule-Based Metrics

Provides transparent, mathematical measures of formation quality.

• Symmetry ,Spacing uniformity, Center bias, Balance

OVERVIEW OF THE COMPONENT

HOW WE MEASURE SUCCESS

How to Evaluate / Prove Success

Data Availability

​

System accuracy is tested by comparing outputs with expert judgment.

• Match FQI + ML results against choreographer ratings.
• Measure accuracy and consistency of evaluations.
• Test across multiple hip-hop styles.

​

Datasets are curated and anonymized to ensure both quality and privacy.

• AIST++ dataset →

hip - hop dance labels.

• Custom dataset → choreographer - annotated ratings.

​

HOW TO USE IT IN A REAL-WORLD SCENARIO

In Real Use

• Choreographers get ranked formation options.
• Pick best quality/style match.
• Saves time, reduces bias, supports creativity.

​

COMMERCIALIZATION & BUSINESS POTENTIAL

Appendix

THANK YOU!

This way, we move from a slow manual process to an automated, data-driven one

letting choreographers focus more on creativity