aQwertyOn

Fun with a computer keyboard plus sound/audio/music/video/bluetooth

​

​

What is aQwertyOn?

• A new approach to the QWERTY keyboard as a musical interface

​

• A modular API that could be incorporated into several MusEDLab projects

QWERTY as Musical Interface

• Map ASCII to notes or chords within a scale
• Horizontal / Vertical strumming
• Recording, i.e. video soundtrack

​

AQWERTYON API

• Easily map keys to trigger different sounds, as well as images and video clips
• Library for logging keystrokes, saving takes
• Functionality, datasets, and design elements that will be used across several MusEDLab projects

Noteflight →

​

​

​

​

Music Data Structures

Modes

1-to-1 - notes & chords

Density - mapped to higher level dimensions

Strings - articulation plus note; articulation envelope mapped into video opacity envelope

Puzzle Cards - Virtual Chunking (section points; echonest?)

Wind Quintet

Multitrack - mute/solo, pan, zoom

Rap Lyric integration & "beat 4" muting

Chord grid (hex-ish layout)

Applications in MusEDLab Projects

Play With Your Film Music (current demo)

• Keys trigger audio, mapped onto video playback timeline

​

Chamber Music Society of Detroit (piano duet)

• Articulation envelope mapped onto video opacity envelope of piano and/or string quartet
• String quartet from Juilliard
• MadPad-inspired interface
• Top pizz, bottom more sustained arco
• Yo-Yo Ma? Interactive map of the Silk Road…

​

UAP (lyrics sync)

• Remix triggers for music/sound effects and vocal samples
• EchoNest could analyze and return beats from user-created music
• EchoNest could also analyze other music and slice it up for easy beat-creation
• Drag & Drop samples or record your own
• smARTbomb has reverse engineered rap genius. try sending multitracks to echo nest and see what we get back

Applications in MusEDLab Projects

mk1000

• hold down key to map samples onto key
• music is color, speech is greyscale
• web-based audio editor with blocks, cross timing all set up
• record your voice saying the words
• login system for urban arts, track by student

​

smARTbomb / FreshEd / Tools for performing a song

• Echo Nest API is not good at inferring meter and downbeats
• press D for downbeat and it'll remap the whole thing
• press V for verse / C for chorus etc...then student is involved in doing form/metrical analysis

Modules

• UI - select take, select voice, scale/chord/root, vol, mode (arp/pattern/chord/note?)
• Video - ability to search for any video on youtube
• Sound
• Key Logging → hashed url
• Mobile / web multitouch hooks
• Timeline
• editable
• mute/unmute
• multitracks?
• play with mood / emoticon / sticker?
• Quantization →
• additional timeline UI for tempo/bpm
• File Export: MIDI, JSON, MP3, RT?
• Save take data to unique hash url / database
• QWERTY ghost display / Lyrics / Images to display on keypress
• Analyze keylog data to provide feedback
• Drag ‘n Drop (folders?)
• Database
• user profiles (student, teacher), classroom
• takes

Data to Visualize Creative Process

• Increment listens per take
• Improv vs recording time
• Add this data to the unique hash URL
• What time are they using? Hours per week. Correlate with how they’re doing in school over the same time **success**

​

​

Design

• Timeline shows melodic contour, lights up notes as they play
• Qwerty ghost display: outline of keyboard. Don't see grid of keys. Just see letter come up at appropriate place.