ARTS 4880: Interdisciplinary Research Seminar - Music & AI

Arts Department, Rensselaer Polytechnic Institute

Tuesday/Friday, 12:00-1:50am, SAGE 2510

Instructor: Rob Hamilton, West Hall 305/6

e: hamilr4 [at] rpi [dot] edu


Office Hours: TBD

COURSE SYLLABUS - Spring, 2024


ARTS 2380 or ARTS 2020, or permission of instructor


Advances in machine learning and artificially-intelligent systems are currently being applied to creative tasks such as musical composition and performance. This course is an advanced seminar focusing on the current state of AI research and application in musical domains. Topics to be discussed include the ethics of AI's use in music, the way(s) that trained systems are being applied towards the creation and performance of music and the hands-on application of AI/ML toolsets and frameworks for musical expression.


This is a course introducing music majors to advanced research topics of the Rensselaer music faculty. Each semester a member of the music faculty will focus the seminar on a research topic or paradigm related to their own body of artistic and technological research. Sample topics might include Spatial music and sound, New Instrument Design, Network Music, Music Information Retrieval, Ethnomusicology, Sonification Art and Science, Music and Logic, Spectralism and Beyond, Music Herstory (feminist music composition), Experimental music and sound history. Through hands-on creative research, students will explore questions of both musical and technological significance while engaging that same topic through their own hands-on creative practice.


    Students who successfully complete this course will demonstrate...
  1. an understanding and appreciation of the current statue of AI/ML systems for musical creation
  2. basic technical facility in the application of current tools and frameworks for AI/ML assisted composition, source separation, classification, sound synthesis and post production.
  3. creativity and resourcefulness through the creation of musical AI/ML systems and creation/composition of their own sonic projects


Evaluation is based on the following:


You will be required to present some of your assignments to the class, to show your work within the software environment you used to create it, and to engage the class in discussion of your work. When you are not presenting your own work, you need to be attentive to whoever is presenting, and to engage them in discussion of their work. Failure to participate in class will lower your grade.


You must attend class to succeed in this course.

  1. Since much of the class is focused on listening to and discussing work in class, attendance is mandatory.
  2. ** More then two unexcused absences will affect your grade, detracting 1/2 grade each additional 2 unexcused absences. **
  3. Late arrivals are very disruptive - continued late arrival will affect your grade.
  4. It should go without saying but no use of mobile devices or personal computers during class time (except for as required by the coursework itself) is acceptable. Continued violations will be treated as an unexcused absence.


Collaboration between students in this course is strongly encouraged. Likewise, students are encouraged—indeed, to some extent required—to exchange ideas, opinions and information . You are also encouraged to help each other in the lab and with performance, production, and presentation of composition projects.

Plagiarism of any kind is in direct violation of University policy on Academic Dishonesty as defined in the Rensselaer Handbook, and penalties for plagiarism can be severe. In this class you will be expected to attribute due credit to the originator of any ideas, words, sounds, or music which you incorporate substantially into your own work. This applies particularly to citation of sources for sonic "samples" included in your compositions.

The use of automated technical aids (e.g. ChatBots, AI code plugins) is not strictly forbidden but if used MUST be documented in great detail and discussed with the Professor prior to use in a graded project/assignment.

Submission of any assignment that is in violation of this policy may result in a grade of F for the assignment in question. Violation of this policy will be reported, as defined in the Rensselaer Handbook


Students requiring assistance are encouraged to contact Disability Services: to discuss any special accommodations or needs for this course.


Office Hours for Spring, 2024 will be TBD



The proposed course topics and schedule will be as follows (take note of project due dates!). Based on class progress and interests, this schedule is subject to change. Special topics, guest lectures, supplemental reading, listening and additional assignments to be announced.

Week 1:
Friday, 1/12

Algorithmic Composition: David Cope and EMI

Voice before, and in the age of AI

THE VODER @ Bell Labs
Long (audio + images):
Short (video):

Who owns a song?
Fogerty vs. Fantasy
Blurred Lines: Thicke/Pharrell vs. Estate of Marvin Gaye

Who owns a voice?

Midler vs Ford:
Ford commercial:
Midler version:

Waits vs Frito-Lay:

Week 2:
Friday, 1/19

Due: Tone Transfer Sketch

Simple interfacers/instruments for "public" use: Ocarina by Smule:
Feedback loop - Alvin Lucier's "I Am Sitting In A Room":

Week 3:
Tuesday, 1/23

Machine Learning and Music
George Lewis: Too Many Notes: Complexity and Culture in Voyager (2000)
George Lewis demonstrates Voyager:
George Lewis "Why Do We Want Our Computers To Improvise" (Talk):

Friday, 1/26

Fun with ChucK:
* Joystick Chanting: Joy of Chant
* ChucK AI: ChAI, AI Tools
* Ge Wang (2/16): CCRMA Homepage

Chuck/Markov example:
More on Markov Processes: Simoni: Algorithmic Composition Using Probabilistic Methods
Lisp Markov Example: markov.lisp

Rebecca Fiebrink's WEKINATOR

For class on Tuesday (1/30): Select one AIMC paper and present it to the class
* AIMC Archives:

Week 4
Friday, 2/2


Reembodied Sound Symposium @ EMPAC:

Week 5
Tuesday, 2/6

Student AIMC paper presentations pt. 1

Friday, 2/9

Student AIMC paper presentation pt. 2

Week 6
Tuesday, 2/13

ChAI (Chuck AI) Overview

Friday, 2/16

Ge Wang Visit: CCRMA Homepage

ChucK Community Discord:

Week 7
Tuesday, 2/20

NO CLASS (Presidant's Day Observed)

Friday, 2/23

Student Project Proposal Presentations pt. 1

Week 8
Tuesday, 2/27

Student Project Proposal Presentations pt. 2

Friday, 3/1

Preliminary Musical AI & Ethics discussion (EASE intro)

Week 9
Tuesday, 3/5


Friday, 3/8


Week 10
Tuesday, 3/12

EASE Demo/Workshop in class

Week 11
Friday, 3/22

Guest Speaker: Anna-Kaisa Kaila, PhD Candidate, Creative AI Researcher
KTH Royal Institute of Technology, Media Technology and Interaction Design

Week 12
Tuesday, 3/26

Prompt-interfaces for Music, Language and LLMs

Friday, 3/29

Prompt-interfaces for Music (continued)


Please complete the following assignment and bring your work to class on Friday to share and discuss:

Given the propensity of prompt-based interfaces for contemporary AI systems, a deeper look at how language is being used in musical AI feels important. For class on Friday, please create an account on one of the following AI Music services (you can use a "burner" email account if you wish) and investigate how that systems converts your human language prompt to criteria for Generative music creation.

- write a one or two sentence description of your goal musical output, this is not necessarily the language you will use for a prompt but instead the language you'd use to communicate to your classmates what it is you want.
- use the prompt interface of the service you're using to try to make your desired output.
- Document your successes, failures, and each step of the process (like you would do for a mathematical proof). Of particular interest are the changes you make to your prompts to make your output "better" or more in line with your target goal. So for each prompt you share with the class, there should be a generated audio output.

You can use any musical AI site/app out there, but here are a few choices that seem to have free tier usages and some level of credibility around them:

Week 13
Tuesday, 4/2

The disturbing complexities of AI voice recreation:
Suno pop song output:
Peter Jackson's Beatles Documentary: Audio Cleanup -
New Scientist Article: Beatles Documentary AI Cleanup

Audio Watermarking, How to tell if something was AI Generated (text, images, sound)

Friday, 4/5

One on One Project meetings: sign-up sheet

Week 14
Tuesday, 4/9

One on One Project meetings: sign-up sheet

Friday, 4/12

Guest Speaker: Kosmas Giannoutakis
Ph.D. Candidate, Rensselaer Polytechnic Institute
Film: "Nick Land's Meltdown AI-fication"

Week 15
Tuesday, 4/16

Guest Speaker: Spencer Salazar, Ph.D.
Principal Engineer @Output

Friday, 4/19

Class Presentations

Week 16
Tuesday, 4/23

Class Presentations (continued)