A Human-Computer Duet System for Music Performance

In spite of the attractiveness of virtual musicians, most of them cannot participate in jointly with human musicians subsequent their tempo or make their possess behaviors without having the help of human figures. The authors of a latest paper developed a virtual violinist possessing these qualities.

Graphic credit score: 刘睿忱 through Wikimedia (CC BY-SA 4.)

It can keep track of tunes and adapt to the human pianist’s tempo different with time and with overall performance, making the two voices harmonized. The virtual musician’s entire body actions are produced specifically from the tunes. The movement generator is skilled on a tunes video dataset of violin overall performance and a pose sequence synchronized with live overall performance is produced.

These options necessarily mean that the human musician can observe, rehearse, and conduct tunes with the virtual musician like with a actual human, by subsequent the tunes content. The proposed program has successfully performed in a ticket-offering live performance, where a movement from Beethoven’s Spring Sonata was played. 

Digital musicians have grow to be a impressive phenomenon in the modern multimedia arts. Nevertheless, most of the virtual musicians presently have not been endowed with skills to make their possess behaviors, or to conduct tunes with human musicians. In this paper, we for starters make a virtual violinist, who can collaborate with a human pianist to conduct chamber tunes routinely without having any intervention. The program incorporates the approaches from many fields, like actual-time tunes tracking, pose estimation, and entire body movement generation. In our program, the virtual musician’s habits is produced dependent on the supplied tunes audio alone, and such a program benefits in a low-price tag, successful and scalable way to create human and virtual musicians’ co-overall performance. The proposed program has been validated in public concert events. Aim high quality assessment strategies and possible strategies to systematically increase the program are also talked over.

Url: https://arxiv.org/stomach muscles/2009.07816