28 July 2022
July is truly a joyous time of the year. While 1 July marked the 25th anniversary of the establishment of the HKSAR and a new chapter for its Government, 14 July signified an important milestone for our multidisciplinary research team led by Professor Guo Yike, Vice-President (Research and Development), and Professor Johnny Poon, Associate Vice-President (Interdisciplinary Research) and Founding Dean of the School of Creative Arts, as witnessed by an audience of 800, they successfully staged an innovative human-machine collaboration between the HKBU Symphony Orchestra and our AI super artists at the Annual Gala Concert.
We all realise that the arts and culture in all forms, from traditional practices to new media, can soothe our minds and enrich our souls. However, when our artists and scientists meet at HKBU, amazing things happen. In this year’s Gala Concert, dubbed “A Lovers’ Reunion”, apart from the melodious classical music performed by our award-winning students with the Orchestra, the audience was enchanted by a highly expressive performance that involved an AI virtual choir, AI virtual dancers and an AI media artist as well as the Orchestra. The innovative concert was the first of its kind in the world, and it is an initial output of the art-tech project entitled “Building Platform Technologies for Symbiotic Creativity in Hong Kong”, a five-year initiative that received HK$52.8 million in funding last year from the Theme-based Research Scheme under the Research Grants Council.
To prepare for this performance, our researchers from HKBU’s Augmented Creativity Lab trained the AI “kids” night and day from scratch. Over the course of a few months, these AI “kids” became AI super artists. After learning from a comprehensive art data repository, they could sing, draw and dance based on the music and lyrics that they “heard”. The AI virtual choir, which sang the song Pearl of the Orient with the voices of 320 virtual performers, was trained on the generative models of singing developed by the team, and they achieved this by extracting and disentangling key contributing features of vocal singing from a collection of songs recorded by professional singers.
At the same time, the AI media artist created a stunning cross-media visual narrative of the song according to its interpretation of the underlying meaning of the lyrics. The visuals changed along with the melody and the lyrics, giving the audience an immersive cross-media experience to remember. Unlike the conventional AI machines that import images as a reference for the algorithms to mimic, the AI media artist used textual lyrics as the sole input to create the vivid artwork, showcasing how visual storytelling has evolved in the age of AI. But the story doesn’t end there. When sharing the stage with the students in the Orchestra, the AI artists had to interpret signals and instructions from Professor Johnny Poon, our conductor, who put on sensors so that the AI artists could read his movements. While our scientists are creative, our artists are scientific!
Accompanied live by the Orchestra, another highlight of the Concert was a ballet performance by the AI virtual dancers to Ravel’s Daphnis et Chloé. With the help of professional dancers from the Hong Kong Dance Company, the AI virtual dancers learned to interpret the underlying emotional and aesthetic connections between the music and the dance. They then choreographed the dance movements, which were inspired by a newly discovered species of box jellyfish in Hong Kong.
This is a wonderful breakthrough in terms of the use of AI to create an AI-augmented performance that involved not only humans and machines working together, but also various forms of the arts, and it was a spectacular showcase of how HKBU is using technology to push the envelope of human imagination in the arts and cultural sphere. Please join me in giving the performers and the researchers a standing ovation after reading how the super AI artists were trained, and watching the highlights of the HKBU Symphony Orchestra Annual Gala Concert.
On 9 August, we will be holding the Human, Machine, Art, Creativity: International Symposium at the Hong Kong Convention and Exhibition Centre. Renowned researchers from academia and industry will gather at this international symposium to jointly discuss emerging topics in human-machine symbiotic art creativity. With the combined efforts of our artists and scientists, they will unlock endless possibilities in art creation and take us to a higher plane in art appreciation. Their insightful thoughts on the topic will definitely light the way forward and take the worldwide development of art-tech to new heights. We will also launch the Turing AI Orchestra, a decentralised and international platform for collecting art data, gathering artists, and staging AI orchestral performances. We imagine that very soon we will be able to enjoy a wide array of performances with the input of artists from all over the world, and the best artistic creations anytime, anywhere!
The Gala Concert is just one of the many examples of the fascinating research work going on at HKBU, and how the transdisciplinary environment inspires both artists and scientists. Other scholars at HKBU are also working on a variety of art-tech projects, and some have already been seen by the public. As our artists and scientists continue to develop new technologies to unveil the unlimited potential of human-machine symbiotic creativity, HKBU will shape the landscape in which works of art are created, shared and appreciated, establishing Hong Kong as an art-tech leader on the global stage.