Innovation Design Engineering (MA/MSc)
China Academy of Art, Hangzhou, China, 09/2013-06/2018
Major: Architecture (Architecture and Art)
Degree: Bachelor of Engineering
Royal College of Art, Imperial College, London, the United Kingdom, 09/2018-08/2020
Major: Innovation Design Engineering
Degree: Master of Art; Master of Science
A dreamer who dreams to bring joy to people.
A designer who wants to convey the beauty and tenderness to the world.
Bardsong — Bardsong helps people discover the inspirational potential of soundscapes in real-time and helps generate context-aware music according to the true audio environment without previous music knowledge.
Sound Separation — Bardsong separates the sound into 3 parts in real-time. Foreground, a hi-fi sound that can be recognized clearly or contains certain specific content. Background, which forms the ambient of a certain area or a certain sound, used to contrast other sounds. and Instrument sample, which is a short signal and can be used as the timbre of midi. All of them will be saved as raw material for future music creation.
We seldom appreciate what we hear to the same extent as what we see, using cameras to record their lives, but few people have the habit of recording sound. On the other hand, the main functions of current audio products related to environmental sounds, are warnings, voice enhancement, and noise reduction. They tend to be more functional while ignoring the sensitivity of listening.
In the future, our earphones may be in the form of bone conduction, our ears are completely liberated in the environment. We can hear the audio content and the ambient sound at the same time. How will our audio experience change at that time?
Wearable — The binaural microphone records the sound environment in a way that is similar to our actual listening experience. People can also take it off and place it wherever they want to record sound.
Attachable — Many people have bought high-quality headphones. The pair of Bardsong microphones are designed to be attached to the products people are already using. The earbuds component is offered separately.
The wearable microphone module allows people to record their audio environment.
Discover — People can preset what they want to listen and how it’s going to integrate into their music. And then, the app will analyse the acoustic environment and separate them into sound materials.
Rapid prototype & Sound map — The rapid prototype allows people to do some quick editing, brings music creation outside to the studio. Involving the power of sharing could attract more participation in soundscape practice. The randomness of sound in different places and different cultures is the main motivation for people to explore soundscapes.
The APP analyses the soundscape and generates context-aware music in real-time.
Rhythm & Gesture — Participants can manipulate the direction of the soundwave in the acoustic environment play them in different directions.
Timbre & Stick — The stick will extract the texture by pointing to the environment, and generate timbre from it. Participants can use any sound around to create a beat.
Sound materials & Eye tracking — The green dot is an eye-tracking point, where participants seeing will generate a sound relate to the visual and mix into the music.
Frequency & App — The participants can filter the music and sounds based on the spectrogram in the app, and combine them together.
Moreover, too many active interactions will disturb people's sensitivity to sound itself. Therefore, we have reduced the action/operation as much as possible, and let people experience the soundscape passively with the help of machine learning.
The third stage is a transparent listening product which helps you an optimized acoustic environment for everyone And it could be applicated to more fields.
For example, the accessories of augmented reality headset and glasses. When playing games, the ambient sounds could be modified to match the emotions of the game, to create a more scary or cheerful background sound. This project potentially brings people a "film-like" experience in the real world.
Also, existing hearing aids focus on the enhancement of specific sounds such as human voices and TV sounds while ignoring the desire for the actual auditory experience of the hearing impaired. This technology may bring a more realistic and personalized listening experience to people with hearing impaired.