Innovation Design Engineering (MA/MSc)
Yiyang Shen
China Academy of Art, Hangzhou, China, 09/2013-06/2018
Major: Architecture (Architecture and Art)
Degree: Bachelor of Engineering
Royal College of Art, Imperial College, London, the United Kingdom, 09/2018-08/2020
Major: Innovation Design Engineering
Degree: Master of Art; Master of Science
A dreamer who dreams to bring joy to people.
A designer who wants to convey the beauty and tenderness to the world.
Bardsong — Bardsong helps people discover the inspirational potential of soundscapes in real-time and helps generate context-aware music according to the true audio environment without previous music knowledge.
Sound Separation — Bardsong separates the sound into 3 parts in real-time. Foreground, a hi-fi sound that can be recognized clearly or contains certain specific content. Background, which forms the ambient of a certain area or a certain sound, used to contrast other sounds. and Instrument sample, which is a short signal and can be used as the timbre of midi. All of them will be saved as raw material for future music creation.
A soundscape is a sound or combination of sounds that are formed from a specific location. It dynamically records the surroundings and trigger emotions, playing a vital role in our lives, but we almost ignore the infinite beauty of it.
We seldom appreciate what we hear to the same extent as what we see, using cameras to record their lives, but few people have the habit of recording sound. On the other hand, the main functions of current audio products related to environmental sounds, are warnings, voice enhancement, and noise reduction. They tend to be more functional while ignoring the sensitivity of listening.
In the future, our earphones may be in the form of bone conduction, our ears are completely liberated in the environment. We can hear the audio content and the ambient sound at the same time. How will our audio experience change at that time?
We seldom appreciate what we hear to the same extent as what we see, using cameras to record their lives, but few people have the habit of recording sound. On the other hand, the main functions of current audio products related to environmental sounds, are warnings, voice enhancement, and noise reduction. They tend to be more functional while ignoring the sensitivity of listening.
In the future, our earphones may be in the form of bone conduction, our ears are completely liberated in the environment. We can hear the audio content and the ambient sound at the same time. How will our audio experience change at that time?
Bardsong — A soundscape is a sound or combination of sounds that are formed from a specific location. It dynamically records the surroundings and trigger emotions, playing a vital role in our lives, but we almost ignore the infinite beauty of it.
Inspired by public soundscape research, Bardsong provides an acoustic experience to promote public participation in soundscape listening. It is a product that connects the worlds inside and outside the headset and encourages people to treat environmental sound as a resource not to be wasted.
Bardsong helps people discover the inspirational potential of soundscapes in real-time and helps generate context-aware music according to the true audio environment without previous music knowledge.
Wearable — The binaural microphone records the sound environment in a way that is similar to our actual listening experience. People can also take it off and place it wherever they want to record sound.
Attachable — Many people have bought high-quality headphones. The pair of Bardsong microphones are designed to be attached to the products people are already using. The earbuds component is offered separately.
Bardsong has several elements, wearable microphone modules and a mobile application.
The wearable microphone module allows people to record their audio environment.
The wearable microphone module allows people to record their audio environment.
Discover — People can preset what they want to listen and how it’s going to integrate into their music. And then, the app will analyse the acoustic environment and separate them into sound materials.
Rapid prototype & Sound map — The rapid prototype allows people to do some quick editing, brings music creation outside to the studio. Involving the power of sharing could attract more participation in soundscape practice. The randomness of sound in different places and different cultures is the main motivation for people to explore soundscapes.
Bardsong has several elements, wearable microphone modules and a mobile application.
The APP analyses the soundscape and generates context-aware music in real-time.
The APP analyses the soundscape and generates context-aware music in real-time.
Rhythm & Gesture — Participants can manipulate the direction of the soundwave in the acoustic environment play them in different directions.
Timbre & Stick — The stick will extract the texture by pointing to the environment, and generate timbre from it. Participants can use any sound around to create a beat.
Sound materials & Eye tracking — The green dot is an eye-tracking point, where participants seeing will generate a sound relate to the visual and mix into the music.
Frequency & App — The participants can filter the music and sounds based on the spectrogram in the app, and combine them together.
For amateurs without previous knowledge of music, even if they were given the ability to extract sounds from the environment, they won't be able to create music in a short time. This is the main reason for stifling their enthusiasm for the music product.
Moreover, too many active interactions will disturb people's sensitivity to sound itself. Therefore, we have reduced the action/operation as much as possible, and let people experience the soundscape passively with the help of machine learning.
Moreover, too many active interactions will disturb people's sensitivity to sound itself. Therefore, we have reduced the action/operation as much as possible, and let people experience the soundscape passively with the help of machine learning.
Further Application
This product hopes to involve more people to engage with our acoustic environment through the concept of context-aware music social media. Through the experience, the product collects sound information and trains an optimized sound separation model. This is beneficial to the research and audio processing algorithm.
The third stage is a transparent listening product which helps you an optimized acoustic environment for everyone And it could be applicated to more fields.
For example, the accessories of augmented reality headset and glasses. When playing games, the ambient sounds could be modified to match the emotions of the game, to create a more scary or cheerful background sound. This project potentially brings people a "film-like" experience in the real world.
Also, existing hearing aids focus on the enhancement of specific sounds such as human voices and TV sounds while ignoring the desire for the actual auditory experience of the hearing impaired. This technology may bring a more realistic and personalized listening experience to people with hearing impaired.
The third stage is a transparent listening product which helps you an optimized acoustic environment for everyone And it could be applicated to more fields.
For example, the accessories of augmented reality headset and glasses. When playing games, the ambient sounds could be modified to match the emotions of the game, to create a more scary or cheerful background sound. This project potentially brings people a "film-like" experience in the real world.
Also, existing hearing aids focus on the enhancement of specific sounds such as human voices and TV sounds while ignoring the desire for the actual auditory experience of the hearing impaired. This technology may bring a more realistic and personalized listening experience to people with hearing impaired.