Loading...
Advancing Acoustic Sensing from the Laboratory to Real World: Theories, Applications, and Practical Considerations
Citations
Altmetric:
Abstract
With the proliferation of voice assistants, speakers and microphones are essential components in billions of smart devices that people interact with on a daily basis, such as smartphones, smart watches, smart speakers, home appliances, etc. This dissertation explores the transformation of these devices from simple audio tools into sophisticated acoustic radars, expanding their applications beyond basic audio playback and voice interactions to include gesture tracking, vital sign monitoring, and eye blink detection. We address fundamental technical challenges and practical considerations, which not only resolve existing system limitations but also facilitate the creation of new applications.
One major challenge in acoustic sensing is tracking multiple targets simultaneously due to the inherent nature of contact-free tracking. Signals reflected from multiple targets are mixed at the microphone, and thus, it is difficult to separate them to obtain the context information of each individual target. FM-Track pioneers in enabling contact-free multi-target tracking using acoustic signals. A signal model is introduced to characterize the location and motion status of targets by fusing the information from multiple dimensions (i.e., range, velocity, and angle of targets). Then a series of techniques are developed to separate signals reflected from multiple targets and accurately track each individual target. FM-Track can successfully differentiate two targets with a spacing as small as 1 cm.
Another significant challenge for acoustic sensing is the extremely limited sensing range, particularly for fine-grained activities due to weak signal reflections. LASense dramatically increases the sensing range for fine-grained human activities by introducing a virtual transceiver idea that purely leverages delicate signal processing techniques in software. LASense can significantly increase the sensing range of respiration monitoring from the state-of-the-art 2 m to 6 m, and enhance the sensing range for finger tapping and eye blink detection by 150% and 80%, respectively.
Additionally, this dissertation demonstrates how to apply acoustic sensing techniques to enable new applications, i.e., “listening” to your hand gestures using smart speakers. In SpeakerGesture, we develop a series of novel signal processing techniques and implement our system on two commodity smart speaker prototypes. SpeakerGesture can achieve over 90% accuracy in gesture recognition even when the user is 4 m away from the smart speaker and there is strong interference.
At last, this dissertation shares the experience and findings when transitioning acoustic sensing systems from laboratory settings to real-world environments. We identify multiple practical considerations that were not paid attention to in the research community and propose the corresponding solutions. The challenges include: (i) there exists annoying audible sound leakage caused by acoustic sensing; (ii) acoustic sensing actually affects music play and voice calls; (iii) acoustic sensing consumes a significant amount of power, degrading the battery life; (iv) real-world device mobility can fail acoustic sensing.
Type
Dissertation (Open Access)
Date
2024-09
Publisher
Degree
Advisors
License
License
http://creativecommons.org/licenses/by/4.0/