My work is very inter-disciplinary and often cuts across domains. Thus, it is often hard to put the work into discrete buckets. Still, in case you are interested in learning about my work in a particular area, we have tried to categorize the work into four broad categories: Health Sensing, Activity Recognition, Privacy-aware Sensing, New Interfaces, and Situational Impairments. Many publications are in overlapping areas and you will find those publications in multiple categories.
2024
PrISM-Observer: Intervention Agent to Help Users Perform Everyday Procedures Sensed using a Smartwatch
Bring Privacy To The Table: Interactive Negotiation for Privacy Settings of Shared Sensing Devices
EITPose: Wearable and Practical Electrical Impedance Tomography for Continuous Hand Pose Estimation
PDF Video
Kirigami: Lightweight Speech Filtering for Privacy-Preserving Activity Recognition using Audio
JoulesEye: Energy Expenditure Estimation and Respiration Sensing From Thermal Imagery While Exercising
PDF Video
2023
VAX: Using Existing Video and Audio-based Activity Recognition Models to Bootstrap Privacy-Sensitive Sensors
MI-Poser: Human Body Pose Tracking using Magnetic and Inertial Sensor Fusion with Metal Interference Mitigation
PDF Video
LemurDx: Using Unconstrained Passive Sensing for an Objective Measurement of Hyperactivity in Children with no Parent Input
uKnit: A Position-Aware Reconfigurable Machine-Knitted Wearable for Gestural Interaction and Passive Sensing using Electrical Impedance Tomography
PDF Video
IMUPoser: Full-Body Pose Estimation using IMUs in Phones, Watches, and Earbuds.
PDF Video Code
PrISM-Tracker: A Framework for Multimodal Procedure Tracking Using Wearable Sensors and State Transition Information with User-Driven Handling of Errors and Uncertainty
2022
A semantic-based approach to digital content placement for immersive environments
RGBDGaze: Gaze Tracking on Smartphones with RGB and Depth Data
PDF Video
SAMoSA: Sensing Activities with Motion and Subsampled Audio
PDF Video
Objective Measurement of Hyperactivity Using Mobile Sensing and Machine Learning: A Pilot Study
FitNibble: A Field Study to Evaluate the Utility and Usability of Automatic Diet Monitoring in Food Journaling Using an Eyeglasses-based Wearable.
PDF Video Code
2021
IMU2Doppler: Cross-Modal Domain Adaptation for Doppler-based Activity Recognition Using IMU Data
PDF Video
EyeMU Interactions: Gaze + IMU Gestures on Mobile Devices
PDF Video
Vid2Doppler: Synthesizing Doppler Radar Data from Videos for Training Privacy-Preserving Activity Recognition
PDF Video
Pose-on-the-Go: Approximating User Pose with Smartphone Sensor Fusion and Inverse Kinematics
PDF Video
PrivacyMic: Utilizing Inaudible Frequencies for Privacy Preserving Daily Activity Recognition
2020
Detecting Depression and Predicting its Onset Using Longitudinal Symptoms Captured by Passive Sensing: A Machine Learning Approach With Robust Feature Selection.
BodySLAM: Opportunistic User Digitization in Multi-User AR/VR Experiences
PDF Video
Direction-of-Voice (DoV) Estimation for Intuitive Speech Interaction with Smart Devices Ecosystems
PDF Video
Eyes on the Road: Detecting Phone Usage by Drivers Using On-Device Cameras
PDF Video
Digital Ventriloquism: Giving Voice to Everyday Objects
PDF Video
FitByte: Automatic Diet Monitoring in Unconstrained Situations Using Multimodal Sensing on Eyeglasses.
PDF Video Dataset
2019
MeCap: Whole-Body Digitization for Low-Cost VR/AR Headsets
PDF Video
LightAnchors: Appropriating Point Lights for Spatially-Anchored Augmented Reality Interfaces
PDF Video
Detachable Smartwatch: More Than A Wearable
PDF Video
Interferi: Gesture Sensing using On-Body Acoustic Interferometry
PDF Video
BeamBand: Hand Gesture Sensing with Ultrasonic Beamforming
PDF Video
GymCam: Detecting, recognizing, and tracking simultaneous exercises in unconstrained scenes
PDF Video
Challenges in Realizing Smartphone-based Health Sensing
2018
RainCheck: Overcoming Capacitive Interference Caused by Rainwater on Smartphones
Ubicoustics: Plug-and-Play Acoustic Activity Recognition
PDF Video Code
EyeSpyVR: Interactive Eye Sensing Using Off-the-Shelf, Smartphone-Based VR Headsets
PDF Video
2017
Carpacio: Repurposing Capacitive Sensors to Distinguish Driver and Passenger Touches on In-Vehicle Screens
PDF Video
EarBit: using wearable sensors to detect eating episodes in unconstrained environments
DigiTouch: Reconfigurable Thumb-to-Finger Input and Text Entry on Head-mounted Displays
PDF Video
2016
Design and Learnability of Vortex Whistles for Managing Chronic Lung Function via Smartphones
SpiroCall: Measuring Lung Function over a Phone Call
PDF Video
2015
MagnifiSense: Inferring Device Interaction Using Wrist-worn Passive Magneto-inductive Sensors
HyperCam: Hyperspectral Imaging for Ubiquitous Computing Applications
PDF Video
Tongue-in-Cheek: Using Wireless Signals to Enable Non-Intrusive and Flexible Facial Gestures Detection
PDF Video
SwitchBack: Using Focus and Saccade Tracking to Guide Users’ Attention for Mobile Task Resumption
Wibreathe: Estimating respiration rate using wireless signals in natural settings in the home
2014
BiliCam: Using Mobile Phones to Monitor Newborn Jaundice
PDF Video
AirLink: Sharing Files Between Multiple Devices Using In-air Gestures
PDF Video
SurfaceLink: Using Inertial and Acoustic Sensing to Enable Multi-device Interaction on a Surface
PDF Video
2013
DopLink: Using the Doppler Effect for Multi-device Interaction
PDF Video
ContextType: Using Hand Posture Information to Improve Mobile Touch Screen Text Entry
2012
GripSense: Using Built-in Sensors to Detect Hand Posture and Pressure on Commodity Mobile Phones
PDF Video
SpiroSmart: Using a Microphone to Measure Lung Function on a Mobile Phone
WalkType: using accelerometer data to accomodate situational impairments in mobile touch screen text entry
Open data kit sensors: a sensor integration framework for android at the application-level
Open Data Kit Sensors: Mobile Data Collection with Wired and Wireless Sensors
Health Sensing Publications
JoulesEye: Energy Expenditure Estimation and Respiration Sensing From Thermal Imagery While Exercising
PDF Video
LemurDx: Using Unconstrained Passive Sensing for an Objective Measurement of Hyperactivity in Children with no Parent Input
uKnit: A Position-Aware Reconfigurable Machine-Knitted Wearable for Gestural Interaction and Passive Sensing using Electrical Impedance Tomography
PDF Video
PrISM-Tracker: A Framework for Multimodal Procedure Tracking Using Wearable Sensors and State Transition Information with User-Driven Handling of Errors and Uncertainty
Objective Measurement of Hyperactivity Using Mobile Sensing and Machine Learning: A Pilot Study
FitNibble: A Field Study to Evaluate the Utility and Usability of Automatic Diet Monitoring in Food Journaling Using an Eyeglasses-based Wearable.
PDF Video Code
Detecting Depression and Predicting its Onset Using Longitudinal Symptoms Captured by Passive Sensing: A Machine Learning Approach With Robust Feature Selection.
Understanding Client Support Strategies to Improve Clinical Outcomes in an Online Mental Health Intervention
PDF Video
FitByte: Automatic Diet Monitoring in Unconstrained Situations Using Multimodal Sensing on Eyeglasses.
PDF Video Dataset
GymCam: Detecting, recognizing, and tracking simultaneous exercises in unconstrained scenes
PDF Video
Challenges in Realizing Smartphone-based Health Sensing
Design and Learnability of Vortex Whistles for Managing Chronic Lung Function via Smartphones
SpiroCall: Measuring Lung Function over a Phone Call
PDF Video
HyperCam: Hyperspectral Imaging for Ubiquitous Computing Applications
PDF Video
Wibreathe: Estimating respiration rate using wireless signals in natural settings in the home
BiliCam: Using Mobile Phones to Monitor Newborn Jaundice
PDF Video
SpiroSmart: Using a Microphone to Measure Lung Function on a Mobile Phone
Open data kit sensors: a sensor integration framework for android at the application-level
Open Data Kit Sensors: Mobile Data Collection with Wired and Wireless Sensors
Privacy-Aware Sensing Publications
Bring Privacy To The Table: Interactive Negotiation for Privacy Settings of Shared Sensing Devices
Kirigami: Lightweight Speech Filtering for Privacy-Preserving Activity Recognition using Audio
VAX: Using Existing Video and Audio-based Activity Recognition Models to Bootstrap Privacy-Sensitive Sensors
SAMoSA: Sensing Activities with Motion and Subsampled Audio
PDF Video
IMU2Doppler: Cross-Modal Domain Adaptation for Doppler-based Activity Recognition Using IMU Data
PDF Video
Vid2Doppler: Synthesizing Doppler Radar Data from Videos for Training Privacy-Preserving Activity Recognition
PDF Video
PrivacyMic: Utilizing Inaudible Frequencies for Privacy Preserving Daily Activity Recognition
GymCam: Detecting, recognizing, and tracking simultaneous exercises in unconstrained scenes
PDF Video
Activity Recognition Publications
PrISM-Observer: Intervention Agent to Help Users Perform Everyday Procedures Sensed using a Smartwatch
VAX: Using Existing Video and Audio-based Activity Recognition Models to Bootstrap Privacy-Sensitive Sensors
IMUPoser: Full-Body Pose Estimation using IMUs in Phones, Watches, and Earbuds.
PDF Video Code
PrISM-Tracker: A Framework for Multimodal Procedure Tracking Using Wearable Sensors and State Transition Information with User-Driven Handling of Errors and Uncertainty
SAMoSA: Sensing Activities with Motion and Subsampled Audio
PDF Video
FitNibble: A Field Study to Evaluate the Utility and Usability of Automatic Diet Monitoring in Food Journaling Using an Eyeglasses-based Wearable.
PDF Video Code
IMU2Doppler: Cross-Modal Domain Adaptation for Doppler-based Activity Recognition Using IMU Data
PDF Video
Vid2Doppler: Synthesizing Doppler Radar Data from Videos for Training Privacy-Preserving Activity Recognition
PDF Video
PrivacyMic: Utilizing Inaudible Frequencies for Privacy Preserving Daily Activity Recognition
FitByte: Automatic Diet Monitoring in Unconstrained Situations Using Multimodal Sensing on Eyeglasses.
PDF Video Dataset
GymCam: Detecting, recognizing, and tracking simultaneous exercises in unconstrained scenes
PDF Video
Ubicoustics: Plug-and-Play Acoustic Activity Recognition
PDF Video Code
EarBit: using wearable sensors to detect eating episodes in unconstrained environments
MagnifiSense: Inferring Device Interaction Using Wrist-worn Passive Magneto-inductive Sensors
New Interfaces Publications
PrISM-Observer: Intervention Agent to Help Users Perform Everyday Procedures Sensed using a Smartwatch
EITPose: Wearable and Practical Electrical Impedance Tomography for Continuous Hand Pose Estimation
PDF Video
MI-Poser: Human Body Pose Tracking using Magnetic and Inertial Sensor Fusion with Metal Interference Mitigation
PDF Video
uKnit: A Position-Aware Reconfigurable Machine-Knitted Wearable for Gestural Interaction and Passive Sensing using Electrical Impedance Tomography
PDF Video
IMUPoser: Full-Body Pose Estimation using IMUs in Phones, Watches, and Earbuds.
PDF Video Code
A semantic-based approach to digital content placement for immersive environments
RGBDGaze: Gaze Tracking on Smartphones with RGB and Depth Data
PDF Video
TouchPose: Hand Pose Prediction, Depth Estimation, and Touch Classification from Capacitive Images
PDF Video
EyeMU Interactions: Gaze + IMU Gestures on Mobile Devices
PDF Video
Pose-on-the-Go: Approximating User Pose with Smartphone Sensor Fusion and Inverse Kinematics
PDF Video
Classroom Digital Twins with Instrumentation-Free Gaze Tracking
PDF Video
CoolMoves: User Motion Accentuation in Virtual Reality
BodySLAM: Opportunistic User Digitization in Multi-User AR/VR Experiences
PDF Video
Direction-of-Voice (DoV) Estimation for Intuitive Speech Interaction with Smart Devices Ecosystems
PDF Video
Eyes on the Road: Detecting Phone Usage by Drivers Using On-Device Cameras
PDF Video
Beyond the Prototype: Understanding the Challenge of Scaling Hardware Device Production
Digital Ventriloquism: Giving Voice to Everyday Objects
PDF Video
MeCap: Whole-Body Digitization for Low-Cost VR/AR Headsets
PDF Video
LightAnchors: Appropriating Point Lights for Spatially-Anchored Augmented Reality Interfaces
PDF Video
Detachable Smartwatch: More Than A Wearable
PDF Video
Interferi: Gesture Sensing using On-Body Acoustic Interferometry
PDF Video
BeamBand: Hand Gesture Sensing with Ultrasonic Beamforming
PDF Video
RainCheck: Overcoming Capacitive Interference Caused by Rainwater on Smartphones
In only 3 minutes: perceived exertion limits of smartwatch use
EyeSpyVR: Interactive Eye Sensing Using Off-the-Shelf, Smartphone-Based VR Headsets
PDF Video
DigiTouch: Reconfigurable Thumb-to-Finger Input and Text Entry on Head-mounted Displays
PDF Video
HyperCam: Hyperspectral Imaging for Ubiquitous Computing Applications
PDF Video
Tongue-in-Cheek: Using Wireless Signals to Enable Non-Intrusive and Flexible Facial Gestures Detection
PDF Video
SwitchBack: Using Focus and Saccade Tracking to Guide Users’ Attention for Mobile Task Resumption
AirLink: Sharing Files Between Multiple Devices Using In-air Gestures
PDF Video
SurfaceLink: Using Inertial and Acoustic Sensing to Enable Multi-device Interaction on a Surface
PDF Video
DopLink: Using the Doppler Effect for Multi-device Interaction
PDF Video
GripSense: Using Built-in Sensors to Detect Hand Posture and Pressure on Commodity Mobile Phones
PDF Video
Situational Impairments Publications
Eyes on the Road: Detecting Phone Usage by Drivers Using On-Device Cameras
PDF Video
Detachable Smartwatch: More Than A Wearable
PDF Video
RainCheck: Overcoming Capacitive Interference Caused by Rainwater on Smartphones
Carpacio: Repurposing Capacitive Sensors to Distinguish Driver and Passenger Touches on In-Vehicle Screens
PDF Video
Tongue-in-Cheek: Using Wireless Signals to Enable Non-Intrusive and Flexible Facial Gestures Detection
PDF Video
SwitchBack: Using Focus and Saccade Tracking to Guide Users’ Attention for Mobile Task Resumption
ContextType: Using Hand Posture Information to Improve Mobile Touch Screen Text Entry
GripSense: Using Built-in Sensors to Detect Hand Posture and Pressure on Commodity Mobile Phones
PDF Video
WalkType: using accelerometer data to accomodate situational impairments in mobile touch screen text entry