My work is very inter-disciplinary and often cuts across domains. Thus, it is often hard to put the work into discrete buckets. Still, in case you are interested in learning about my work in a particular area, we have tried to categorize the work into four broad categories: Health Sensing, Activity Recognition, Privacy-aware Sensing, New Interfaces, and Situational Impairments. Many publications are in overlapping areas and you will find those publications in multiple categories.

2024

 

PrISM-Q&A: Step-Aware Question Answering with Large Language Models Enabled by Multimodal Procedure Tracking using a Smartwatch.

Riku Arakawa, Jill Lehman, Mayank Goel

To appear in Proceedings of the ACM on Interactive, Mobile, Wearable, and Ubiquitous Technologies (IMWUT)

 

PrISM-Observer: Intervention Agent to Help Users Perform Everyday Procedures Sensed using a Smartwatch

Riku Arakawa, Hiromu Yakura, Mayank Goel

Proceedings of User Interface Software and Technology (UIST)
PDF Video

 

Perception of Indoor Lighting: A Comparative Study in Physical and Virtual Environments

Niloofar Nikookar, Azadeh Omidfar Sawyer, Ankita Sinha, Riya Malhotra, Siobhan Rockcastle, Mayank Goel

Illuminating Engineering Society
PDF

 

Bring Privacy To The Table: Interactive Negotiation for Privacy Settings of Shared Sensing Devices

Haozhe Zhou, Mayank Goel, Yuvraj Agarwal

Proceedings of the Annual ACM Conference on Human Factors in Computing Systems (CHI)
PDF

 

Investigating the Impact of Combined Daylight and Electric Light on Human Perception of Indoor Spaces

Niloofar Nikookar, Azadeh Omidfar Sawyer, Mayank Goel, Siobhan Rockcastle,

Sustainability
PDF

 

EITPose: Wearable and Practical Electrical Impedance Tomography for Continuous Hand Pose Estimation

Alexander Kyu, Hongyu Mao, Junyi Zhu, Mayank Goel, Karan Ahuja

Proceedings of the Annual ACM Conference on Human Factors in Computing Systems (CHI)
PDF Video

 

Kirigami: Lightweight Speech Filtering for Privacy-Preserving Activity Recognition using Audio

Sudershan Boovaraghavan, Haozhe Zhou, Mayank Goel, Yuvraj Agarwal

Proceedings of the ACM on Interactive, Mobile, Wearable, and Ubiquitous Technologies (IMWUT)
PDF

 

JoulesEye: Energy Expenditure Estimation and Respiration Sensing From Thermal Imagery While Exercising

Rishiraj Adhikary, Maite Sadeh, Nipun Batra, Mayank Goel

Proceedings of the ACM on Interactive, Mobile, Wearable, and Ubiquitous Technologies (IMWUT)
PDF Video

2023

 

VAX: Using Existing Video and Audio-based Activity Recognition Models to Bootstrap Privacy-Sensitive Sensors

Prasoon Patidar, Mayank Goel, Yuvraj Agarwal

Proceedings of the ACM on Interactive, Mobile, Wearable, and Ubiquitous Technologies (IMWUT)
PDF

 

MI-Poser: Human Body Pose Tracking using Magnetic and Inertial Sensor Fusion with Metal Interference Mitigation

Riku Arakawa, Bing Zhou, Gurunandan Krishnan, Mayank Goel, Shree K. Nayar

Proceedings of the ACM on Interactive, Mobile, Wearable, and Ubiquitous Technologies (IMWUT)
PDF Video

 

LemurDx: Using Unconstrained Passive Sensing for an Objective Measurement of Hyperactivity in Children with no Parent Input

Riku Arakawa, Karan Ahuja, Kristie Mak, Gwendolyn Thompson, Sam Shaaban, Oliver Lindhiem, Mayank Goel

Proceedings of the ACM on Interactive, Mobile, Wearable, and Ubiquitous Technologies (IMWUT)
PDF

 

uKnit: A Position-Aware Reconfigurable Machine-Knitted Wearable for Gestural Interaction and Passive Sensing using Electrical Impedance Tomography

Catherine Tianhong Yu, Riku Arakawa, James McCann, Mayank Goel

Proceedings of the Annual ACM Conference on Human Factors in Computing Systems (CHI)
PDF Video

 

IMUPoser: Full-Body Pose Estimation using IMUs in Phones, Watches, and Earbuds.

Vimal Mollyn, Riku Arakawa, Mayank Goel, Chris Harrison, Karan Ahuja

Proceedings of the Annual ACM Conference on Human Factors in Computing Systems (CHI) 2023, Honorable Mention Award
PDF Video Code

 

PrISM-Tracker: A Framework for Multimodal Procedure Tracking Using Wearable Sensors and State Transition Information with User-Driven Handling of Errors and Uncertainty

Riku Arakawa, Hiromu Yakura, Vimal Mollyn, Suzanne Nie, Emma Russell, Dustin Demeo, Haarika Reddy, Alexander Maytin, Bryan Carroll, Jill Fain Lehman, Mayank Goel

Proceedings of the ACM on Interactive, Mobile, Wearable, and Ubiquitous Technologies (IMWUT)
PDF

2022

 

A semantic-based approach to digital content placement for immersive environments

Jingyang Liu, Yunzhi Li, Mayank Goel

The Visual Computer 2022
PDF

 

RGBDGaze: Gaze Tracking on Smartphones with RGB and Depth Data

Riku Arakawa, Mayank Goel, Chris Harrison, Karan Ahuja

Proceedings of the International Conference on Multimodal Interaction (ICMI) 2022.
PDF Video

 

SAMoSA: Sensing Activities with Motion and Subsampled Audio

Vimal Mollyn, Karan Ahuja, Dhruv Verma, Chris Harrison, Mayank Goel

Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies (IMWUT) 2022.
PDF Video

 

Predicting Multiple Sclerosis Outcomes during the COVID-19 Stay-at-Home Period: Observational Study Using Passively Sensed Behaviors and Digital Phenotyping.

Prerna Chikersal, Shruthi Venkatesh, Karmen Masown, Elizabeth Walker, Danyal Quraishi, Anind Dey, Mayank Goel, Zongqi Xia

JMIR Mental Health 2022
PDF

 

Objective Measurement of Hyperactivity Using Mobile Sensing and Machine Learning: A Pilot Study

Oliver Lindhiem, Mayank Goel, Sam Shaaban, Kristie Mak, Prerna Chikersal, Jamie Feldman, Jordan Harris

JMIR Formative Research 2022
PDF

 

FitNibble: A Field Study to Evaluate the Utility and Usability of Automatic Diet Monitoring in Food Journaling Using an Eyeglasses-based Wearable.

Abdelkareem Bedri, Yuchen Liang, Sudershan Boovaraghavan, Geoff Kaufman, Mayank Goel

Proceedings of the Annual Conference on Intelligent User Interfaces (IUI) 2022
PDF Video Code

2021

 

IMU2Doppler: Cross-Modal Domain Adaptation for Doppler-based Activity Recognition Using IMU Data

Sejal Bhalla, Mayank Goel, Rushil Khurana

Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies (IMWUT) 2022
PDF Video

 

EyeMU Interactions: Gaze + IMU Gestures on Mobile Devices

Andy Kong, Karan Ahuja, Mayank Goel, Chris Harrison

Proceedings of the International Conference on Multimodal Interaction (ICMI) 2021
PDF Video

 

Vid2Doppler: Synthesizing Doppler Radar Data from Videos for Training Privacy-Preserving Activity Recognition

Karan Ahuja, Yue Jiang, Mayank Goel, Chris Harrison

Proceedings of the Annual ACM Conference on Human Factors in Computing Systems (CHI) 2021
PDF Video

 

Pose-on-the-Go: Approximating User Pose with Smartphone Sensor Fusion and Inverse Kinematics

Karan Ahuja, Sven Mayer, Mayank Goel, Chris Harrison

Proceedings of the Annual ACM Conference on Human Factors in Computing Systems (CHI) 2021
PDF Video

 

PrivacyMic: Utilizing Inaudible Frequencies for Privacy Preserving Daily Activity Recognition

Yasha Iravantchi, Karan Ahuja, Mayank Goel, Chris Harrison, Alanson Sample

Proceedings of the Annual ACM Conference on Human Factors in Computing Systems (CHI), Honorable Mention Award
PDF

2020

 

Detecting Depression and Predicting its Onset Using Longitudinal Symptoms Captured by Passive Sensing: A Machine Learning Approach With Robust Feature Selection.

Prerna Chikersal, Afsaneh Doryab, Michael Tumminia, Daniella K. Villalba, Janine M. Dutcher, Xinwen Liu, Sheldon Cohen, Kasey G. Creswell, Jennifer Mankoff, J. David Creswell, Mayank Goel, Anind Dey

ACM Transactions on Computer-Human Interaction (TOCHI) 2020
PDF

 

BodySLAM: Opportunistic User Digitization in Multi-User AR/VR Experiences

Karan Ahuja, Mayank Goel, Chris Harrison

Proceedings of SUI 2020
PDF Video

 

Direction-of-Voice (DoV) Estimation for Intuitive Speech Interaction with Smart Devices Ecosystems

Karan Ahuja, Andy Kong, Mayank Goel, Chris Harrison

Proceedings of UIST 2020
PDF Video

 

Eyes on the Road: Detecting Phone Usage by Drivers Using On-Device Cameras

Rushil Khurana, Mayank Goel

Proceedings of the Annual ACM Conference on Human Factors in Computing Systems (CHI) 2020
PDF Video

 

Digital Ventriloquism: Giving Voice to Everyday Objects

Yasha Iravantchi, Mayank Goel, Chris Harrison

Proceedings of the Annual ACM Conference on Human Factors in Computing Systems (CHI) 2020
PDF Video

 

FitByte: Automatic Diet Monitoring in Unconstrained Situations Using Multimodal Sensing on Eyeglasses.

Abdelkareem Bedri, Diana Li, Rushil Khurana, Kunal Bhuwalka, Mayank Goel

Proceedings of the Annual ACM Conference on Human Factors in Computing Systems (CHI) 2020
PDF Video Dataset

2019

 

MeCap: Whole-Body Digitization for Low-Cost VR/AR Headsets

Karan Ahuja, Chris Harrison, Mayank Goel, Robert Xiao

UIST 2019, Honorable Mention Award
PDF Video

 

LightAnchors: Appropriating Point Lights for Spatially-Anchored Augmented Reality Interfaces

Karan Ahuja, Sujeath Pareddy, Robert Xiao, Mayank Goel, Chris Harrison

Proceedings of UIST 2019
PDF Video

 

Detachable Smartwatch: More Than A Wearable

Rushil Khurana, Mayank Goel, Kent Lyons

Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies (IMWUT), 2019
PDF Video

 

Interferi: Gesture Sensing using On-Body Acoustic Interferometry

Yasha Iravantchi, Yang Zhang, Evi Bernitsas, Mayank Goel, Chris Harrison

Proceedings of the 2019 CHI Conference on Human Factors in Computing Systems (CHI ’19), 2019, Honorable Mention Award
PDF Video

 

BeamBand: Hand Gesture Sensing with Ultrasonic Beamforming

Yasha Iravantchi, Mayank Goel, Chris Harrison

Proceedings of the 2019 CHI Conference on Human Factors in Computing Systems (CHI ’19), 2019
PDF Video

 

GymCam: Detecting, recognizing, and tracking simultaneous exercises in unconstrained scenes

Rushil Khurana, Karan Ahuja, Zac Yu, Jennifer Mankoff, Chris Harrison, Mayank Goel

Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies (IMWUT), 2019
PDF Video

 

Challenges in Realizing Smartphone-based Health Sensing

Alex Mariakakis, Edward J. Wang, Shwetak Patel, Mayank Goel

IEEE Pervasive 2019, Best Paper Award Runner-Up
PDF

2018

 

RainCheck: Overcoming Capacitive Interference Caused by Rainwater on Smartphones

Ying-Chao Tung, Mayank Goel, Isaac Zinda, Jacob Wobbrock

International Conference on Multimodal Interaction (ICMI) 2018
PDF

 

Ubicoustics: Plug-and-Play Acoustic Activity Recognition

Gierad Laput, Karan Ahuja, Mayank Goel, Chris Harrison

Conference on Human Factors in Computing Systems (CHI), 2016
PDF Video Code

 

EyeSpyVR: Interactive Eye Sensing Using Off-the-Shelf, Smartphone-Based VR Headsets

Karan Ahuja, Rahul Islam, Varun Parashar, Kuntal Dey, Chris Harrison, Mayank Goel

Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies (IMWUT), 2018
PDF Video

2017

 

Carpacio: Repurposing Capacitive Sensors to Distinguish Driver and Passenger Touches on In-Vehicle Screens

Edward Wang, Jake Garrison, Eric Whitmire, Mayank Goel, Shwetak Patel

30th Annual ACM Symposium on User Interface Software and Technology (UIST), 2017
PDF Video

 

EarBit: using wearable sensors to detect eating episodes in unconstrained environments

Abdelkareem Bedri, Richard Li, Malcolm Haynes, Raj Prateek Kosaraju, Ishaan Grover, Temiloluwa Prioleau, Min Yan Beh, Mayank Goel, Thad Starner, Gregory Abowd

Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies (IMWUT), 2017
PDF

 

DigiTouch: Reconfigurable Thumb-to-Finger Input and Text Entry on Head-mounted Displays

Eric Whitmire, Mohit Jain, Divye Jain, Greg Nelson, Ravi Karkar, Shwetak Patel, Mayank Goel

Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies (IMWUT), 2017
PDF Video

 

Use of a smartphone app to assess neonatal jaundice

James A. Taylor, James W. Stout, Lilian de Greef, Mayank Goel, Shwetak Patel, Esther K. Chung, Aruna Koduri, Shawn McMahon, Jane Dickerson, Elizabeth A. Simpson, Eric Larson

Pediatrics Sep 2017
PDF Video

2016

 

Design and Learnability of Vortex Whistles for Managing Chronic Lung Function via Smartphones

Spencer Kaiser, Ashley Parks, Patrick Leopard, Charlie Albright, Jake Carlson, Mayank Goel, Damoun Nassehi, Eric Larson

Proceedings of the 2016 ACM International Joint Conference on Pervasive and Ubiquitous Computing (UbiComp '16)
PDF

 

SpiroCall: Measuring Lung Function over a Phone Call

Mayank Goel, Elliot Saba, Maia Stiber, Eric Whitmire, Josh Fromm, Eric Larson, Gaetano Borriello, Shwetak Patel

Conference on Human Factors in Computing Systems (CHI), 2016, Honorable Mention Award
PDF Video

2015

 

MagnifiSense: Inferring Device Interaction Using Wrist-worn Passive Magneto-inductive Sensors

Edward Wang, Tien-Jui Lee, Alex Mariakakis, Mayank Goel, Sidhant Gupta, Shwetak Patel

ACM International Joint Conference on Pervasive and Ubiquitous Computing (UbiComp), 2015
PDF

 

HyperCam: Hyperspectral Imaging for Ubiquitous Computing Applications

Mayank Goel, Eric Whitmire, Alex Mariakakis, Scott Saponas, Neel Joshi, Dan Morris, Brian Guenter, Marcel Gavriliu, Gaetano Borriello, Shwetak Patel

2015 ACM International Joint Conference on Pervasive and Ubiquitous Computing (UbiComp '15), Honorable Mention Award
PDF Video

 

Tongue-in-Cheek: Using Wireless Signals to Enable Non-Intrusive and Flexible Facial Gestures Detection

Mayank Goel, Chen Zhao, Ruth Vinisha, Shwetak Patel

Conference on Human Factors in Computing Systems (CHI), 2015, Honorable Mention Award
PDF Video

 

SwitchBack: Using Focus and Saccade Tracking to Guide Users’ Attention for Mobile Task Resumption

Alex Mariakakis, Mayank Goel, Md. Tanvir Islam Aumi, Shwetak Patel, Jacob Wobbrock

Conference on Human Factors in Computing Systems (CHI), 2015
PDF

 

Wibreathe: Estimating respiration rate using wireless signals in natural settings in the home

Ruth Vinisha Ravichandran, Elliot Saba, Ke-Yu Chen, Mayank Goel, Sidhant Gupta, Shwetak Patel

IEEE International Conference on Pervasive Computing and Communications (PerCom), 2015, Best Paper Runner-up
PDF

2014

 

BiliCam: Using Mobile Phones to Monitor Newborn Jaundice

Lilian de Greef, Mayank Goel, Min Joon Seo, Eric Larson, James W. Stout, James A. Taylor, Shwetak Patel

ACM International Joint Conference on Pervasive and Ubiquitous Computing (UbiComp), 2014, Honorable Mention Award
PDF Video

 

AirLink: Sharing Files Between Multiple Devices Using In-air Gestures

Ke-Yu Chen, Daniel Ashbrook, Mayank Goel, Sung-Hyuck Lee, Shwetak Patel

ACM International Joint Conference on Pervasive and Ubiquitous Computing (UbiComp) 2014
PDF Video

 

SurfaceLink: Using Inertial and Acoustic Sensing to Enable Multi-device Interaction on a Surface

Mayank Goel, Brendan Lee, Md. Tanvir Islam Aumi, Shwetak Patel, Gaetano Borriello, Stacie Hibino, Bo Begole

Conference on Human Factors in Computing Systems (CHI), 2014
PDF Video

2013

 

DopLink: Using the Doppler Effect for Multi-device Interaction

Md. Tanvir Islam Aumi, Sidhant Gupta, Mayank Goel, Eric Larson, Shwetak Patel

ACM International Joint Conference on Pervasive and Ubiquitous Computing (UbiComp), 2013
PDF Video

 

ContextType: Using Hand Posture Information to Improve Mobile Touch Screen Text Entry

Mayank Goel, Alex Jansen, Travis Mandel, Shwetak Patel, Jacob O. Wobbrock

Conference on Human Factors in Computing Systems (CHI), 2013
PDF

 

Tracking Lung Function on Any Phone

Eric Larson, Mayank Goel, Morgan Redfield, Gaetano Boriello, Margaret Rosenfeld, Shwetak Patel

ACM DEV '13
PDF

2012

 

GripSense: Using Built-in Sensors to Detect Hand Posture and Pressure on Commodity Mobile Phones

Mayank Goel, Jacob Wobbrock, Shwetak Patel

ACM symposium on User Interface Software and Technology (UIST), 2012
PDF Video

 

SpiroSmart: Using a Microphone to Measure Lung Function on a Mobile Phone

Mayank Goel, Eric Larson, Gaetano Boriello, Sonya Heltshe, Margaret Rosenfeld, Shwetak Patel

ACM International Joint Conference on Pervasive and Ubiquitous Computing (UbiComp), 2012, Honorable Mention Award
PDF

 

WalkType: using accelerometer data to accomodate situational impairments in mobile touch screen text entry

Mayank Goel, Leah Findlater, Jacob Wobbrock

Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (CHI), 2012, Honorable Mention
PDF

 

Open data kit sensors: a sensor integration framework for android at the application-level

Waylon Brunette, Rita Sodt, Rohit Chaudhri, Mayank Goel, Michael Falcone, Jaylen Van Orden, Gaetano Borriello

Proceedings of the 10th international conference on Mobile systems, applications, and services, 2012
PDF

 

Open Data Kit Sensors: Mobile Data Collection with Wired and Wireless Sensors

Rohit Chaudhri, Waylon Brunette, Mayank Goel, Rita Sodt, Jaylen Van Orden, Michael Falcone, Gaetano Borriello

ACM DEV '12 Proceedings of the 2nd ACM Symposium on Computing for Development, 2012
PDF

Health Sensing Publications


 

JoulesEye: Energy Expenditure Estimation and Respiration Sensing From Thermal Imagery While Exercising

Rishiraj Adhikary, Maite Sadeh, Nipun Batra, Mayank Goel

Proceedings of the ACM on Interactive, Mobile, Wearable, and Ubiquitous Technologies (IMWUT)
PDF Video

 

LemurDx: Using Unconstrained Passive Sensing for an Objective Measurement of Hyperactivity in Children with no Parent Input

Riku Arakawa, Karan Ahuja, Kristie Mak, Gwendolyn Thompson, Sam Shaaban, Oliver Lindhiem, Mayank Goel

Proceedings of the ACM on Interactive, Mobile, Wearable, and Ubiquitous Technologies (IMWUT)
PDF

 

uKnit: A Position-Aware Reconfigurable Machine-Knitted Wearable for Gestural Interaction and Passive Sensing using Electrical Impedance Tomography

Catherine Tianhong Yu, Riku Arakawa, James McCann, Mayank Goel

Proceedings of the Annual ACM Conference on Human Factors in Computing Systems (CHI)
PDF Video

 

PrISM-Tracker: A Framework for Multimodal Procedure Tracking Using Wearable Sensors and State Transition Information with User-Driven Handling of Errors and Uncertainty

Riku Arakawa, Hiromu Yakura, Vimal Mollyn, Suzanne Nie, Emma Russell, Dustin Demeo, Haarika Reddy, Alexander Maytin, Bryan Carroll, Jill Fain Lehman, Mayank Goel

Proceedings of the ACM on Interactive, Mobile, Wearable, and Ubiquitous Technologies (IMWUT)
PDF

 

Predicting Multiple Sclerosis Outcomes during the COVID-19 Stay-at-Home Period: Observational Study Using Passively Sensed Behaviors and Digital Phenotyping.

Prerna Chikersal, Shruthi Venkatesh, Karmen Masown, Elizabeth Walker, Danyal Quraishi, Anind Dey, Mayank Goel, Zongqi Xia

JMIR Mental Health 2022
PDF

 

Objective Measurement of Hyperactivity Using Mobile Sensing and Machine Learning: A Pilot Study

Oliver Lindhiem, Mayank Goel, Sam Shaaban, Kristie Mak, Prerna Chikersal, Jamie Feldman, Jordan Harris

JMIR Formative Research 2022
PDF

 

FitNibble: A Field Study to Evaluate the Utility and Usability of Automatic Diet Monitoring in Food Journaling Using an Eyeglasses-based Wearable.

Abdelkareem Bedri, Yuchen Liang, Sudershan Boovaraghavan, Geoff Kaufman, Mayank Goel

Proceedings of the Annual Conference on Intelligent User Interfaces (IUI) 2022
PDF Video Code

 

Detecting Depression and Predicting its Onset Using Longitudinal Symptoms Captured by Passive Sensing: A Machine Learning Approach With Robust Feature Selection.

Prerna Chikersal, Afsaneh Doryab, Michael Tumminia, Daniella K. Villalba, Janine M. Dutcher, Xinwen Liu, Sheldon Cohen, Kasey G. Creswell, Jennifer Mankoff, J. David Creswell, Mayank Goel, Anind Dey

ACM Transactions on Computer-Human Interaction (TOCHI) 2020
PDF

 

Understanding Client Support Strategies to Improve Clinical Outcomes in an Online Mental Health Intervention

Prerna Chikersal, Danielle Belgrave, Gavin Doherty, Angel Enrique, Jorge E. Palacios, Derek Richards, Anja Thieme

Proceedings of the Annual ACM Conference on Human Factors in Computing Systems (CHI) 2020
PDF Video

 

FitByte: Automatic Diet Monitoring in Unconstrained Situations Using Multimodal Sensing on Eyeglasses.

Abdelkareem Bedri, Diana Li, Rushil Khurana, Kunal Bhuwalka, Mayank Goel

Proceedings of the Annual ACM Conference on Human Factors in Computing Systems (CHI) 2020
PDF Video Dataset

 

GymCam: Detecting, recognizing, and tracking simultaneous exercises in unconstrained scenes

Rushil Khurana, Karan Ahuja, Zac Yu, Jennifer Mankoff, Chris Harrison, Mayank Goel

Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies (IMWUT), 2019
PDF Video

 

Challenges in Realizing Smartphone-based Health Sensing

Alex Mariakakis, Edward J. Wang, Shwetak Patel, Mayank Goel

IEEE Pervasive 2019, Best Paper Award Runner-Up
PDF

 

Use of a smartphone app to assess neonatal jaundice

James A. Taylor, James W. Stout, Lilian de Greef, Mayank Goel, Shwetak Patel, Esther K. Chung, Aruna Koduri, Shawn McMahon, Jane Dickerson, Elizabeth A. Simpson, Eric Larson

Pediatrics Sep 2017
PDF Video

 

Design and Learnability of Vortex Whistles for Managing Chronic Lung Function via Smartphones

Spencer Kaiser, Ashley Parks, Patrick Leopard, Charlie Albright, Jake Carlson, Mayank Goel, Damoun Nassehi, Eric Larson

Proceedings of the 2016 ACM International Joint Conference on Pervasive and Ubiquitous Computing (UbiComp '16)
PDF

 

SpiroCall: Measuring Lung Function over a Phone Call

Mayank Goel, Elliot Saba, Maia Stiber, Eric Whitmire, Josh Fromm, Eric Larson, Gaetano Borriello, Shwetak Patel

Conference on Human Factors in Computing Systems (CHI), 2016, Honorable Mention Award
PDF Video

 

HyperCam: Hyperspectral Imaging for Ubiquitous Computing Applications

Mayank Goel, Eric Whitmire, Alex Mariakakis, Scott Saponas, Neel Joshi, Dan Morris, Brian Guenter, Marcel Gavriliu, Gaetano Borriello, Shwetak Patel

2015 ACM International Joint Conference on Pervasive and Ubiquitous Computing (UbiComp '15), Honorable Mention Award
PDF Video

 

Wibreathe: Estimating respiration rate using wireless signals in natural settings in the home

Ruth Vinisha Ravichandran, Elliot Saba, Ke-Yu Chen, Mayank Goel, Sidhant Gupta, Shwetak Patel

IEEE International Conference on Pervasive Computing and Communications (PerCom), 2015, Best Paper Runner-up
PDF

 

BiliCam: Using Mobile Phones to Monitor Newborn Jaundice

Lilian de Greef, Mayank Goel, Min Joon Seo, Eric Larson, James W. Stout, James A. Taylor, Shwetak Patel

ACM International Joint Conference on Pervasive and Ubiquitous Computing (UbiComp), 2014, Honorable Mention Award
PDF Video

 

Tracking Lung Function on Any Phone

Eric Larson, Mayank Goel, Morgan Redfield, Gaetano Boriello, Margaret Rosenfeld, Shwetak Patel

ACM DEV '13
PDF

 

SpiroSmart: Using a Microphone to Measure Lung Function on a Mobile Phone

Mayank Goel, Eric Larson, Gaetano Boriello, Sonya Heltshe, Margaret Rosenfeld, Shwetak Patel

ACM International Joint Conference on Pervasive and Ubiquitous Computing (UbiComp), 2012, Honorable Mention Award
PDF

 

Open data kit sensors: a sensor integration framework for android at the application-level

Waylon Brunette, Rita Sodt, Rohit Chaudhri, Mayank Goel, Michael Falcone, Jaylen Van Orden, Gaetano Borriello

Proceedings of the 10th international conference on Mobile systems, applications, and services, 2012
PDF

 

Open Data Kit Sensors: Mobile Data Collection with Wired and Wireless Sensors

Rohit Chaudhri, Waylon Brunette, Mayank Goel, Rita Sodt, Jaylen Van Orden, Michael Falcone, Gaetano Borriello

ACM DEV '12 Proceedings of the 2nd ACM Symposium on Computing for Development, 2012
PDF

Privacy-Aware Sensing Publications


 

Bring Privacy To The Table: Interactive Negotiation for Privacy Settings of Shared Sensing Devices

Haozhe Zhou, Mayank Goel, Yuvraj Agarwal

Proceedings of the Annual ACM Conference on Human Factors in Computing Systems (CHI)
PDF

 

Kirigami: Lightweight Speech Filtering for Privacy-Preserving Activity Recognition using Audio

Sudershan Boovaraghavan, Haozhe Zhou, Mayank Goel, Yuvraj Agarwal

Proceedings of the ACM on Interactive, Mobile, Wearable, and Ubiquitous Technologies (IMWUT)
PDF

 

VAX: Using Existing Video and Audio-based Activity Recognition Models to Bootstrap Privacy-Sensitive Sensors

Prasoon Patidar, Mayank Goel, Yuvraj Agarwal

Proceedings of the ACM on Interactive, Mobile, Wearable, and Ubiquitous Technologies (IMWUT)
PDF

 

SAMoSA: Sensing Activities with Motion and Subsampled Audio

Vimal Mollyn, Karan Ahuja, Dhruv Verma, Chris Harrison, Mayank Goel

Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies (IMWUT) 2022.
PDF Video

 

IMU2Doppler: Cross-Modal Domain Adaptation for Doppler-based Activity Recognition Using IMU Data

Sejal Bhalla, Mayank Goel, Rushil Khurana

Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies (IMWUT) 2022
PDF Video

 

Vid2Doppler: Synthesizing Doppler Radar Data from Videos for Training Privacy-Preserving Activity Recognition

Karan Ahuja, Yue Jiang, Mayank Goel, Chris Harrison

Proceedings of the Annual ACM Conference on Human Factors in Computing Systems (CHI) 2021
PDF Video

 

PrivacyMic: Utilizing Inaudible Frequencies for Privacy Preserving Daily Activity Recognition

Yasha Iravantchi, Karan Ahuja, Mayank Goel, Chris Harrison, Alanson Sample

Proceedings of the Annual ACM Conference on Human Factors in Computing Systems (CHI), Honorable Mention Award
PDF

 

GymCam: Detecting, recognizing, and tracking simultaneous exercises in unconstrained scenes

Rushil Khurana, Karan Ahuja, Zac Yu, Jennifer Mankoff, Chris Harrison, Mayank Goel

Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies (IMWUT), 2019
PDF Video

Activity Recognition Publications


 

PrISM-Q&A: Step-Aware Question Answering with Large Language Models Enabled by Multimodal Procedure Tracking using a Smartwatch.

Riku Arakawa, Jill Lehman, Mayank Goel

To appear in Proceedings of the ACM on Interactive, Mobile, Wearable, and Ubiquitous Technologies (IMWUT)

 

PrISM-Observer: Intervention Agent to Help Users Perform Everyday Procedures Sensed using a Smartwatch

Riku Arakawa, Hiromu Yakura, Mayank Goel

Proceedings of User Interface Software and Technology (UIST)
PDF Video

 

VAX: Using Existing Video and Audio-based Activity Recognition Models to Bootstrap Privacy-Sensitive Sensors

Prasoon Patidar, Mayank Goel, Yuvraj Agarwal

Proceedings of the ACM on Interactive, Mobile, Wearable, and Ubiquitous Technologies (IMWUT)
PDF

 

IMUPoser: Full-Body Pose Estimation using IMUs in Phones, Watches, and Earbuds.

Vimal Mollyn, Riku Arakawa, Mayank Goel, Chris Harrison, Karan Ahuja

Proceedings of the Annual ACM Conference on Human Factors in Computing Systems (CHI) 2023, Honorable Mention Award
PDF Video Code

 

PrISM-Tracker: A Framework for Multimodal Procedure Tracking Using Wearable Sensors and State Transition Information with User-Driven Handling of Errors and Uncertainty

Riku Arakawa, Hiromu Yakura, Vimal Mollyn, Suzanne Nie, Emma Russell, Dustin Demeo, Haarika Reddy, Alexander Maytin, Bryan Carroll, Jill Fain Lehman, Mayank Goel

Proceedings of the ACM on Interactive, Mobile, Wearable, and Ubiquitous Technologies (IMWUT)
PDF

 

SAMoSA: Sensing Activities with Motion and Subsampled Audio

Vimal Mollyn, Karan Ahuja, Dhruv Verma, Chris Harrison, Mayank Goel

Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies (IMWUT) 2022.
PDF Video

 

FitNibble: A Field Study to Evaluate the Utility and Usability of Automatic Diet Monitoring in Food Journaling Using an Eyeglasses-based Wearable.

Abdelkareem Bedri, Yuchen Liang, Sudershan Boovaraghavan, Geoff Kaufman, Mayank Goel

Proceedings of the Annual Conference on Intelligent User Interfaces (IUI) 2022
PDF Video Code

 

IMU2Doppler: Cross-Modal Domain Adaptation for Doppler-based Activity Recognition Using IMU Data

Sejal Bhalla, Mayank Goel, Rushil Khurana

Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies (IMWUT) 2022
PDF Video

 

Vid2Doppler: Synthesizing Doppler Radar Data from Videos for Training Privacy-Preserving Activity Recognition

Karan Ahuja, Yue Jiang, Mayank Goel, Chris Harrison

Proceedings of the Annual ACM Conference on Human Factors in Computing Systems (CHI) 2021
PDF Video

 

PrivacyMic: Utilizing Inaudible Frequencies for Privacy Preserving Daily Activity Recognition

Yasha Iravantchi, Karan Ahuja, Mayank Goel, Chris Harrison, Alanson Sample

Proceedings of the Annual ACM Conference on Human Factors in Computing Systems (CHI), Honorable Mention Award
PDF

 

FitByte: Automatic Diet Monitoring in Unconstrained Situations Using Multimodal Sensing on Eyeglasses.

Abdelkareem Bedri, Diana Li, Rushil Khurana, Kunal Bhuwalka, Mayank Goel

Proceedings of the Annual ACM Conference on Human Factors in Computing Systems (CHI) 2020
PDF Video Dataset

 

GymCam: Detecting, recognizing, and tracking simultaneous exercises in unconstrained scenes

Rushil Khurana, Karan Ahuja, Zac Yu, Jennifer Mankoff, Chris Harrison, Mayank Goel

Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies (IMWUT), 2019
PDF Video

 

Ubicoustics: Plug-and-Play Acoustic Activity Recognition

Gierad Laput, Karan Ahuja, Mayank Goel, Chris Harrison

Conference on Human Factors in Computing Systems (CHI), 2016
PDF Video Code

 

EarBit: using wearable sensors to detect eating episodes in unconstrained environments

Abdelkareem Bedri, Richard Li, Malcolm Haynes, Raj Prateek Kosaraju, Ishaan Grover, Temiloluwa Prioleau, Min Yan Beh, Mayank Goel, Thad Starner, Gregory Abowd

Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies (IMWUT), 2017
PDF

 

MagnifiSense: Inferring Device Interaction Using Wrist-worn Passive Magneto-inductive Sensors

Edward Wang, Tien-Jui Lee, Alex Mariakakis, Mayank Goel, Sidhant Gupta, Shwetak Patel

ACM International Joint Conference on Pervasive and Ubiquitous Computing (UbiComp), 2015
PDF

New Interfaces Publications


 

PrISM-Q&A: Step-Aware Question Answering with Large Language Models Enabled by Multimodal Procedure Tracking using a Smartwatch.

Riku Arakawa, Jill Lehman, Mayank Goel

To appear in Proceedings of the ACM on Interactive, Mobile, Wearable, and Ubiquitous Technologies (IMWUT)

 

PrISM-Observer: Intervention Agent to Help Users Perform Everyday Procedures Sensed using a Smartwatch

Riku Arakawa, Hiromu Yakura, Mayank Goel

Proceedings of User Interface Software and Technology (UIST)
PDF Video

 

EITPose: Wearable and Practical Electrical Impedance Tomography for Continuous Hand Pose Estimation

Alexander Kyu, Hongyu Mao, Junyi Zhu, Mayank Goel, Karan Ahuja

Proceedings of the Annual ACM Conference on Human Factors in Computing Systems (CHI)
PDF Video

 

MI-Poser: Human Body Pose Tracking using Magnetic and Inertial Sensor Fusion with Metal Interference Mitigation

Riku Arakawa, Bing Zhou, Gurunandan Krishnan, Mayank Goel, Shree K. Nayar

Proceedings of the ACM on Interactive, Mobile, Wearable, and Ubiquitous Technologies (IMWUT)
PDF Video

 

uKnit: A Position-Aware Reconfigurable Machine-Knitted Wearable for Gestural Interaction and Passive Sensing using Electrical Impedance Tomography

Catherine Tianhong Yu, Riku Arakawa, James McCann, Mayank Goel

Proceedings of the Annual ACM Conference on Human Factors in Computing Systems (CHI)
PDF Video

 

IMUPoser: Full-Body Pose Estimation using IMUs in Phones, Watches, and Earbuds.

Vimal Mollyn, Riku Arakawa, Mayank Goel, Chris Harrison, Karan Ahuja

Proceedings of the Annual ACM Conference on Human Factors in Computing Systems (CHI) 2023, Honorable Mention Award
PDF Video Code

 

A semantic-based approach to digital content placement for immersive environments

Jingyang Liu, Yunzhi Li, Mayank Goel

The Visual Computer 2022
PDF

 

RGBDGaze: Gaze Tracking on Smartphones with RGB and Depth Data

Riku Arakawa, Mayank Goel, Chris Harrison, Karan Ahuja

Proceedings of the International Conference on Multimodal Interaction (ICMI) 2022.
PDF Video

 

TouchPose: Hand Pose Prediction, Depth Estimation, and Touch Classification from Capacitive Images

Karan Ahuja, Paul Streli, Christian Holz

Proceedings of User Interface Software and Technology (UIST) 2021
PDF Video

 

EyeMU Interactions: Gaze + IMU Gestures on Mobile Devices

Andy Kong, Karan Ahuja, Mayank Goel, Chris Harrison

Proceedings of the International Conference on Multimodal Interaction (ICMI) 2021
PDF Video

 

Pose-on-the-Go: Approximating User Pose with Smartphone Sensor Fusion and Inverse Kinematics

Karan Ahuja, Sven Mayer, Mayank Goel, Chris Harrison

Proceedings of the Annual ACM Conference on Human Factors in Computing Systems (CHI) 2021
PDF Video

 

Classroom Digital Twins with Instrumentation-Free Gaze Tracking

Karan Ahuja, Deval Shah, Sujeath Pareddy, Franceska Xhakaj, Amy Ogan, Yuvraj Agarwal, Chris Harrison

Proceedings of the Annual ACM Conference on Human Factors in Computing Systems (CHI) 2021
PDF Video

 

CoolMoves: User Motion Accentuation in Virtual Reality

Karan Ahuja, Eyal Ofek, Mar Gonzalez-Franco, Christian Holz, Andrew D. Wilson

Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies (IMWUT) 2021
PDF

 

BodySLAM: Opportunistic User Digitization in Multi-User AR/VR Experiences

Karan Ahuja, Mayank Goel, Chris Harrison

Proceedings of SUI 2020
PDF Video

 

Direction-of-Voice (DoV) Estimation for Intuitive Speech Interaction with Smart Devices Ecosystems

Karan Ahuja, Andy Kong, Mayank Goel, Chris Harrison

Proceedings of UIST 2020
PDF Video

 

Eyes on the Road: Detecting Phone Usage by Drivers Using On-Device Cameras

Rushil Khurana, Mayank Goel

Proceedings of the Annual ACM Conference on Human Factors in Computing Systems (CHI) 2020
PDF Video

 

Beyond the Prototype: Understanding the Challenge of Scaling Hardware Device Production

Rushil Khurana, Steve Hodges

Proceedings of the Annual ACM Conference on Human Factors in Computing Systems (CHI) 2020, Best Paper Award
PDF

 

Digital Ventriloquism: Giving Voice to Everyday Objects

Yasha Iravantchi, Mayank Goel, Chris Harrison

Proceedings of the Annual ACM Conference on Human Factors in Computing Systems (CHI) 2020
PDF Video

 

MeCap: Whole-Body Digitization for Low-Cost VR/AR Headsets

Karan Ahuja, Chris Harrison, Mayank Goel, Robert Xiao

UIST 2019, Honorable Mention Award
PDF Video

 

LightAnchors: Appropriating Point Lights for Spatially-Anchored Augmented Reality Interfaces

Karan Ahuja, Sujeath Pareddy, Robert Xiao, Mayank Goel, Chris Harrison

Proceedings of UIST 2019
PDF Video

 

Detachable Smartwatch: More Than A Wearable

Rushil Khurana, Mayank Goel, Kent Lyons

Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies (IMWUT), 2019
PDF Video

 

Interferi: Gesture Sensing using On-Body Acoustic Interferometry

Yasha Iravantchi, Yang Zhang, Evi Bernitsas, Mayank Goel, Chris Harrison

Proceedings of the 2019 CHI Conference on Human Factors in Computing Systems (CHI ’19), 2019, Honorable Mention Award
PDF Video

 

BeamBand: Hand Gesture Sensing with Ultrasonic Beamforming

Yasha Iravantchi, Mayank Goel, Chris Harrison

Proceedings of the 2019 CHI Conference on Human Factors in Computing Systems (CHI ’19), 2019
PDF Video

 

RainCheck: Overcoming Capacitive Interference Caused by Rainwater on Smartphones

Ying-Chao Tung, Mayank Goel, Isaac Zinda, Jacob Wobbrock

International Conference on Multimodal Interaction (ICMI) 2018
PDF

 

In only 3 minutes: perceived exertion limits of smartwatch use

Rushil Khurana, Nikola Banovic, Kent Lyons

International Symposium on Wearable Computers (ISWC), 2018
PDF

 

EyeSpyVR: Interactive Eye Sensing Using Off-the-Shelf, Smartphone-Based VR Headsets

Karan Ahuja, Rahul Islam, Varun Parashar, Kuntal Dey, Chris Harrison, Mayank Goel

Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies (IMWUT), 2018
PDF Video

 

DigiTouch: Reconfigurable Thumb-to-Finger Input and Text Entry on Head-mounted Displays

Eric Whitmire, Mohit Jain, Divye Jain, Greg Nelson, Ravi Karkar, Shwetak Patel, Mayank Goel

Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies (IMWUT), 2017
PDF Video

 

HyperCam: Hyperspectral Imaging for Ubiquitous Computing Applications

Mayank Goel, Eric Whitmire, Alex Mariakakis, Scott Saponas, Neel Joshi, Dan Morris, Brian Guenter, Marcel Gavriliu, Gaetano Borriello, Shwetak Patel

2015 ACM International Joint Conference on Pervasive and Ubiquitous Computing (UbiComp '15), Honorable Mention Award
PDF Video

 

Tongue-in-Cheek: Using Wireless Signals to Enable Non-Intrusive and Flexible Facial Gestures Detection

Mayank Goel, Chen Zhao, Ruth Vinisha, Shwetak Patel

Conference on Human Factors in Computing Systems (CHI), 2015, Honorable Mention Award
PDF Video

 

SwitchBack: Using Focus and Saccade Tracking to Guide Users’ Attention for Mobile Task Resumption

Alex Mariakakis, Mayank Goel, Md. Tanvir Islam Aumi, Shwetak Patel, Jacob Wobbrock

Conference on Human Factors in Computing Systems (CHI), 2015
PDF

 

AirLink: Sharing Files Between Multiple Devices Using In-air Gestures

Ke-Yu Chen, Daniel Ashbrook, Mayank Goel, Sung-Hyuck Lee, Shwetak Patel

ACM International Joint Conference on Pervasive and Ubiquitous Computing (UbiComp) 2014
PDF Video

 

SurfaceLink: Using Inertial and Acoustic Sensing to Enable Multi-device Interaction on a Surface

Mayank Goel, Brendan Lee, Md. Tanvir Islam Aumi, Shwetak Patel, Gaetano Borriello, Stacie Hibino, Bo Begole

Conference on Human Factors in Computing Systems (CHI), 2014
PDF Video

 

DopLink: Using the Doppler Effect for Multi-device Interaction

Md. Tanvir Islam Aumi, Sidhant Gupta, Mayank Goel, Eric Larson, Shwetak Patel

ACM International Joint Conference on Pervasive and Ubiquitous Computing (UbiComp), 2013
PDF Video

 

GripSense: Using Built-in Sensors to Detect Hand Posture and Pressure on Commodity Mobile Phones

Mayank Goel, Jacob Wobbrock, Shwetak Patel

ACM symposium on User Interface Software and Technology (UIST), 2012
PDF Video

Situational Impairments Publications


 

Eyes on the Road: Detecting Phone Usage by Drivers Using On-Device Cameras

Rushil Khurana, Mayank Goel

Proceedings of the Annual ACM Conference on Human Factors in Computing Systems (CHI) 2020
PDF Video

 

Detachable Smartwatch: More Than A Wearable

Rushil Khurana, Mayank Goel, Kent Lyons

Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies (IMWUT), 2019
PDF Video

 

RainCheck: Overcoming Capacitive Interference Caused by Rainwater on Smartphones

Ying-Chao Tung, Mayank Goel, Isaac Zinda, Jacob Wobbrock

International Conference on Multimodal Interaction (ICMI) 2018
PDF

 

Carpacio: Repurposing Capacitive Sensors to Distinguish Driver and Passenger Touches on In-Vehicle Screens

Edward Wang, Jake Garrison, Eric Whitmire, Mayank Goel, Shwetak Patel

30th Annual ACM Symposium on User Interface Software and Technology (UIST), 2017
PDF Video

 

Tongue-in-Cheek: Using Wireless Signals to Enable Non-Intrusive and Flexible Facial Gestures Detection

Mayank Goel, Chen Zhao, Ruth Vinisha, Shwetak Patel

Conference on Human Factors in Computing Systems (CHI), 2015, Honorable Mention Award
PDF Video

 

SwitchBack: Using Focus and Saccade Tracking to Guide Users’ Attention for Mobile Task Resumption

Alex Mariakakis, Mayank Goel, Md. Tanvir Islam Aumi, Shwetak Patel, Jacob Wobbrock

Conference on Human Factors in Computing Systems (CHI), 2015
PDF

 

ContextType: Using Hand Posture Information to Improve Mobile Touch Screen Text Entry

Mayank Goel, Alex Jansen, Travis Mandel, Shwetak Patel, Jacob O. Wobbrock

Conference on Human Factors in Computing Systems (CHI), 2013
PDF

 

GripSense: Using Built-in Sensors to Detect Hand Posture and Pressure on Commodity Mobile Phones

Mayank Goel, Jacob Wobbrock, Shwetak Patel

ACM symposium on User Interface Software and Technology (UIST), 2012
PDF Video

 

WalkType: using accelerometer data to accomodate situational impairments in mobile touch screen text entry

Mayank Goel, Leah Findlater, Jacob Wobbrock

Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (CHI), 2012, Honorable Mention
PDF