ArmTrak: I am a Smartwatch and I can Track my User's Arm

What is ArmTrak?

ArmTrak is a system designed to track the geometric motion of the human arm using only the motion and magnetic sensors on the smartwatch.

Core Idea

While arm posture space can be very large, ArmTrak continuously estimates the orientation of the smartwatch (worn on the wrist), and together with human arm kinematics, ArmTrak reduces its uncertainty on elbow/wrist location to a small area (called point clouds). The remaining uncertainty is further mitigated by feeding the inferred elbow acceleration into modified HMM framework and inferring the correct elbow/wrist locations within the point clouds.


Demo 1: Use ArmTrak as a Kinect

Demo 2: Use ArmTrak to Write in the Air

Key Research Components

  • Arm posture is essentially determined by 5 joint angles.

    While arm posture contains elbow location, wrist location and wrist/watch orientation, they are essentially a function of 5 joint angles. Given 5 angle values and upper/lower arm lengths, the whole arm posture is fixed.

    Arm Model

  • Measured (noisy) elbow acceleration is fed into modified HMM framework for elbow tracking.

    We modified the traditional HMM framework so that the best sequence of elbow/wrist locations within the point clouds can be inferred efficiently with Viterbi algorithm, using elbow acceleration as the observation.

  • Given watch/wrist orientation, possible space of wrist and elbow locations is quite limited.

    The figure below shows 5 different watch orientations and the corresponding wrist and elbow point clouds (shown in light green and dark red, respectively). The point clouds (essentially the space of feasible wrist and elbow locations for the given orientation) are relatively small and narrows down the uncertainty of the user's arm posture.

    Point Clouds


  • I am a Smartwatch and I can Track my User’s Arm
    Sheng Shen, He Wang, Romit Roy Choudhury
    ACM MobiSys, 2016 [pdf]


Sheng Shen

Sheng Shen
PhD Student
Dept. of ECE, UIUC

He Wang

He Wang
PhD Candidate
Dept. of ECE, UIUC

Romit Roy Choudhury

Romit Roy Choudhury
Associate Professor
Dept. of ECE & CS, UIUC