Thesis Demonstrates 3D Motion Tracking on Mobile Device
Amar Nikhanj, an engineering graduate school student at the University
of Wisconsin – Milwaukee, published a thesis that demonstrates 3D motion
tracking is possible on smart phones. Acceptance of this new approach in the
marketplace would cause a paradigm shift for the motion capture industry and,
in particular, kinesiology.
In his thesis, Adaptation of Moiré Phase Tracking to a Mobile Device
for Field 3D Data Collections, Nikhanj combined the successful history of moiré
phase tracking (MPT) in studying biomechanics with the mobility provided by a
mobile device to focus on an immediate market need: quantitative Rapid Upper
Limb Assessment (RULA).
The RULA is a survey method developed to perform risk analysis of upper
limb injury in the workplace. It has been used to measure musculoskeletal risk,
improve workstation designs, increase productivity, and educate workers on
improving their motion to reduce their risk of injury.
Nikhanj and his team adapted existing MPT technology from Metria Innovation to work on a mobile device. The 3D motion tracking device features camera
controls, processing power, and calibration abilities. They applied the
technology to the RULA to allow for a quicker and more precise instrumented
RULA versus simply an observation analysis.
The RULA
RULA was developed to be completed visually, where an observer analyzes
the worst-case posture, or in some cases multiple postures, in a motion by
estimating the joint angles. Angles of concern are upper arm flexion or
extension, upper arm abduction, elbow flexion or extension, wrist flexion or
extension, wrist radial or ulnar deviation, neck flexion or extension, and
trunk flexion or extension.
The final score determines if the motion is acceptable and how quickly
changes to the motion are required to prevent injury. But the standard RULA
measurement has downfalls:
-
The observer performing the RULA must use their
judgement on which posture to use
-
The observer must estimate the angles through which each limb is rotating
RULA is a good candidate for 3D motion tracking: it consists of
information in 3D space that must be decomposed into angles. Unfortunately,
many 3D motion tracking technologies require more than one camera and have
complicated calibration processes. RULA also is typically performed in the
field in a potentially messy, enclosed, or dangerous environment — where having
any camera system may not be realistic. This equipment barrier includes the
Metria MPT Series 2 system since it requires a calibrated camera and a computer
to calculate six degrees of freedom: forward/back, up/down, left/right, pitch,
yaw, and roll.
The Device
Accelerated technologies in mobile devices such as tablets and phones
provide an ability to perform high intensity calculations all while obtaining
precise data, including through photogrammetry — making measurements through
photographs.
Nikhanj and his team adapted Metria's MPT software to work on a Google Nexus tablet. Manual focus and exposure control is absolutely
necessary for MPT and ideally the settings are repeatable with a high level of
accuracy. The team used the Android platform to develop MPT processing, camera
control, camera calibration, and the algorithm to calculate limb angles.
When tracking motion, there is no complicated setup. No tripods with
expensive and specialized cameras on them. Just a subject with markers and an operator
holding the mobile device. The operator/image collector has freedom to move
around, as space allows, and can take images from several angles.
The Results
Nikhanj received Institutional Review Board (IRB) #16.137 approval to
perform a study using the device. The RULA study required a total of six motion
tracking markers placed in the room, and then on the subject’s torso, upper
arm, lower arm, wrist, and head. The room marker was placed so the Y-Axis was
aligned with gravity.
A successful RULA measurement was captured in the study via the mobile
3D tracking device. Subjects were not required to remove jewelry or any other
reflective objects.
The table below shows the RULA angles calculated by MPT compared with
three observed RULAs on the same motion. All of the angles are reasonably
similar and most angles are within 15° of each other, but the quantitative
calculations are precise where some of the observational calculations are
ranges. And smaller angles can now be calculated, for example the Torso
Rotation, which was normally rounded to 0° by two of the observers performing
the RULA.
Upper Arm (Shoulder)
Rotation
|
48.3° |
15-45° |
15-45° |
15-45° |
Lower Arm (Elbow)
Angulation to Torso
|
50.7° |
60-100° |
60-100° |
60-100° |
Wrist Rotation
|
10.1°
Downwards |
0° to 15°
Downwards |
0° to 15°
Downwards |
0° to 15°
Downwards |
| Neck Rotation |
18.4° Flexion |
10° to 20° Flexion |
0° to 10° Flexion |
0° to 10° Flexion |
| Torso Rotation |
4.8° Forward |
0°-20° Forward |
0° (Erect) |
0° (Erect) |
RULA angle comparison with three observed RULAs on the same motion. All of the angles are comparable.
The study determined that 3D motion tracking on a mobile device in the
field is possible. Since the data capturing source can be mobile, this 3D
motion tracking technology is suited for field use and does not need
significant control of its environment.
Nikhanj’s thesis lays the foundation for future research to improve
upon accuracy, speed, and robustness of MPT on a mobile device. Much of the
ground work has been established, with the basic challenges solved in his
thesis such as processing, camera control, and calibration. Tracking motion in
3D space on a device that can fit in a pocket truly is a paradigm shift.
Download the full thesis here.