Checking the Statistical Assumptions Underlying the Application of the Standard Deviation and RMS Error to Eye-Movement Time Series: A Comparison between Human and Artificial Eyes

Date

2022-02

Authors

Friedman, Lee
Hanson, Timothy
Stern, Hal
Aziz, Samantha D.
Komogortsev, Oleg

Journal Title

Journal ISSN

Volume Title

Publisher

Abstract

Spatial precision of eye movement is often measured using the standard deviation (SD) of the eye position signal or the root mean square (RMS) of the sample-to-sample signal differences (StoS) during fixation. Both SD and RMS of StoS signal differences are common statistical measures, but there are underlying assumptions that impact their interpretation. As a single summary measure of the variability of a distribution, the SD is most useful when applied to unimodal distributions. Both measures assume stationarity, which means that the statistical properties of the signals are stable over time. Both metrics assume the samples of the signals are independent. The presence of autocorrelation indicates that the samples in the time series are not independent. We tested these assumptions with multiple fixations from two studies, a publicly available dataset that included both human and artificial eyes (“HA Dataset”, N=336 fixations), and data from our laboratory of 14 subjects (“TXstate”, N=166 fixations). Many position signal distributions were multimodal (median for HA fixations: 40.6\%, TXstate: 84\%). No fixation position signals were stationary. All position signals were statistically significantly autocorrelated (p < 0.01). Thus, the statistical assumptions of the SD were not met for any fixation. All StoS signals were unimodal. Some StoS signals were stationary (median for HA fixations: 35.6\% , TXstate: 37.5\%). Almost all StoS signals were statistically significantly autocorrelated (p < 0.01). For TXstate, 16 of 166 (9.6\%) fixations met all assumptions. Thus, the statistical assumptions of the RMS were generally not met. The failure of these assumptions calls into question the appropriateness of the SD or the RMS-StoS as metrics of precision for eye-trackers. We compared the SD and the RMS to non-parametric versions of these measures (e.g., Mean Absolute Deviation (MAD) versus SD). The parametric and non-parametric indices were highly correlated. Nonparametric measures are more robust to deviations from normality, so in this sense, they are an improvement. However, the use of nonparametric measures does not remove the requirements for independence and stationarity.

Description

Article is submitted and under review.
Note: ALL_Fixation_Blocks_HA.zip file contains 137 TIFF images of fixation blocks from the HA study.

Keywords

standard deviation, RMS error, time series, Computer Science

Citation

Rights

Rights Holder

Rights License

Rights URI