Augmenting VR/AR applications with EEG/EOG monitoring and oculo-vestibular recoupling

John K. Zao; Tzyy Ping Jung; Hung Ming Chang; Tchin Tze Gan; Yu Te Wang; Yuan Pin Lin; Wen Hao Liu; Guang Yu Zheng; Chin Kuo Lin; Chia Hung Lin; Yu Yi Chien; Fang Cheng Lin; Yi Pai Huang; Sergio José Rodríguez Méndez; Felipe A. Medeiros

doi:10.1007/978-3-319-39955-3_12

Augmenting VR/AR applications with EEG/EOG monitoring and oculo-vestibular recoupling

John K. Zao^*, Tzyy Ping Jung, Hung Ming Chang, Tchin Tze Gan, Yu Te Wang, Yuan Pin Lin, Wen Hao Liu, Guang Yu Zheng, Chin Kuo Lin, Chia Hung Lin, Yu Yi Chien, Fang Cheng Lin, Yi Pai Huang, Sergio José Rodríguez Méndez, Felipe A. Medeiros

^*Corresponding author for this work

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution › peer-review

20 Citations (Scopus)

Abstract

Head-mounted virtual reality and augmented reality displays (a.k.a. VR/AR goggles) created a revolutionary multimedia genre that is seeking ever-broadening applications and novel natural human interfaces. Adding neuromonitoring and neurofeedback to this genre is expected to introduce a new dimension to user interaction with the cyber-world. This study presents the development of a Neuromonitoring VR/AR Goggle armed with electroencephalo- gram and electrooculogram sensors, programmable milli-Ampere current stimulators and wireless fog/cloud computing support. Beside of its potential use in mitigating cybersickness, this device may have potential applications in augmented cognition ranging from feedback-controlled perceptual training to on-line learning and virtual social interactions. A prototype of the device has been made from a Samsung Gear VR for S6. This study explains its technical design to ensure precision data sampling, synchronous event marking, real-time signal processing and big data cloud computing support. This study also demonstrates the effective-ness in measuring the event-related potentials during a visual oddball experiment.

Original language	English
Title of host publication	Foundations of Augmented Cognition
Subtitle of host publication	Neuroergonomics and Operational Neuroscience - 10th International Conference, AC 2016 and Held as Part of HCI International 2016, Proceedings
Editors	Cali M. Fidopiastis, Dylan D. Schmorrow
Publisher	Springer Verlag
Pages	121-131
Number of pages	11
ISBN (Print)	9783319399546
DOIs	https://doi.org/10.1007/978-3-319-39955-3_12
Publication status	Published - 2016
Externally published	Yes
Event	10th International Conference on Foundations of Augmented Cognition: Neuroergonomics and Operational Neuroscience, AC 2016 and Held as Part of 18th International Conference on Human-Computer Interaction, HCI International 2016 - Toronto, Canada Duration: 17 Jul 2016 → 22 Jul 2016

Publication series

Name	Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)
Volume	9743
ISSN (Print)	0302-9743
ISSN (Electronic)	1611-3349

Conference

Conference	10th International Conference on Foundations of Augmented Cognition: Neuroergonomics and Operational Neuroscience, AC 2016 and Held as Part of 18th International Conference on Human-Computer Interaction, HCI International 2016
Country/Territory	Canada
City	Toronto
Period	17/07/16 → 22/07/16

Access to Document

10.1007/978-3-319-39955-3_12

Cite this

Zao, J. K., Jung, T. P., Chang, H. M., Gan, T. T., Wang, Y. T., Lin, Y. P., Liu, W. H., Zheng, G. Y., Lin, C. K., Lin, C. H., Chien, Y. Y., Lin, F. C., Huang, Y. P., Méndez, S. J. R., & Medeiros, F. A. (2016). Augmenting VR/AR applications with EEG/EOG monitoring and oculo-vestibular recoupling. In C. M. Fidopiastis, & D. D. Schmorrow (Eds.), Foundations of Augmented Cognition: Neuroergonomics and Operational Neuroscience - 10th International Conference, AC 2016 and Held as Part of HCI International 2016, Proceedings (pp. 121-131). (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics); Vol. 9743). Springer Verlag. https://doi.org/10.1007/978-3-319-39955-3_12

Zao, John K. ; Jung, Tzyy Ping ; Chang, Hung Ming et al. / Augmenting VR/AR applications with EEG/EOG monitoring and oculo-vestibular recoupling. Foundations of Augmented Cognition: Neuroergonomics and Operational Neuroscience - 10th International Conference, AC 2016 and Held as Part of HCI International 2016, Proceedings. editor / Cali M. Fidopiastis ; Dylan D. Schmorrow. Springer Verlag, 2016. pp. 121-131 (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)).

@inproceedings{0c1c0556945641cfad889b053a55184f,

title = "Augmenting VR/AR applications with EEG/EOG monitoring and oculo-vestibular recoupling",

abstract = "Head-mounted virtual reality and augmented reality displays (a.k.a. VR/AR goggles) created a revolutionary multimedia genre that is seeking ever-broadening applications and novel natural human interfaces. Adding neuromonitoring and neurofeedback to this genre is expected to introduce a new dimension to user interaction with the cyber-world. This study presents the development of a Neuromonitoring VR/AR Goggle armed with electroencephalo- gram and electrooculogram sensors, programmable milli-Ampere current stimulators and wireless fog/cloud computing support. Beside of its potential use in mitigating cybersickness, this device may have potential applications in augmented cognition ranging from feedback-controlled perceptual training to on-line learning and virtual social interactions. A prototype of the device has been made from a Samsung Gear VR for S6. This study explains its technical design to ensure precision data sampling, synchronous event marking, real-time signal processing and big data cloud computing support. This study also demonstrates the effective-ness in measuring the event-related potentials during a visual oddball experiment.",

keywords = "Augmented cognition, Cybersickness, Electroencephalography, Electrooculography, Mixed reality, Oculo-vestibular recoupling, Virtual reality",

author = "Zao, {John K.} and Jung, {Tzyy Ping} and Chang, {Hung Ming} and Gan, {Tchin Tze} and Wang, {Yu Te} and Lin, {Yuan Pin} and Liu, {Wen Hao} and Zheng, {Guang Yu} and Lin, {Chin Kuo} and Lin, {Chia Hung} and Chien, {Yu Yi} and Lin, {Fang Cheng} and Huang, {Yi Pai} and M{\'e}ndez, {Sergio Jos{\'e} Rodr{\'i}guez} and Medeiros, {Felipe A.}",

note = "Publisher Copyright: {\textcopyright} Springer International Publishing Switzerland 2016.; 10th International Conference on Foundations of Augmented Cognition: Neuroergonomics and Operational Neuroscience, AC 2016 and Held as Part of 18th International Conference on Human-Computer Interaction, HCI International 2016 ; Conference date: 17-07-2016 Through 22-07-2016",

year = "2016",

doi = "10.1007/978-3-319-39955-3_12",

language = "English",

isbn = "9783319399546",

series = "Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)",

publisher = "Springer Verlag",

pages = "121--131",

editor = "Fidopiastis, {Cali M.} and Schmorrow, {Dylan D.}",

booktitle = "Foundations of Augmented Cognition",

address = "Germany",

}

Zao, JK, Jung, TP, Chang, HM, Gan, TT, Wang, YT, Lin, YP, Liu, WH, Zheng, GY, Lin, CK, Lin, CH, Chien, YY, Lin, FC, Huang, YP, Méndez, SJR & Medeiros, FA 2016, Augmenting VR/AR applications with EEG/EOG monitoring and oculo-vestibular recoupling. in CM Fidopiastis & DD Schmorrow (eds), Foundations of Augmented Cognition: Neuroergonomics and Operational Neuroscience - 10th International Conference, AC 2016 and Held as Part of HCI International 2016, Proceedings. Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics), vol. 9743, Springer Verlag, pp. 121-131, 10th International Conference on Foundations of Augmented Cognition: Neuroergonomics and Operational Neuroscience, AC 2016 and Held as Part of 18th International Conference on Human-Computer Interaction, HCI International 2016, Toronto, Canada, 17/07/16. https://doi.org/10.1007/978-3-319-39955-3_12

Augmenting VR/AR applications with EEG/EOG monitoring and oculo-vestibular recoupling. / Zao, John K.; Jung, Tzyy Ping; Chang, Hung Ming et al.
Foundations of Augmented Cognition: Neuroergonomics and Operational Neuroscience - 10th International Conference, AC 2016 and Held as Part of HCI International 2016, Proceedings. ed. / Cali M. Fidopiastis; Dylan D. Schmorrow. Springer Verlag, 2016. p. 121-131 (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics); Vol. 9743).

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution › peer-review

TY - GEN

T1 - Augmenting VR/AR applications with EEG/EOG monitoring and oculo-vestibular recoupling

AU - Zao, John K.

AU - Jung, Tzyy Ping

AU - Chang, Hung Ming

AU - Gan, Tchin Tze

AU - Wang, Yu Te

AU - Lin, Yuan Pin

AU - Liu, Wen Hao

AU - Zheng, Guang Yu

AU - Lin, Chin Kuo

AU - Lin, Chia Hung

AU - Chien, Yu Yi

AU - Lin, Fang Cheng

AU - Huang, Yi Pai

AU - Méndez, Sergio José Rodríguez

AU - Medeiros, Felipe A.

N1 - Publisher Copyright: © Springer International Publishing Switzerland 2016.

PY - 2016

Y1 - 2016

N2 - Head-mounted virtual reality and augmented reality displays (a.k.a. VR/AR goggles) created a revolutionary multimedia genre that is seeking ever-broadening applications and novel natural human interfaces. Adding neuromonitoring and neurofeedback to this genre is expected to introduce a new dimension to user interaction with the cyber-world. This study presents the development of a Neuromonitoring VR/AR Goggle armed with electroencephalo- gram and electrooculogram sensors, programmable milli-Ampere current stimulators and wireless fog/cloud computing support. Beside of its potential use in mitigating cybersickness, this device may have potential applications in augmented cognition ranging from feedback-controlled perceptual training to on-line learning and virtual social interactions. A prototype of the device has been made from a Samsung Gear VR for S6. This study explains its technical design to ensure precision data sampling, synchronous event marking, real-time signal processing and big data cloud computing support. This study also demonstrates the effective-ness in measuring the event-related potentials during a visual oddball experiment.

AB - Head-mounted virtual reality and augmented reality displays (a.k.a. VR/AR goggles) created a revolutionary multimedia genre that is seeking ever-broadening applications and novel natural human interfaces. Adding neuromonitoring and neurofeedback to this genre is expected to introduce a new dimension to user interaction with the cyber-world. This study presents the development of a Neuromonitoring VR/AR Goggle armed with electroencephalo- gram and electrooculogram sensors, programmable milli-Ampere current stimulators and wireless fog/cloud computing support. Beside of its potential use in mitigating cybersickness, this device may have potential applications in augmented cognition ranging from feedback-controlled perceptual training to on-line learning and virtual social interactions. A prototype of the device has been made from a Samsung Gear VR for S6. This study explains its technical design to ensure precision data sampling, synchronous event marking, real-time signal processing and big data cloud computing support. This study also demonstrates the effective-ness in measuring the event-related potentials during a visual oddball experiment.

KW - Augmented cognition

KW - Cybersickness

KW - Electroencephalography

KW - Electrooculography

KW - Mixed reality

KW - Oculo-vestibular recoupling

KW - Virtual reality

UR - http://www.scopus.com/inward/record.url?scp=84978877994&partnerID=8YFLogxK

U2 - 10.1007/978-3-319-39955-3_12

DO - 10.1007/978-3-319-39955-3_12

M3 - Conference contribution

AN - SCOPUS:84978877994

SN - 9783319399546

T3 - Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)

SP - 121

EP - 131

BT - Foundations of Augmented Cognition

A2 - Fidopiastis, Cali M.

A2 - Schmorrow, Dylan D.

PB - Springer Verlag

T2 - 10th International Conference on Foundations of Augmented Cognition: Neuroergonomics and Operational Neuroscience, AC 2016 and Held as Part of 18th International Conference on Human-Computer Interaction, HCI International 2016

Y2 - 17 July 2016 through 22 July 2016

ER -

Zao JK, Jung TP, Chang HM, Gan TT, Wang YT, Lin YP et al. Augmenting VR/AR applications with EEG/EOG monitoring and oculo-vestibular recoupling. In Fidopiastis CM, Schmorrow DD, editors, Foundations of Augmented Cognition: Neuroergonomics and Operational Neuroscience - 10th International Conference, AC 2016 and Held as Part of HCI International 2016, Proceedings. Springer Verlag. 2016. p. 121-131. (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)). doi: 10.1007/978-3-319-39955-3_12

Augmenting VR/AR applications with EEG/EOG monitoring and oculo-vestibular recoupling

Abstract

Publication series

Conference

Access to Document

Other files and links

Fingerprint

Cite this