The StepKinnection Test is an interactive system for the elderly that incorporates mechanisms to simultaneously perform a hybrid clinical test for fall risk assessment.
This clinical test includes a simple stepping task along with three voice-enabled cognitive activities allowing for the assessment of stepping performance under the dual-task paradigm.
The Step Kinnection Test, a game-like system that delivers step training exercises to older adults and simultaneously measure stepping performance through the use of a hybrid version of the Choice Stepping Reaction Time (CSRT) task , a time-based clinical test that has shown to reliably predict falls in older adults.
The main motivation for choosing the Kinect as the primary input device is that it allows for:
- a wider degree of freedom for the user;
- an intuitive and natural interaction with the game as no controllers or wearable sensors are required for its operation;
- a better provision of feedback allowing the display of a full body avatar to mirror the users’ movements.
All these features are ideal for elderly users with minimal or no computer literacy.
In addition to this, Step Kinnection also allows for the collection of spatial information such as postural sway and stepping accuracy. More importantly, the assessment of stepping performance under the dual task paradigm can also be achieved through the incorporation of a series of cognitive activities . Poor dual tasking has been frequently associated with falls and balance impairments in older people, providing evidence for the importance of specific cognitive functions in postural stability . These features make this system potentially useful in actual clinical practice to evaluate various dimensions involved in the diagnosis of fall risk in older people, all in a single system.
The Stepping Task
In order to start the game, the player is required to stand in front of a computer screen or TV connected to a Kinect PC. The representation of the player in the system is a pair of shoes mirroring the person’s feet. Six symmetrically distributed square-shaped panels are then drawn on the screen surrounding the player’s avatar(Figure 1). When one of the panels changes its color to green on the screen, the player is expected to step on it in space and back to the center as quickly as possible. As soon as the player returns to the initial position the process starts over. The sequence is presented randomly as well as the time between trials so that the user is unable to anticipate the time and location of the next stimulus. It is worth mentioning that these virtual panels are dynamically located based on the user’s height, making the stepping task equally challenging for short and tall participants. While playing, time based measurements such as reaction times are simultaneously collected. Also the positioning of the foot is recorded in order to determine the accuracy on stepping as well as the step length.
The Cognitive Tasks
In addition to the stepping game, three voice-controlled concurrent cognitive activities were incorporated to assess the performance of the patient under differing cognitive and motor conditions concurrently. The increased cognitive load affects the user performance while stepping, resulting in noticeably slower reaction times for users that are likely to fall.
The three activities are:
‘Read the Word’ Task:
During this task the user is required to say the color out loud while performing the stepping exercises. As the color of the word and its semantic meaning are identical, this task creates a minimally increased cognitive load for the user.
‘Name the Color’ Task:
For this task the semantic meaning of the word and the color of the word do not match. Once again the user has to say the color out loud, but in this case there is interference between the meaning and the color of the word. While the mind automatically determines the meaning of the word, the player actually needs to identify the color that the word is written in. This means the player needs to consciously re-evaluate his/her instinctive response. This interference, also known as the Stroop interference, results in a delay and the extra processing required normally results in a slowing down of the stepping test performance.
‘Maths Workout’ Task:
This task requires the user to answer a math question that is read by the system. While there is no interference effect as with the Stroop test, the user is still required to interpret what they have heard to answer the question.
For all the above tasks, the accuracy of the answer is automatically processed by the voice recognition system built into the Kinect.
- BANERJEE, T., KELLER, J.M., and SKUBIC, M., 2012. Resident identification using kinect depth image data and fuzzy clustering techniques. In Engineering in Medicine and Biology Society (EMBC), 2012 Annual International Conference of the IEEE, 5102-5105. DOI= http://dx.doi.org/10.1109/EMBC.2012.6347141.
- GANESAN, S. and ANTHONY, L., 2012. Using the kinect to encourage older adults to exercise: a prototype. In Proceedings of the 2012 ACM annual conference extended abstracts on Human Factors in Computing Systems Extended Abstracts ACM, 2297-2302.
- GERLING, K., LIVINGSTON, I., NACKE, L., and MANDRYK, R., 2012. Full-body motion-based game interaction for older adults. In Proceedings of the 2012 ACM annual conference on Human Factors in Computing Systems ACM, 1873-1882.
- HENNIG, B. and REITHINGER, N., 2012. Developing of a multimodal interactive training system in therapeutic calisthenics for elderly people. In KI 2012: Advances in Artificial Intelligence Springer, 61-72.
- KAYAMA, H., OKAMOTO, K., NISHIGUCHI, S., NAGAI, K., YAMADA, M., and AOYAMA, T., 2012. Concept Software Based on Kinect for Assessing Dual-Task Ability of Elderly People. GAMES FOR HEALTH: Research, Development, and Clinical Applications 1, 5, 348-352.
- KIM, J., SON, J., KO, N., and YOON, B., 2012. Unsupervised Virtual Reality-Based Exercise Program Improves Hip Muscles Strength and Balance Control in the Elderly: A Pilot Study. Archives of physical medicine and rehabilitation.
- LOHMANN, O., LUHMANN, T., and HEIN, A., 2012. Skeleton Timed Up and Go. In Bioinformatics and Biomedicine (BIBM), 2012 IEEE International Conference on IEEE, 1-5.
- LORD, S.R. and FITZPATRICK, R.C., 2001. Choice Stepping Reaction Time: A Composite Measure of Falls Risk in Older People. Journals of Gerontology Series A: Biological Sciences & Medical Sciences 56A, 10, M627.
- OBDRZALEK, S., KURILLO, G., OFLI, F., BAJCSY, R., SETO, E., JIMISON, H., and PAVEL, M., 2012. Accuracy and robustness of Kinect pose estimation in the context of coaching of elderly population. In Engineering in Medicine and Biology Society (EMBC), 2012 Annual International Conference of the IEEE IEEE, 1188-1193.
- PISAN, Y., GARCIA, J.A., and FELIX NAVARRO, K., 2013. Improving Lives: Using Microsoft Kinect to Predict the Loss of Balance for Elderly Users under Cognitive Load. In Proceedings of the Interactive Entertainment 2013 (IE2013) (RMIT University, Melbourne, Australia2013).
- RANTZ, M., SKUBIC, M., ABBOTT, C., PAK, Y., STONE, E.E., and MILLER, S.J., 2012. Automated Fall Risk Assessment and Detection in the Home: A Preliminary Investigation.
- STONE, E. and SKUBIC, M., 2011. Evaluation of an inexpensive depth camera for in-home gait assessment. Journal of Ambient Intelligence and Smart Environments 3, 4, 349-361.