Using on-line surveys to measure three key constructs of the quality of human-computer interaction in web sites

Psychometric properties and implications

Paul Van Schaik, Jonathan Ling

Research output: Contribution to journalArticleResearchpeer-review

31 Citations (Scopus)

Abstract

On-line surveys are now an important tool for data collection on the World Wide Web (the Web). Determining the psychometric properties of key constructs such as disorientation, ease of use and flow is of paramount importance in establishing the quality of users' interactions with web sites. The current study used techniques of experimental research and on-line surveys to investigate the psychometric properties of existing instruments for measuring these constructs using two response formats: visual analogue scale and Likert scale. A 2 × 2 design with response format and orientation support as independent variables was used. Ninety participants carried out an information retrieval task using an experimental on-line shopping site before completing the scales. Factor analysis confirmed the existence of three distinct scales that possessed high reliability. Evidence for validity, and to a lesser extent, sensitivity, was found. Although psychometric results generally converged, some differences between the two response formats were found. A framework for the comprehensive investigation of response formats of on-line questionnaires is proposed as a basis for future research. Practical implications for the on-line measurement of the quality of users' interactions with web-based systems are discussed.

Original languageEnglish
Pages (from-to)545-567
Number of pages23
JournalInternational Journal of Human Computer Studies
Volume59
Issue number5
DOIs
Publication statusPublished - 1 Jan 2003

Fingerprint

Human computer interaction
World Wide Web
Websites
Factor analysis
Information retrieval

Cite this

@article{3473fca9c7f947f3a33372d470feabd7,
title = "Using on-line surveys to measure three key constructs of the quality of human-computer interaction in web sites: Psychometric properties and implications",
abstract = "On-line surveys are now an important tool for data collection on the World Wide Web (the Web). Determining the psychometric properties of key constructs such as disorientation, ease of use and flow is of paramount importance in establishing the quality of users' interactions with web sites. The current study used techniques of experimental research and on-line surveys to investigate the psychometric properties of existing instruments for measuring these constructs using two response formats: visual analogue scale and Likert scale. A 2 × 2 design with response format and orientation support as independent variables was used. Ninety participants carried out an information retrieval task using an experimental on-line shopping site before completing the scales. Factor analysis confirmed the existence of three distinct scales that possessed high reliability. Evidence for validity, and to a lesser extent, sensitivity, was found. Although psychometric results generally converged, some differences between the two response formats were found. A framework for the comprehensive investigation of response formats of on-line questionnaires is proposed as a basis for future research. Practical implications for the on-line measurement of the quality of users' interactions with web-based systems are discussed.",
author = "{Van Schaik}, Paul and Jonathan Ling",
year = "2003",
month = "1",
day = "1",
doi = "10.1016/S1071-5819(03)00078-8",
language = "English",
volume = "59",
pages = "545--567",
journal = "International Journal of Human Computer Studies",
issn = "1071-5819",
publisher = "Elsevier",
number = "5",

}

Using on-line surveys to measure three key constructs of the quality of human-computer interaction in web sites : Psychometric properties and implications. / Van Schaik, Paul; Ling, Jonathan.

In: International Journal of Human Computer Studies, Vol. 59, No. 5, 01.01.2003, p. 545-567.

Research output: Contribution to journalArticleResearchpeer-review

TY - JOUR

T1 - Using on-line surveys to measure three key constructs of the quality of human-computer interaction in web sites

T2 - Psychometric properties and implications

AU - Van Schaik, Paul

AU - Ling, Jonathan

PY - 2003/1/1

Y1 - 2003/1/1

N2 - On-line surveys are now an important tool for data collection on the World Wide Web (the Web). Determining the psychometric properties of key constructs such as disorientation, ease of use and flow is of paramount importance in establishing the quality of users' interactions with web sites. The current study used techniques of experimental research and on-line surveys to investigate the psychometric properties of existing instruments for measuring these constructs using two response formats: visual analogue scale and Likert scale. A 2 × 2 design with response format and orientation support as independent variables was used. Ninety participants carried out an information retrieval task using an experimental on-line shopping site before completing the scales. Factor analysis confirmed the existence of three distinct scales that possessed high reliability. Evidence for validity, and to a lesser extent, sensitivity, was found. Although psychometric results generally converged, some differences between the two response formats were found. A framework for the comprehensive investigation of response formats of on-line questionnaires is proposed as a basis for future research. Practical implications for the on-line measurement of the quality of users' interactions with web-based systems are discussed.

AB - On-line surveys are now an important tool for data collection on the World Wide Web (the Web). Determining the psychometric properties of key constructs such as disorientation, ease of use and flow is of paramount importance in establishing the quality of users' interactions with web sites. The current study used techniques of experimental research and on-line surveys to investigate the psychometric properties of existing instruments for measuring these constructs using two response formats: visual analogue scale and Likert scale. A 2 × 2 design with response format and orientation support as independent variables was used. Ninety participants carried out an information retrieval task using an experimental on-line shopping site before completing the scales. Factor analysis confirmed the existence of three distinct scales that possessed high reliability. Evidence for validity, and to a lesser extent, sensitivity, was found. Although psychometric results generally converged, some differences between the two response formats were found. A framework for the comprehensive investigation of response formats of on-line questionnaires is proposed as a basis for future research. Practical implications for the on-line measurement of the quality of users' interactions with web-based systems are discussed.

UR - http://www.scopus.com/inward/record.url?scp=0242303033&partnerID=8YFLogxK

U2 - 10.1016/S1071-5819(03)00078-8

DO - 10.1016/S1071-5819(03)00078-8

M3 - Article

VL - 59

SP - 545

EP - 567

JO - International Journal of Human Computer Studies

JF - International Journal of Human Computer Studies

SN - 1071-5819

IS - 5

ER -