TY - UNPB
T1 - Expert Evaluation of ChatGPT Performance for Risk Management Process based on ISO 31000 Standard
AU - Al-Mhdawi, M.K.S.
AU - Qazi, Abroon
AU - Alzarrad, Ammar
AU - Dacre, Nicholas
AU - Rahimian, Farzad
AU - Buniya, Mohanad K.
AU - Zhang, Hanqin
PY - 2023/7/8
Y1 - 2023/7/8
N2 - ChatGPT is widely known for its ability to facilitate knowledge exchange, support research endeavours, and enhance problem-solving across various scientific disciplines. However, to date, no empirical research has been undertaken to evaluate ChatGPT's performance against established standards or professional guidelines. Consequently, the present study aims to evaluate the performance of ChatGPT for the risk management (RM) process based on ISO 31000 standard using expert evaluation. The authors (1) identified the key indicators for measuring the performance of ChatGPT in managing construction risks based on ISO 31000 and determined the key assessment criteria for evaluating the identified indicators using a focus group session with Iraqi experts; and (2) quantitatively analysed the level of performance of ChatGPT under a fuzzy environment. The findings indicated that ChatGPT's overall performance was high. Specifically, its ability to provide relevant risk mitigation strategies was identified as its strongest aspect. However, the research also revealed that ChatGPT's consistency in risk assessment and prioritization was the least effective aspect. This research serves as a foundation for future studies and developments in the field of AI-driven risk management, advancing our theoretical understanding of the application of AI models like ChatGPT in real-world risk scenarios.
AB - ChatGPT is widely known for its ability to facilitate knowledge exchange, support research endeavours, and enhance problem-solving across various scientific disciplines. However, to date, no empirical research has been undertaken to evaluate ChatGPT's performance against established standards or professional guidelines. Consequently, the present study aims to evaluate the performance of ChatGPT for the risk management (RM) process based on ISO 31000 standard using expert evaluation. The authors (1) identified the key indicators for measuring the performance of ChatGPT in managing construction risks based on ISO 31000 and determined the key assessment criteria for evaluating the identified indicators using a focus group session with Iraqi experts; and (2) quantitatively analysed the level of performance of ChatGPT under a fuzzy environment. The findings indicated that ChatGPT's overall performance was high. Specifically, its ability to provide relevant risk mitigation strategies was identified as its strongest aspect. However, the research also revealed that ChatGPT's consistency in risk assessment and prioritization was the least effective aspect. This research serves as a foundation for future studies and developments in the field of AI-driven risk management, advancing our theoretical understanding of the application of AI models like ChatGPT in real-world risk scenarios.
UR - http://dx.doi.org/10.2139/ssrn.4504409
U2 - 10.2139/ssrn.4504409
DO - 10.2139/ssrn.4504409
M3 - Preprint
T3 - SSRN Electronic Journal
BT - Expert Evaluation of ChatGPT Performance for Risk Management Process based on ISO 31000 Standard
PB - Elsevier
ER -