TY - JOUR
T1 - Evaluating the Effectiveness of an Online Journal Club
T2 - Experience from the International Urology Journal Club
AU - Thangasamy, Isaac A.
AU - Loeb, Stacy
AU - Sathianathen, Niranjan J.
AU - Leveridge, Michael
AU - Stork, Brian
AU - Davies, Benjamin J.
AU - Woo, Henry H.
N1 - Publisher Copyright:
© 2019 European Association of Urology
PY - 2021/3
Y1 - 2021/3
N2 - Background: The International Urology Journal Club attracts global participation; however, its effectiveness as an educational resource is undetermined. Objective: To evaluate the effectiveness of the international Twitter-based urology journal club (#urojc) using Kirkpatrick's evaluation model and to quantify the development of #urojc into a robust educational platform. Design, setting, and participants: A 13-item survey was sent to 3127 followers of #urojc. Twitter metrics including number of users, tweets, and impressions during 24 mo of operation were obtained via a third-party service “Symplur.” Tweets were qualitatively analyzed to assess for relevance. Kirkpatrick's four-level evaluation model (reaction, learning, behavior, and results) was applied to analyze impact and effectiveness. Results and limitations: Survey participation rate was 48% (after accounting for spam and inactive accounts). The reaction (Kirkpatrick level 1) to #urojc was overwhelmingly positive. Consultant urologists were the most active participants while trainees reported highest educational value. As much as 45% of junior consultant urologists and trainees reported a change in practice as a result of #urojc participation. The number of unique users of #urojc almost doubled from yr 1 to 2. The number of tweets per month and author participation increased from yr 1 to 2. The proportion of content-relevant tweets remained stably high at approximately 60%. This study was unable to document any impact on knowledge, behavior, or outcomes of #urojc (Kirkpatrick levels 2–4). Silent observers who do not follow #urojc were not captured. Conclusions: To our knowledge, this is the first time Kirkpatrick's evaluation model has been applied to a journal club. The reaction to #urojc among survey responders is positive with change in practice also reported. It appears that #urojc achieves level 1 of Kirkpatrick's evaluation model. Further investigation is required for levels 2–4. Twitter-based journal clubs draw robust global participation and have inspired journal clubs in other specialties. Patient summary: This report highlights the importance of the International Urology Journal Club as a potential educational resource of consultant urologists and trainees alike. Global participation was a major advantage of this platform. This journal club has inspired other medical specialties to follow suit.
AB - Background: The International Urology Journal Club attracts global participation; however, its effectiveness as an educational resource is undetermined. Objective: To evaluate the effectiveness of the international Twitter-based urology journal club (#urojc) using Kirkpatrick's evaluation model and to quantify the development of #urojc into a robust educational platform. Design, setting, and participants: A 13-item survey was sent to 3127 followers of #urojc. Twitter metrics including number of users, tweets, and impressions during 24 mo of operation were obtained via a third-party service “Symplur.” Tweets were qualitatively analyzed to assess for relevance. Kirkpatrick's four-level evaluation model (reaction, learning, behavior, and results) was applied to analyze impact and effectiveness. Results and limitations: Survey participation rate was 48% (after accounting for spam and inactive accounts). The reaction (Kirkpatrick level 1) to #urojc was overwhelmingly positive. Consultant urologists were the most active participants while trainees reported highest educational value. As much as 45% of junior consultant urologists and trainees reported a change in practice as a result of #urojc participation. The number of unique users of #urojc almost doubled from yr 1 to 2. The number of tweets per month and author participation increased from yr 1 to 2. The proportion of content-relevant tweets remained stably high at approximately 60%. This study was unable to document any impact on knowledge, behavior, or outcomes of #urojc (Kirkpatrick levels 2–4). Silent observers who do not follow #urojc were not captured. Conclusions: To our knowledge, this is the first time Kirkpatrick's evaluation model has been applied to a journal club. The reaction to #urojc among survey responders is positive with change in practice also reported. It appears that #urojc achieves level 1 of Kirkpatrick's evaluation model. Further investigation is required for levels 2–4. Twitter-based journal clubs draw robust global participation and have inspired journal clubs in other specialties. Patient summary: This report highlights the importance of the International Urology Journal Club as a potential educational resource of consultant urologists and trainees alike. Global participation was a major advantage of this platform. This journal club has inspired other medical specialties to follow suit.
KW - Education
KW - Evaluation
KW - Journal club
KW - Kirkpatrick
KW - Social media
KW - Urology
UR - http://www.scopus.com/inward/record.url?scp=85074495336&partnerID=8YFLogxK
U2 - 10.1016/j.euf.2019.10.006
DO - 10.1016/j.euf.2019.10.006
M3 - Article
SN - 2405-4569
VL - 7
SP - 482
EP - 488
JO - European Urology Focus
JF - European Urology Focus
IS - 2
ER -