Evaluating the Effectiveness of an Online Journal Club: Experience from the International Urology Journal Club

Isaac A. Thangasamy, Stacy Loeb, Niranjan J. Sathianathen, Michael Leveridge, Brian Stork, Benjamin J. Davies, Henry H. Woo*

*Corresponding author for this work

Research output: Contribution to journalArticlepeer-review

22 Citations (Scopus)


Background: The International Urology Journal Club attracts global participation; however, its effectiveness as an educational resource is undetermined. Objective: To evaluate the effectiveness of the international Twitter-based urology journal club (#urojc) using Kirkpatrick's evaluation model and to quantify the development of #urojc into a robust educational platform. Design, setting, and participants: A 13-item survey was sent to 3127 followers of #urojc. Twitter metrics including number of users, tweets, and impressions during 24 mo of operation were obtained via a third-party service “Symplur.” Tweets were qualitatively analyzed to assess for relevance. Kirkpatrick's four-level evaluation model (reaction, learning, behavior, and results) was applied to analyze impact and effectiveness. Results and limitations: Survey participation rate was 48% (after accounting for spam and inactive accounts). The reaction (Kirkpatrick level 1) to #urojc was overwhelmingly positive. Consultant urologists were the most active participants while trainees reported highest educational value. As much as 45% of junior consultant urologists and trainees reported a change in practice as a result of #urojc participation. The number of unique users of #urojc almost doubled from yr 1 to 2. The number of tweets per month and author participation increased from yr 1 to 2. The proportion of content-relevant tweets remained stably high at approximately 60%. This study was unable to document any impact on knowledge, behavior, or outcomes of #urojc (Kirkpatrick levels 2–4). Silent observers who do not follow #urojc were not captured. Conclusions: To our knowledge, this is the first time Kirkpatrick's evaluation model has been applied to a journal club. The reaction to #urojc among survey responders is positive with change in practice also reported. It appears that #urojc achieves level 1 of Kirkpatrick's evaluation model. Further investigation is required for levels 2–4. Twitter-based journal clubs draw robust global participation and have inspired journal clubs in other specialties. Patient summary: This report highlights the importance of the International Urology Journal Club as a potential educational resource of consultant urologists and trainees alike. Global participation was a major advantage of this platform. This journal club has inspired other medical specialties to follow suit.

Original languageEnglish
Pages (from-to)482-488
Number of pages7
JournalEuropean Urology Focus
Issue number2
Publication statusPublished - Mar 2021
Externally publishedYes


Dive into the research topics of 'Evaluating the Effectiveness of an Online Journal Club: Experience from the International Urology Journal Club'. Together they form a unique fingerprint.

Cite this