Differences

This shows you the differences between two versions of the page.

--- jnl:rajan2019 [2020/05/12 02:12]
admin
+++ jnl:rajan2019 [2020/07/31 07:44] (current)
admin [Commentary]
@@ Line 1: / Line 1: @@
-=====Title=====
+=====Objective Assessment of Microsurgery Competency-In Search of a Validated Tool=====
-Objective Assessment of Microsurgery Competency-In Search of a Validated Tool.
 ====Abstract====
-Microsurgical skill acquisition is an integral component of training in plastic surgery. Current microsurgical training is based on the subjective Halstedian model. An ideal microsurgery assessment tool should be able to deconstruct all the subskills of microsurgery and assess them objectively and reliably. For our study, to analyze the feasibility, reliability, and validity of microsurgery skill assessment, a video-based objective structured assessment of technical skill tool was chosen. Two blinded experts evaluated 40 videos of six residents performing microsurgical anastomosis for arteriovenous fistula surgery. The generic Reznick's global rating score (GRS) and University of Western Ontario microsurgical skills acquisition/assessment (UWOMSA) instrument were used as checklists. Correlation coefficients of 0.75 to 0.80 (UWOMSA) and 0.71 to 0.77 (GRS) for interrater and intrarater reliability showed that the assessment tools were reliable. Convergent validity of the UWOMSA tool with the prevalidated GRS tool showed good agreement. The mean improvement of scores with years of residency was measured with analysis of variance. Both UWOMSA ( p -value: 0.034) and GRS ( p -value: 0.037) demonstrated significant improvement in scores from postgraduate year 1 (PGY1) to PGY2 and a less marked improvement from PGY2 to PGY3. We conclude that objective assessment of microsurgical skills in an actual clinical setting is feasible. Tools like UWOMSA are valid and reliable for microsurgery assessment and provide feedback to chart progression of learning. Acceptance and validation of such objective assessments will help to improve training and bring uniformity to microsurgery education.
+Microsurgical skill acquisition is an integral component of training in plastic surgery. Current microsurgical training is based on the subjective Halstedian model. An ideal microsurgery assessment tool should be able to deconstruct all the subskills of microsurgery and assess them objectively and reliably. For our study, to analyze the feasibility, reliability, and validity of microsurgery skill assessment, a video-based objective structured assessment of technical skill tool was chosen. Two blinded experts evaluated 40 videos of six residents performing microsurgical anastomosis for arteriovenous fistula surgery. The generic [[ac:grs-reznick|Reznick's global rating score (GRS)]] and University of Western Ontario microsurgical skills acquisition/assessment ([[ac:UWOMSA]]) instrument were used as checklists. Correlation coefficients of 0.75 to 0.80 (UWOMSA) and 0.71 to 0.77 ([[ac:grs-reznick|GRS]]) for interrater and intrarater reliability showed that the assessment tools were reliable. Convergent validity of the [[ac:UWOMSA]] tool with the prevalidated GRS tool showed good agreement. The mean improvement of scores with years of residency was measured with analysis of variance. Both UWOMSA ( p -value: 0.034) and GRS ( p -value: 0.037) demonstrated significant improvement in scores from postgraduate year 1 (PGY1) to PGY2 and a less marked improvement from PGY2 to PGY3. We conclude that objective assessment of microsurgical skills in an actual clinical setting is feasible. Tools like [[ac:UWOMSA]] are valid and reliable for microsurgery assessment and provide feedback to chart progression of learning. Acceptance and validation of such objective assessments will help to improve training and bring uniformity to microsurgery education.
 ====Commentary====
-Model used - AVF surgery in patients
+Model used
-Assessment - video assessment using GRS and UWOMSA
+  * AVF surgery in patients -- //in my view not a real microsurgery model//
-Good: - blinding, independent assessors
+Assessment
-- fairly good inter- and intra-rater reliability
+  * video assessment using GRS and [[ac:UWOMSA]]. see link for scores: https://www.ncbi.nlm.nih.gov/pmc/articles/PMC6785309/bin/10-1055-s-0039-1695658_s11118.pdf
-Issues:
+Good:
-  - Not stated in abstract: UWOMSA only statistically significant for knot-tying and not anastomosis
+  - blinding, independent assessors
+  - fairly good inter- and intra-rater reliability
+  - references look fairly complete
+Issues/Noteworthy:
+  - Uses GRS as a gold standard -- is that appropriate?
+  - Not stated in abstract: [[ac:UWOMSA]] only statistically significant for knot-tying and not anastomosis
   - How long was each anastomosis? How does time taken affect result?
-  - Use of different PGY years as comparison - may not be the best
+  - Use of different PGY years as comparison - may not be the best; assumption is that PGY 3 better than 2 better than 1
   - very small n for each group in PGY
+  - Inter-rater reliability seems better than intra-rater (time interval?) - //perhaps training the assessors would help//
+  - [[ac:UWOMSA]] anastomosis module only 0.53-0.56 R^2
+  - no data spread shown - only mean scores
+  - [[ac:UWOMSA]] with GRS - R2 is 0.71-0.74
+  - Strangely enough, the authors recommend [[ac:UWOMSA]], although GRS seems to have better intra- and inter-rater reliability. Perhaps this is because, as a tool to provide feedback, it is not useful. That is a weakness of global/ generic type scoring systems.
 ====Source=====
 Rajan, S., Sathyan, R., Sreelesh, L. S., Kallerey, A. A., Antharjanam, A., Sumitha, R., … Soumya, S. (2019). Objective Assessment of Microsurgery Competency-In Search of a Validated Tool. Indian Journal of Plastic Surgery : Official Publication of the Association of Plastic Surgeons of India, 52(2), 216–221. https://doi.org/10.1055/s-0039-1695658
+====See also====
+[[ac:microsurgery_assessment]]

Alphonsus Chong's Personal Wiki

User Tools

Site Tools

Differences

Page Tools