| Title |
Year |
Summary +/- Comments |
| 1. DEVELOPMENT |
| Patterson, F., Ferguson, E., Lane, P., Farrell, K., Martlew, J. and Wells, A., 2000. A competency model for general practice: implications for selection, training, and development. Br J Gen Pract, 50(452), pp.188-193. |
2000 |
In the UK, organisational psychology principles are used to identify and develop selection criteria and methods by identifying core and specialty specific competencies. Early work included this 2000 paper: 3 studies from a single Midlands based Practice N= 31-35, in 2000. 11 competencies underlying good GP were identified as necessary for job performance |
Article |
| Patterson, F., Ferguson, E. and Thomas, S., 2008. Using job analysis to identify core and specific competencies: implications for selection and recruitment. Medical Education, 42(12), pp.1195-1204. |
2008 |
Using job analysis in this way Patterson et al. (2008) later undertook three multisource, multimethod independent studies to explore core and specific competencies in anesthesia, obstetrics and gynecology and pediatrics. The outcome comprised 14 general competency domains common to all specialties. However, there were significant between‐specialty differences in both definitions of domains and the ratings of importance attached to them. |
Article |
| Patterson, F., Lievens, F., Kerrin, M., Munro, N. and Irish, B., 2013. The predictive validity of selection for entry into postgraduate training in general practice: evidence from three longitudinal studies. Br J Gen Pract, 63(616), pp.e734-e741. |
2013 |
The 2008 study was replicated by Patterson, Tavabie et al. (2013) to explore competencies for general practice training which resulted in 11 competency domains, of which empathy and perspective taking, communication skills, clinical knowledge and expertise and professional integrity were rated as the most important domains. |
Article |
| Patterson, F., Zibarras, L., Kerrin, M., Lopes, S. and Price, R., 2014. Development of competency models for assessors and simulators in high-stakes selection processes. Medical teacher, 36(12), pp.1082-1085. |
2014 |
Patterson et al.(2014) later extended this competency model approach to examine specific knowledge, skills and attributes linked to simulations in British GP Selection Centres (Stage 3 of the national Recruitment process). |
Article |
| 2. DESIGN, PILOT & IMPLEMENT |
| Burke, S., Davison, I., Brown, C., Bullock, A. and Kelly, S., 2005. The involvement of lay people in selection to general practice training schemes. Education for Primary Care, 16(4). pp. 450-457 |
2005 |
This 2005 paper showed high correlation between lay assessor and medical assessor scores, reflecting a shared agreement in the best candidates. In the main, these ‘Lay’ people were found to have some medical or educational connection. Lay assessors continue to be used in Selection Centres as part of the GP recruitment process, and many medical disciplines have now adopted a similar approach. |
Article |
| Patterson, F., Baron, H., Carr, V. Developing a national clinical selection test for General Practice. (Technical Report)(2006). |
2006 |
This documents the creation of a case bank of Clinical Problem Solving (CPST) items, and in 2007, one for the Situational Judgment Test (SJT). These item types are mapped to target competencies derived from job analysis, and together make up Stage 2 (machine marked test) of the national GP recruitment process. |
|
| Patterson, F., Baron, H., Carr, V., Plint, S. and Lane, P., 2009. Evaluation of three short‐listing methodologies for selection into postgraduate training in general practice. Medical education, 43(1), pp.50-57. |
2009 |
This compares three short-listing methodologies: clinical problem-solving test (CPST); structured application form (AFQ) and the novel situational judgement test (SJT). The optimum short-listing methodology was found to be a combination of the CPS and the SJT. The SJT replaced the AFQ in GP selection in 2007. The SJT/CPST combination remains the short-listing methodology for selection into GP training. NB this paper also describes the UK Selection Centre approach, referring to the three job-relevant simulations at the time (patient consultation, group and written simulation exercises). The simulations have since changed in type/format. |
Article |
| 3. EVALUATION & VALIDATION |
| Patterson, F., Baron, H., Carr, V., Plint, S. and Lane, P., 2009. Evaluation of three short‐listing methodologies for selection into postgraduate training in general practice. Medical education, 43(1), pp.50-57. |
2009 |
SJT Reliability. This paper is described above, but included in this section for a different function. In this pilot SJT internal reliability was reported in the range of r.0.80–0.83 |
Article |
| Patterson, F., Carr, V., Zibarras, L., Burr, B., Berkin, L., Plint, S., Irish, B. and Gregory, S., 2009. New machine-marked tests for selection into core medical training: evidence from two validation studies. Clinical Medicine, 9(5), pp.417-420. |
2009 |
CPST reliability shown to be good. r.0.85–0.89
NB CPST correlation with the SJT shown to be varying in the range of r.0.39 to r.0.53 (Patterson, Baron, et al. 2009; (Patterson,
Lievens, et al. 2013). |
Article |
| Ahmed, H., Rhydderch, M. and Matthews, P., 2012. Can knowledge tests and situational judgement tests predict selection centre performance?. Medical education, 46(8), pp.777-784. |
2012 |
The CPST and SJT are shown to be predictive of performance in job simulations at Selection Centre (SC). The study provide evidence for their continued use in selection for general practice training. Subsequently, their predictive validity was shown to hold good for exit level GP training. |
Article |
| Koczwara, A., Patterson, F., Zibarras, L., Kerrin, M., Irish, B. and Wilkinson, M., 2012. Evaluating cognitive ability, knowledge tests and situational judgement tests for postgraduate selection. Medical education, 46(4), pp.399-408. |
2012 |
In terms of operational validity and candidate acceptance, the combination of the current CPST and SJT proved to be the most effective administration of tests in predicting selection outcomes. In terms of construct validity, the SJT was the best single predictor of subsequent performance in the Selection Centre (SC). |
Article |
| Patterson, F., Zibarras, L., Kerrin, M., Lopes, S. and Price, R., 2014. Development of competency models for assessors and simulators in high-stakes selection processes. Medical teacher, 36(12), pp.1082-1085. |
2014 |
This article on selection in the context of an assessment or selection centre is a qualitative study exploring competency models to improve uniformity and calibration of the overall process. Most of the world literature was (and remains in 2018) quantitative, so this is unusual. |
Article |
| 4. ACCEPTABILITY & FEASIBILITY |
| Patterson, F., Zibarras, L., Carr, V., Irish, B. and Gregory, S., 2011. Evaluating candidate reactions to selection practices using organisational justice theory. Medical education, 45(3), pp.289-297. |
2011 |
Candidates generally have positive reactions during both short-listing and SC phases. All selection methods were considered job relevant and both phases fair. The CPST was considered more job relevant than the SJT, and the SC phase was rated as fairer than the short-listing stage. These findings supported previous research in other occupational settings. Candidate reactions were found to improve between 2007 and 2009 for short listing (machine marked) methods whilst the then-current CS was consistently rated as the most job-relevant selection method. Since this study SJTs have become even more widely accepted and are now used routinely from the undergrad stage. |
Article |
| Koczwara, A., Patterson, F., Zibarras, L., Kerrin, M., Irish, B. and Wilkinson, M., 2012. Evaluating cognitive ability, knowledge tests and situational judgement tests for postgraduate selection. Medical education, 46(4), pp.399-408. |
2012 |
This paper is described above, and included in this section for a different function. In this large sample (n.2947), there was a good agreement among respondents (>60%) that the content of SJTs was clearly relevant to GP training and appropriate for the entry level they were applying for. One third agreed that the test gave them sufficient opportunity to indicate their ability for training and would help selectors differentiate between applicants. |
Article |
| Patterson, F., Knight, A., Dowell, J., Nicholson, S., Cousans, F. and Cleland, J., 2016. How effective are selection methods in medical education? A systematic review. Medical Education, 50(1), pp.36-60. |
2016 |
A systematic review of selection methods reported that there is acceptability evidence in the organizational psychology literature for the SJT (not specific to this exam). |
Article |
| 5. PREDICTIVE VALIDITY |
| hmed, H., Rhydderch, M. and Matthews, P., 2012. Do general practice selection scores predict success at MRCGP? An exploratory study. Education for Primary Care, 23(2), pp.95-100. |
2012 |
The score for overall performance at selection achieved statistically significant correlation with examination performance (r.0.49) for the Applied Knowledge Test (AKT) and r.0.53 for the Clinical Skills Assessment (CSA) modules of the exit, licensing MRCGP exams. |
Article |
| Patterson, F., Lievens, F., Kerrin, M., Munro, N. and Irish, B., 2013. The predictive validity of selection for entry into postgraduate training in general practice: evidence from three longitudinal studies. Br J Gen Pract, 63(616), pp.e734-e741. |
2013 |
Patterson et al. found that the CPST and the SJT correlate with the AKT at r=0.73 and 0.43, respectively (uncorrected), and the CSA at r=0.38 and 0.43, respectively (uncorrected). The Selection Centre was found to correlate with the CSA at r=0.32 |
Article |
| Patterson, F., Kerrin, M., Baron, H. and Lopes, S., 2015. Exploring the relationship between general practice selection scores and MRCGP examination performance. Final Report to General Medical Council. |
2015 |
This detailed analysis for the GMC showed findings comparable to, and replicating, previous validity study results.
The CPST and SJT correlate 0.73 and 0.46 with the AKT, again uncorrected for restriction of range. The correlations for the CSA are 0.49 and 0.54, respectively, which is slightly stronger than previously found. Similarly, scores from the Selection Centre correlate at r=0.42 (uncorrected) with the CSA.
All of these results, including previously reported findings, are statistically significant at p < 0.001. Findings predicted applicants with top CPST/SJT test scores as also doing well at SC. The suggestion was made to ‘fast track’ those candidates into offers of training, bypassing the SC. |
Article |
| 6. REVIEWS |
| Davison, I., Mcmanus, C., and Taylor, C., 2016. Evaluation of GP Specialty Selection [Online]. |
2016 |
An in-depth review of the national selection process for GP training at that time, with reference to cost effectiveness and suggestions to consider for continued quality improvement. Overall affirming, with significant constructive challenges that were subsequently considered and, where appropriate, addressed. |
Article |
| Roberts, C., Khanna, P., Rigby, L., Bartle, E., Llewellyn, A., Gustavs, J., Newton, L., Newcombe, J.P., Davies, M., Thistlethwaite, J. and Lynam, J., 2018. Utility of selection methods for specialist medical training: A BEME (best evidence medical education) systematic review: BEME guide no. 45. Medical teacher, 40(1), pp.3-19. |
2017 |
A systematic review of international measures of selection in postgraduate medicine. Very affirming for the UK GP system. Offers suggestions for a research agenda which resonates with the 2018 updated Ottawa consensus statement. |
Article |
| Patterson, F., Roberts, C., Hanson, M.D., Hampe, W., Eva, K., Ponnamperuma, G., Magzoub, M., Tekian, A. and Cleland, J., 2018. 2018 Ottawa consensus statement: Selection and recruitment to the healthcare professions. Medical teacher, pp.1-11. |
2018 |
Latest Ottawa consensus statement. This notes that most research has been on the psychometric behaviour of selection instruments, calling for more discussion as to the strategic aims which should underpin development. |
Article |