Purpose To examine how family physicians’, patients’ and trained clinical raters’ assessments of physician-patient communication compare in analysis of individual appointments. Methods Analysis of survey data from patients attending face-to-face appointments with 45 family physicians at 13 practices in England. Immediately post-appointment, patients and physicians independently completed a questionnaire including seven items assessing communication quality. A sample of videoed appointments was assessed by trained clinical raters, using the same seven communication items. Patient-, physician-, and rater-reported communication scores were compared using correlation coefficients. Results 503 physician-patient pairs were included; 55 of these appointments were also evaluated by trained clinical raters. Physicians scored themselves, on average, lower than patients (mean physician score 74.5; mean patient score 94.4); 63.4% (319) of patient-reported scores were the maximum of 100. The mean trained rater scores from 55 rated appointments was 57.3. There was a near-zero correlation coefficient between physician-reported and patient-reported communication scores (0.009, p=0.854), and between physician-reported and trained rater-reported communication scores (-0.006, p=0.69). However, there was a moderate and statistically significant association between patient and trained rater scores (0.35, p=0.042). Conclusions The lack of correlation between physician scores and those of others indicates that physicians’ perceptions of good communication within their appointments may differ from those of external peer raters and patients. Physicians may not be aware of how patients experience their communication practices; peer assessment of communication skills is an important approach in identifying areas for improvement.