Skip to main content
← Back to Library

Your Performance Review Says More About Your Manager

Watch Video on YouTube
📄 The written version, with every source and every limit: https://keca.co.uk/articles/performance-review-rater-psychology Same work, new manager, lower rating. A pricing analyst is rated exceeds expectations for two years. Her work doesn't change, her manager does, and her rating drops. Organisational psychologist Dr Nick Keca on why a performance review usually says more about the person who wrote it than about the person it describes, and what to do about it. In this episode: how much of a performance rating belongs to the rater, in two samples of more than 2,000 managers each; why the size of that share is argued over, and why different isn't the same as wrong; the rater's temperament and the Big Five; why ratings that decide pay are more generous, not stricter; the halo around the people a manager hired; why the similarity that counts is the sense of thinking alike, not shared age, gender or background; homophily, networks and the bank where breaks with the boss moved careers; the merit story; and what the evidence does and doesn't say about the darker personality traits in raters. The final third is practical: reading a rating as a sentence about two people, checking your own self-rating, getting a second view, treating a new manager as an event, four questions for anybody who rates, why feedback can make performance worse, and how an organisation stops any one rater's psychology deciding a career. Chapters 0:00 Same work, new manager, lower performance rating 1:21 Is your manager biased, or do you need more visibility? 2:14 Idiosyncratic rater effect: how much of a rating is the rater 3:52 Rater disagreement: different isn't the same as wrong 5:21 Rater personality: agreeableness and lenient ratings 7:15 Rating purpose and escalation of commitment 8:46 Similarity bias: the manager who thinks you're like them 10:39 Homophily and visibility: who gets seen at work 12:08 Manager insecurity, the merit story and the Dark Triad 13:34 How to read your performance review 14:57 Get a second view and handle a new manager 16:19 How to rate fairly: four questions for managers 17:45 Reducing rater bias in performance management 19:27 What the research can't tell you about your manager 20:34 The full article, sources and limitations Educational content. Not psychological, legal or employment advice. Nothing in this video diagnoses anybody or describes any real manager. The case that runs through the video is a composite. The presenter is an AI-generated avatar and the music is generated; the narration is Nick Keca's own recorded voice.
/ Official Transcript

Full Video Text

For two years, a pricing analyst was rated as exceeds expectations, the highest in her team. Then her manager changed, her work didn't, her rating dropped to meets expectations. Nobody lied, both managers were honest and competent and each would say his rating described her work performance. Here's the difficulty, a performance rating usually says more about the person giving it than about the person receiving it.

In this episode, I'll show you how much of the rating belongs to the rater, what raters bring to it from their temperament to the sense that you're like them, what personality adds on both sides of the form and what changes it, whether you're being rated or doing the rating. Her first manager was a former actuary, careful and quiet, he read every model line by line.

She was quiet too and just as careful. After the merger, she reported to the head of underwriting, a fluent energetic presenter who argued for someone else to take her role. She built the same models to the same deadlines. Her review said her technical work was strong and that she needed more strategic presence and visibility.

She knew she didn't. so she did what most of us do she took it as information about herself and tried to be more visible her models got checked less carefully two small errors reached a client file and her next rating was lower again the usual question is whether her new manager is biased or whether she just needs to be more visible the more useful question is this when two capable people watch the same year of work and reach different verdicts how much of each verdict describes the work and how much describes the person who wrote it her story is a composite drawn from research and practice i'm nick kecker and this is what the research actually says How much of a rating is really the rater?

The clearest test took two ratings apart. Researchers studied two large groups of managers, more than 2,000 in each. Every manager was rated by two bosses, two peers, two of their own staff and themselves. Let's take those ratings apart. The biggest single part of the differences between ratings came from the rater, from their own way of seeing and scoring people, 62% in the first group and 53% in the second.

Continue reading transcript (36 more paragraphs)

The performance of the person being rated? 21% and 25% respectively. Read plainly, those ratings carried more than twice as much of the person giving the rating as of the person receiving it. That's the finding this episode is built on. The researchers call this the idiosyncratic rater effect. It's the part of a rating that comes from the particular rater, not from you.

And here's what surprised me. The differences between bosses, peers and staff as groups were small. It mattered much less which kind of rater it was than which rater it was. The second way asks how far two supervisors agree when they rate the same person. For 25 years, the answer quoted was moderate at best.

For our analysts, that's the whole question. Her work stayed the same. Her rater changed. You have to stand Before we go further, two cautions. First, the size of that share is argued over. In 2024, two large reviews put agreement between supervisors higher than the older figures. That matters. But no estimate I could find makes the rater a small part of the rating.

A re-analysis of supervisor ratings still found the rater's part larger than the part rater's shared. Second, different isn't the same as wrong. One manager sees you handle the board. Another sees you handle a crisis. Each can be right about what they saw. In a Scandinavian service firm, part of the spread in supervisors' ratings reflected how good the supervisors were at managing.

Not just how lenient they were. Some managers genuinely bring out better or worse work outcomes. So, two honest raters can disagree for three reasons. They see things differently. They manage differently. Or because of what each brings to the judgement. A single rating can't tell you which. This episode is about the third.

It's not a claim that managers are biased. And it's not a personality test. Nothing here tells you what your manager is like. And where I'm offering judgement from practice rather than evidence-based research, I'll say so. Thanks. What do raters bring to the process? Let's start with temperament. Some managers rate generously and some rate hard, and the difference tends to follow them from team to team.

Personality research describes people on five broad traits known as the big five. One of them is agreeableness, meaning warmth and a wish to keep relationships smooth. Pooling 28 studies, raters' personalities accounted for between 6 and 22% of the differences in the ratings they gave, depending on the setting, and an agreeable rater was most generous.

In a smaller study, raters who were agreeable and low in conscientiousness were the most generous of all. Most people assume a generous rater is a soft judge, and a tough rater is an accurate one. The evidence supports only half of that. A tough rater isn't necessarily more accurate. Low emotional stability shows up in the ratings people give too, and anxiety about your own position isn't calibration.

Her first manager was, in my experience of people like him, a careful, conscientious rater who rarely gave top marks, and gave them to her twice. Her second manager was warm and expressive with people he knew well, so his lower rating is hard to read as temperament alone. His generosity was going somewhere else.

That's judgment from practice. So know the rater before you read the rating. A meets expectations from someone who rarely gives more isn't the same verdict as a meets expectations from somebody who gives most of their team more. Organisations hold that information, and few of the people being rated ever see it. kalianぇ Thank Next, Purpose A rating is often written to do something and what it's written to do shapes what it says Most people assume a rating with real consequences is made more carefully The evidence suggests a reverse Across 22 studies covering 57,775 people ratings used for pay and promotion were more generous than ratings given for development and the gap was larger in real organisations than in the lab In interviews with 60 executives some described inflating a problem employee's rating to move them out and others deflating a rating to build a case Those are their own accounts so they can't tell us how often it happens Then the person the rater chose In a large public sector organisation Supervisors rated more favourably the employees they'd helped to hire and more weakly, less favourably, the ones they'd argued against Psychologists call this escalation of commitment which means backing a choice because you made it Remember the head of underwriting argued for someone else in her role?

That doesn't show he marked her down and marking down is the weaker half of the evidence But it was the first rating where her rater had something invested in her not being the right choice That the Now the one that surprised me most, the person who seems like you. Most people worry about visible similarity, like sharing a manager's age, gender or background.

A 2026 review pooled 82 samples of managers and their staff. Similarity in age, gender, race and education showed no link with performance ratings. But similarity in values, personality and attitudes did. Perceived similarity was associated with performance ratings nearly twice as strongly as measured similarity.

An earlier field study found the same split. Measured personality similarity between salespeople and their raters was largely unrelated to rating. The rater's sense of similarity was strongly related to it. And in interviews with 120 evaluators at elite banks, law firms and consultancies, more than half-ranked fit as the most important thing at interview, using their own biographies as a template of merit.

That's hiring, not appraisal. But it's the same kind of people who later write the reviews. Put simply, it's not being like your manager that seems to count. It's your manager thinking you are. Two cautions. Most of these studies measured everything at once. And a good working relationship may create the sense of being alike, rather than the other way around.

And some of it may be real. Two people who genuinely share standards may simply work better together. In my experience, a manager's development advice is often a portrait of themselves. Strategic presence describes how her new manager does his job. Thank A rater can only rate what reaches them, and that depends on the networks and personality as well as the work.

Ties at work form disproportionately between similar people. It's called homophily, and it's one way the sense of similarity gets built. Here's the most striking evidence. In a commercial bank with 14,638 staff, managers were rotated in a way employees didn't choose. Men moved from a female to a male manager rose about 0.6 of a pay grade further over the next 10 quarters.

Smokers moving to a smoking manager gained a similar amount. They didn't work longer or sell more. What changed was how often they took breaks with a manager. That's promotion, not a rating. But the root is contact. Personality plays a part too. People high in self-monitoring, meaning they read the room and adjusted how they came across, were more likely to sit at the centre of their workplace networks.

But being out of sight isn't automatically costly. In a randomised trial at a technology firm, 1612 graduates working from home two days a week got the same performance grades and promotions over two years and quit less. Several people fed into those reviews. They and two more briefly the rater's own security when managers confidence in their own ability was low whether measured in an energy company or lowered in an experiment they rated the people who spoke up less favorably that study was about people offering ideas and challenges so the step to every appraisal is an extension and the merit story across three experiments with 445 participants most of whom had managed people those told their company valued merit gave an equally rated man a larger bonus than a woman calling a system meritocratic can reassure raters the checking's already been done what about the darker personality traits the honest answer is that the evidence is thin one study of 87 supervisors linked the score on the dark triad meaning narcissism machiavellianism and psychopathy to poorer relationships within the team i couldn't find a single study testing whether managers high in these traits rate less accurately most of what we've needs no dark trait at all that's less satisfying and more useful because it means the fixes apply to every rater objective visionors now is not on a So, what can you do about it?

Here's each move and what it rests on. If you're being rated, read the rating as a sentence about two people. Before deciding what it says about you, ask what you know about the rater. How do they rate everyone else? What's the rating for? Did they choose you? How long have they known your work? That doesn't make a poor rating wrong.

It tells you how much weight one person's view can carry. Our analyst read hers as a verdict on herself and she changed the wrong thing. Then start with your own view, because it's probably the generous one. Pooling studies of more than 37,000 people, self-ratings agreed only modestly with supervisors' ratings and in Western samples they ran clearly more generous.

People higher in narcissism do it most on qualities relating to getting ahead, like leadership. The same review found that self and supervisor ratings agreed more when they rested on concrete performance indicators. So, keep your own dated record of what you delivered, so that a conversation about presents can be brought back to the work.

The end of the year's Next, look for the second view. Because two raters agree only moderately, one rating is a weak basis for any conclusion. The view of someone else who's senior work, a former manager, a client or a peer, is the cheapest correction there is and it adds information rather than disputing theirs.

That's judgment from practice, built on the agreement evidence. Let more than one person see the work. Quieter people are rated on less of it. The answer isn't imitating an inauthentic style, it's making sure your work is known to more than one person who could speak to it. In the trial, working out of sight didn't cost people their grades, where several people fed into the review.

And treat a change of rater as an event. A new manager is when the rater's share of your rating changes most and when a first impression forms, it's difficult to change. Managers who believe people's abilities are fixed are less likely to notice when someone improves. In the first months, show your work in concrete terms and ask early what they'll be looking for.

If our analyst had done that in January, she'd have a record, a second voice and a conversation about her work. Let's If you do the rating, everything we've covered applies to you. That includes me. Write down why for somebody else to read. Raters who had to justify their ratings in writing were more accurate and accountability weakened the pull of raters' personalities.

A warning from the same experiment. Raters told their earlier ratings had been too low simply gave higher ones. Accountability works when it asks for reasons and fails when it signals the answer. Then ask yourself four uncomfortable questions. Would I rate this the same if I'd chosen this person or hadn't?

Would I rate it the same if they didn't remind me of myself? Is the person I'm about to mark down the one who most often challenges me? And would I give the same rating to a stranger who'd done the same work? That's judgement from practice. None of the four has been tested. And remember that feedback can make things worse.

Across 607 results, feedback improved performance on average, but more than a third of feedback interventions made performance worse. On the author's own theory, a rating delivered as a verdict on the person rather than the work is the kind most likely to land in that third. There's a If you run the organisation, you can't train the psychology out of your raters, but you can stop any single rater's psychology from deciding someone's career.

Because the rater's share is large, averaging several raters dilutes any one rater's tendencies. Because ratings that decide pay a softer, asking one rating to serve both development and reward asks it to do two jobs that pull apart. Because a merit story without accountability went with more bias, the protection is somebody checking how ratings turn into pay.

In one large employer, the same rating brought women and ethnic minority staff smaller pay rises than white men, about half of 1% a year, every year. And because the similarity that counts is perceived, a criterion like fit or presence invites exactly that perception onto the form. Define good performance in terms of the work.

From practice, four design choices. More than one rater, including one from outside the person's circle. Reasons that a second person reads. Development and pay kept apart where you can. And check on what ratings turn into by group. And watch the moments when the rater's share is largest. A new manager.

A manager rating someone they argued against hiring. And a merger. Calibration meetings are widely used for this. Whether they reduce the rater's share remains unclear. And I couldn't find evidence. What? Now, what this research can't do. It can't tell you what your manager is like. These are patterns across thousands of raters, and they don't reliably describe the person who wrote your review.

A lower rating can mean your new manager saw something real. The colleague who seems like the boss may simply be good. What it does give you is a question to ask of any rating, including the ones you give. What did each person bring to it? Their temperament. What the rating was for. Who they chose. Who seems like them.

Who they see. And the uncomfortable part. Ordinary raters, making a judgement they experience as objective, produce ratings that they say a great deal about themselves, and the organisation decides whether one person's view is allowed to stand for the truth. In the next episode, a related judgement.

How organisations decide who has potential. I haven't juegos of Christopher or��ic!* or��ic!* or��ic!* The written version of this episode is on my website with every citation and limitation all 7 things raters bring and the questions I still can't answer I put the link in the description below and it's on the screen now If you found this episode useful please like and subscribe to my channel Each episode takes a pattern of behaviour that people get wrong at work and subscribing is how you'll hear about the next one And in the comments tell me if you've ever been rated differently by two managers for the same work What changed?

I'm Nick Kecker Thank you for watching I'll see you next time but in the meantime observe and act differently