3 Tips to Better Multi-Rater Performance Reviews

Colin Newbold, 1st April 2022 in Development, Feedback, Hiring, Human Resources, People Management, Testing

Why traditional boss/subordinate performance reviews need updating...

Background

See that word ‘…Better…’ in the headline?  We wanted to use ‘…Transformational…’ but the  younger ones amongst us thought it too exaggerated!  They said it was too salesy, not believable enough.  While they’re probably right, We wanted to convey some simple steps that would make a noticeable difference in the way employees feel after a multi-rater performance review, compared to how they typically feel after the traditional boss/subordinate review.  And if they felt better about the experience, might that lead to greater enthusiasm and job satisfaction?  And might that, in turn, result in employees feeling more engaged, more committed and making greater discretionary effort?  Might the employer be then rewarded with higher numbers in those things that you’re looking to improve and lower numbers in those things that you’re trying to minimize or even eradicate altogether?

As is often the case with HR programs, the evidence to support the case for multi-rater reviews is thin on the ground.  Too much doing, not enough monitoring.  That written, there is a growing amount of supportive evidence on 360-degree feedback, which multi-rater performance reviews tend to emulate.  Let’s consider the advantages and disadvantages of each:

Boss/Subordinate Performance Reviews

Advantages

Disadvantages

Only two parties have to make an effort

Requires a lot of preparation time to be done properly, but often seen as a chore

Inexpensive, apart from time investment

The boss often doesn’t spend enough time preparing

Traditional process: both parties understand their roles  

Boss may have a poor relationship with the subordinate, which can negatively bias the result

Little requirement and incentive for follow-up action means limited future time investment required 

Alternatively, Boss may favor the subordinate, which can bias the result the other way

 

Action plans (if produced at all) are often not as robust as really required

 

Very little follow-up due to few incentives to follow-up or negative consequences for not doing 

 

Multi-Rater Performance Reviews

Advantages

Disadvantages

Little preparation time required

Several people have to make an effort

Depending on the platform chosen, the report can be integrated with a workbook that guides the reviewee to a robust action plan.  Learning resources can be linked from within the action plan.

Multi-rater benefits and process not as widely understood: requires educating the parties involved

Depending on the platform chosen, follow up can be built-in automatically  

Depending on the platform chosen, follow up may require effort and may be time-consuming

What Is A Multi-Rater Performance Review?

Instead of the traditional boss/subordinate review discussion once a year, the multi-rater performance review approach reaches out to a number of raters - colleagues that work around the participant (the person being reviewed and receiving the report) - and asks them to complete a brief questionnaire on the participant’s capabilities.  The participant also completes a self-questionnaire.

The resultant report compares the participant’s self-perceptions to those of his/her colleagues.  Scores and comments help the participant to get a greater sense of self-awareness and to see how their behavior is impacting their colleagues (the two basic tenets of emotional intelligence).

All this is based on the old maxim “How you see yourself is one thing, but it is how others see you - the people that work around you - that will likely determine your results.”

A useful report will start by identifying strengths and then show areas where the participant could be even better.  The more progressive digital reports now available have a workbook integrated within them that enables the participant to interpret the data and guides them to a robust and time-lined action plan.  Some reports even link to learning suggestions and development resources.

Tip #1: What Questions To Ask?

The best questionnaires in our experience are those that are focused on behaviors likely to lead to success in the employee’s role.  If the questions are tailored to each role and to your organization’s mission, vision and values, then they also serve to reinforce what “good” behavior looks like (this is particularly beneficial if your organization has just launched a new competency or values-based framework).  Not only are those model behaviors being imprinted in the role-holder’s mind, they are also reminding the wider multi-rater network of what good looks like.

Questions should be as short and concise as possible.  Consider the idealistic ‘Rule of 7’:

-        No more than 7 clusters of questions

-        No more than 7 questions inside each cluster

-        No more than 7 words in a question

-        No more than 7 letters in a word

In reality of course, that rule is hard to enforce but you get the gist…

Questions should never ask more than one thing at a time and they should always be positive (what you’re trying to lead your people towards rather than away from).  Here’s an example from one of our custom questionnaires:

Leads With Empathy                      

Shows high emotional intelligence in their leadership style

Checks in regularly with each team member

Has genuine concern for the well-being of staff

Leaves team members to fit work in around their lifestyles

Does what they say they'll do when they say they'll do it

Is responsive to feedback

Notice the sentence in italics?  That serves as a scene-setter, a definition, for each cluster.  If you ever want to run your draft questionnaire past us for comment before committing, simply email it to: cpn@click-360.com.

Tip #2: Record An Awareness Briefing

Out of all the steps we recommend in a model implementation, this one probably counts for the most success.  We liken it to ‘preparing the soil before planting’ or ‘preparing the windows, doors and walls before painting’.  Preparation is everything.  Even if your employees have done this before, chances are it wasn’t done all that well.

You might only need to record one version of the video that gets sent to the participant and all raters.  Or you might decide to record one for the participants and a second, somewhat shorter version for raters (those providing feedback to participants).  You might get away with the same video(s) for every implementation, or you may choose to do a new one each time.  Keep them short and include the following elements:

  1.     Why are we doing this (great if you can include a few key words from your head honcho)?
  2.     Why now?
  3.     What do we expect to get out of it? (start with a focus on the benefits for each participant, then for each rater, and finally for the wider organization)
  4.     What are the timelines? (which is why you might need a fresh one for each implementation, or you could provide those in an accompanying communication while keeping the briefing the same each time)
  5.     What is involved? (worth showing a screenshot of the questionnaire interface and the report…remember to mention the reasons for  the questions)
  6.     How to motivate colleagues to get on board and complete their questionnaires? This is best done by the participant having a short verbal ‘invitation conversation’ with each rater and getting their verbal commitment to engage and complete the questionnaire before assigning them as raters.

This last element is why you may need to record a separate briefing if you want to send to colleagues (raters/feedback providers) as well as the participants.  

If you want help in scripting the benefits (step 3) and the invitation conversation (step 6) and you would like a selection of screenshots to include in your briefing, just email us: cpn@click-360.com.  I’m going to write this again, so you understand it’s important…make sure these are verbal conversations (face-to-face/phone/Zoom or Teams etc)...an email just doesn’t cut it in these circumstances.

Make sure you get the briefing(s) out to people before the automated email invites arrive from the system.  If you are encouraging participants to have those invitation conversations - especially if they are responsible for nominating their raters online - you will want the briefing to land 5-10 days before the go-live date, just to give them time to speak to their colleagues before the program goes live.

Tip #3: Ask For Completed Action Plans

There is no method more likely to gain commitment to, and closure on, an implementation than this one.  You need to see the results of the multi-rater reviews so you can plan an organization-wide approach to development needs – an extremely thorough and accurate training-needs analysis.  You can only do this if participants provide you with a copy of their action plans and, with participants knowing that these will be audited, the quality of those plans is likely to be pretty high – especially if you choose a provider that has a built-in planning component to their report/workbook.

Action plans need to be built using SMART principles:

 

No more than 3 development priorities at a time is optimal (depending on how big each one is), 5 at the very most.  Each objective needs to be written down as a specific observable behavior, in other words…something the participant is going to be doing (or not doing) and/or something they are going to be saying (or not saying).  Here’s an example:  

S: Produce a much more comprehensive monthly report.

  1. It will contain:
  • a summary at the front
  • an index
  • page numbers
  • images to bring the important words to life
  • plus it will be delivered on or ahead of time.
  1. My report will follow the example of xxxxxx (a model report).
  2. This helps my boss get a sense of my department’s activity & results and helps him achieve his goals.
  3. Starting MM/DD/YYYY and ongoing.

If you can, choose a vendor that can incorporate an automated checking-in process some months down the line, so that those plans are kept alive and dynamic. 

Conclusions

The annual formal Boss/Subordinate performance review no longer works.  Lack of accountability, no benefit for completion/penalty for non-completion, no monitoring for quality purposes, and risk of bias creeping in, all suggest there must be a better way.

Multi-rater performance reviews is 360-degree feedback and works to highlight positive behaviors that are helping the individual achieve (existing strengths) while suggesting behaviors that could be even better (opportunities for improvement).  The fact that ideally around 10-12 colleagues are involved in each participant’s report reduces the risk of bias.  The results will show trends, which are easier to spot and likely more accurate.  Good software makes the process easy and quick and the experience user-friendly.  Digital output reports are portable, more dynamic and can be integrated with interpretation guidance and action plans.  Cost need not be high and disruption to business can be minimal.  

Multi-rater performance review software has come a long way since the Excel spreadsheets of years past and at click.360.com we offer a total end-to-end turn-key package which enables you to continue with your already very busy schedule.

Call or write for a free consultation:

Phone: UK +44 (0)20 3988 6666; US +1(855) 220-6604

Email: cpn@click-360.com

Thank you for reading, 

Colin Newbold & Todd Cavaluzzi

Colin Newbold is the UK-based CEO of click-360, pioneers in the field of 360-degree feedback and multi-rater performance reviews.  His colleague Todd Cavaluzzi is a US-based adviser.

We use cookies to personalise your experience and to analyse our traffic. Do you want to allow all cookies or view and change settings?