Evidence-based HR: Make better decisions and step up your influence

The basic idea of evidence-based practice is that high-quality decisions and effective practices are based on critically appraised evidence from multiple sources. When we say ‘evidence’, we mean verifiable facts and information (data supporting or contradicting a claim, assumption or hypothesis). This evidence may come from scientific research, the local organisation, experienced professionals or relevant stakeholders. We use the following definition from CEBMa:

“Evidence-based practice is about making decisions through the conscientious, explicit and judicious use of the best available evidence from multiple sources… to increase the likelihood of a favourable outcome.”

The importance of evidence-based practice and the problems it sets out to solve is explained in more detail in our factsheet and thought leadership article but, in essence, it has three main benefits:

1. It ensures that decision-making is based on fact, rather than outdated insights, short-term fads and natural bias (like personal belief and preferences).
2. It creates a stronger body of knowledge and as a result, a more trusted profession.
3. It gives more gravitas to professionals, leads to increased influence on other business leaders and has a more positive impact on the work done by people professionals.

Before making an important decision or introducing a new practice, an evidence-based people professional should start by asking: "What is the available evidence?" As a minimum, people professionals should consider four sources of evidence.

Evidence from people professionals

The expertise and professional judgement of practitioners, such as colleagues, managers, staff members, employees and leaders, is vital for determining whether a people management issue does require attention, if the data from the organisation is reliable, whether research findings are applicable, or whether a proposed solution or practice is likely to work given the organisational context.

Evidence from scientific literature

In past decades, a large number of scientific studies have been published on topics relevant to people professionals, for example on topics such as the characteristics of effective teams, the drivers of knowledge worker performance, the recruitment and selection of personnel, the effect of feedback on employee performance, the early signs/indicators of absenteeism, and the predictors of staff turnover. Empirical studies published in peer-reviewed journals are especially relevant, as they provide the strongest evidence on cause-and-effect relationships, and so what works in practice.

Evidence from the organisation

This can be financial data or performance indicators (for example, number of sales, costs, return on investment, market share), but it can also come from customers (for example, customer satisfaction, brand recognition), or employees (for example, employee performance, job satisfaction, employee engagement scores). It can be ‘hard’ numbers such as staff turnover rates, number of new hires or productivity levels, but it can also include ‘soft’ elements such as perceptions of the organisation’s risk climate or attitudes towards senior management. This source of evidence typically helps leaders identify the existence or scale of a need or problem, possible causes and potential solutions.

Evidence from stakeholders

Stakeholders are people (individuals or groups inside or outside the organisation) whose interests affect or are affected by a decision and its outcomes. For example, internal stakeholders such as employees can be affected by the decision to reduce the number of staff, or external stakeholders such as suppliers may be affected by the decision to apply higher-quality standards. However, stakeholders can also influence the outcome of a decision; for example, employees can go on strike, regulators can block a merger, and the general public can stop buying a company’s products. For this reason, this evidence is often an important guide to what needs or issues an organisation investigates, what improvements it considers and whether any trade-offs or unintended consequences of proposed interventions are acceptable. Stakeholder data can also be useful when assessing if solutions are fit for purpose or if they may need further adjustment to be effective when actually rolled out.

The importance of combining all sources

Finally, none of the four sources are enough on their own – every source of evidence has its limitations and its weaknesses. It may not always be practical to draw on all four sources of evidence or to cover each of them thoroughly, but the more we can do, the better decisions will be. Balancing both conflicting and correlating data sources is key to making an informed decision.

Since the 1990s, evidence-based practice has become an established standard in many professions. The principles and practices were first developed in the field of medicine and following this have been applied in a range of professions. Despite differing contexts, the approach broadly remains the same.

Below, each step is discussed and illustrated with examples. It is important to note that following all six steps will not always be feasible. However, the more that professionals can do – especially when making major or strategic decisions – the better.

Asking questions to clarify the problem and potential solution and to check whether there is evidence in support of that problem and solution is an essential first step. Without these questions the search for evidence will be haphazard, the appraisal of the evidence arbitrary, and its applicability uncertain.

Asking critical questions should be constructive and informative. It is not about tearing apart or dismissing other people's ideas and suggestions. By the same token, evidence-based practice is not an exercise in myth busting, but rather seeking to establish whether claims are likely to be true and potential solutions are likely to be effective.

This example shows the type of critical questions you can ask.

Example: Autonomous teams – an example of asking critical questions

Based on the answers to the questions in Step 1, we should have a good understanding of whether there is sufficient evidence from multiple sources to support the assumed problem and preferred solution. In most cases, however, the available evidence is too limited or important sources are missing. In that case, we proceed with the second step of evidence-based practice: acquiring evidence.

Evidence can be acquired from various sources including:

• Practitioners
• Scientific literature
• Organisation
• Stakeholders

Acquiring evidence from practitioners

This could be through:

• face-to-face conversations: this is the easiest way. While it can be prone to bias, sometimes simply asking people about their experience can give good insight.
• more structured interactive group meetings, workshops or other ways of collecting views, such as surveys.

Practitioner expertise is a useful starting point in evidence-based practice to understand the assumed problem and preferred solutions. It is also helpful in interpreting other sources of evidence – for example, in assessing whether insights from scientific literature are relevant to the current context.

Acquiring evidence from scientific literature

This could be through:

• Peer-reviewed academic journals, which can be found in research databases. These are usually behind a paywall or are accessible only through a university, but CIPD members have access to one such database in EBSCO's Discovery Service.
• ‘Evidence reviews’ such as systematic reviews (see below) and shorter rapid evidence assessments (REAs). These aim to identify and summarise the most relevant studies on a specific topic. They also do the work of identifying the best research, and selecting and critically appraising  studies on the basis of explicit criteria. The CIPD produces evidence reviews on a range of topics – you can access these via our Evidence review hub.

Acquiring evidence from the organisation

This could be through:

• Internal management information: Often the finance department and the HR/personnel department are the key custodians of people data and analytics.
• Internal research and evaluation: This could be conducted via trials of interventions, bespoke surveys or focus groups.
• External sources such as census bureaus, industry bodies, professional associations and regulators. However, sometimes relevant organisational data is not available, either because collecting it is too time-consuming and costly, because of data sensitivities and a lack of disclosure or simply due to a lack of analytical capability in processing and interpreting data.

Acquiring evidence from stakeholders

Organisational decisions often have lots of stakeholders both inside and outside the organisation. A stakeholder map is therefore a useful tool to identify which stakeholders are the most relevant. You can access a flowchart to assist with stakeholder analysis through our change management tools.

A stakeholder’s relevance is determined by two variables:

1. The extent to which the stakeholder’s interests are affected by the decision (harms and benefits).
2. The extent to which the stakeholder can affect the decision (power to influence).

When the most important stakeholders are identified, often qualitative methods such as focus groups and in-depth interviews are used to discuss their concerns.

Unfortunately, evidence is never perfect and can be misleading in many different ways. Sometimes the evidence is so weak that it is hardly convincing at all, while at other times the evidence is so strong that no one doubts its correctness. After we have acquired the evidence we need to critically appraise what evidence is ‘best’ – that is, the most trustworthy.

Appraising evidence from practitioners

When appraising the evidence from practitioners, the first step is to determine whether their insights and opinions are based on relevant experience or personal opinions. Next, we need to determine how valid and reliable that experience is. We can assess this by considering:

• whether the experience concerns repeated experience (consistency)
• whether the situation allowed for direct, objective feedback (objectivity)
• whether the experience was gained within a regular, predictable work environment (continuity). 

This example uses sales agents and business consultants to show how you can use these criteria to determine whether insight and opinion are based on relevant example.

The expertise of a sales agent is more likely to be trustworthy than the expertise of a business consultant specialised in mergers. In general, sales agents work within a relatively steady and predictable work environment, they give their sales pitch several times a week, and they receive frequent, direct and objective feedback. Consultants, however, are involved in a merger only a few times a year (often less), so there are not many opportunities to learn from experience. In addition, the outcome of a merger is often hard to determine – what is regarded as a success by one person may be seen as a failure by another. Finally, consultants accompanying a merger do not typically operate in a regular and predictable environment: contextual factors such as organisational differences, power struggles and economic developments often affect the outcome.

Professional expertise that is not based on valid and reliable experience is especially prone to bias, but any evidence from practitioners is likely to reflect personal views and be open to bias (see Building an evidence-based people profession). For this reason, we should always ask ourselves how a seemingly experienced professional’s judgement could be biased and always look at it alongside other evidence.

Appraising evidence from scientific literature

Appraising scientific evidence requires a certain amount of research understanding. To critically appraise findings from scientific research, we need to understand a study’s ‘design’ (the methods and procedures used to collect and analyse data). Examples of common study designs are cross-sectional studies (surveys), experiments (such as randomised controlled trials, otherwise known as ‘RCTs’), qualitative case studies, and meta-analyses. The first step is to determine whether a study’s design is the best way to answer the research question. This is referred to as ‘methodological appropriateness’.

Different types of research questions occur in the domain of people management. Very often we are concerned with ‘cause-and-effect’ or ‘impact’ questions, for example:

• What works in managing effective virtual teams?
• Does digital work affect mental wellbeing?
• How can managers help employees be more resilient?
• What makes goal setting and feedback more effective in improving performance?

The most appropriate study designs to answer cause-and-effect questions are RCTs and controlled before-after studies, along with systematic reviews and meta-analyses that gather together these types of study.

Other research questions that are relevant to people professionals are questions about:

• prevalence (“How common is burnout among nurses in hospitals?”)
• attitudes (“How do employees feel about working in autonomous teams?”)
• prediction (“What are drivers/predictors of absenteeism?”)
• differences (“Is there a difference in task performance between virtual teams and traditional teams?”).

Each of these questions would ideally have a specific study design to guarantee a valid and reliable (non-biased) answer.

An overview of common study designs and what they involve can be found in the Appendix. Following this (also in the Appendix) is an overview of types of questions and the appropriateness of each design. This gives a useful guide for both designing and appraising studies. For example, if you want information on how prevalent a problem is, a survey will work best; a qualitative study will give the greatest insight into people’s experiences of, or feelings about, the problem; and an RCT or before-after study will give the best information on whether a solution to the problem has the desired impact.

Of course, which design a study uses to answer a research question is not the only important aspect to consider. The quality of the study design – how well it was conducted – is equally important. For example, key considerations in quantitative studies include how participants are selected and whether measures are reliable and valid. For a discussion of reliability and validity in performance measures, see People performance: An evidence review. For systematic reviews and meta-analyses, a key question is how included studies are selected.

Finally, if an impact study seems to be trustworthy, we want to understand what the impact is of the intervention or factor being studied. Statistical measures of ‘effect sizes’ give us this information, both for an intervention or factors of influence itself, and how it compares to others. Being able to compare effect sizes is very important for practice. For example, the critical question is not simply, “Does a new management practice have a small or large effect on performance?” but rather, “Is this the best approach or are other practices more impactful?” For more information on effect sizes, see Effects sizes and interpreting research findings in the Appendix.

Appraising evidence from the organisation

When critically appraising evidence from the organisation, the first thing to determine is whether the data are accurate. Nowadays, many organisations have advanced management information systems that present metrics and KPIs in the form of graphs, charts and appealing visualisations, giving the data a sense of objectivity. However, the data in such systems are often collected by people, which is in fact a social and political endeavour. An important appraisal question therefore is: “Were the data collected, processed and reported in a reliable way?”

In addition to the accuracy of data, several other factors can affect its trustworthiness, such as measurement error, missing contextual information and the absence of a logic model. Some organisations use advanced data-analytic techniques that involve big data, artificial intelligence or machine learning. Big data and AI technology often raise serious social, ethical and political concerns as these techniques are based on complex mathematical algorithms that can have hidden biases and, as a result, may introduce biases into the decision-making process.

Appraising evidence from stakeholders

Unlike the scientific literature and organisational data, which serve to give objectifiable and trustworthy insights, stakeholder evidence concerns subjective feelings and perceptions that can’t be considered as facts. Nonetheless, we can make sure that stakeholder evidence comes from a representative sample, so that it is an accurate reflection of all relevant stakeholders.

Evidence-based practitioners should present stakeholders with a clear view of the other sources of evidence. That is, they should summarise what the body of published research and organisational data tell us, as viewed with the benefit of professional knowledge. This can serve as the basis for a well-informed and meaningful two-way exchange. For example, the scientific literature may point to a certain solution being most effective, but stakeholders may advise on other important aspects that should be weighed up against this evidence – for example, whether the intervention is difficult to implement, ethically questionable or too expensive.

Use the best available evidence

The purpose of critical appraisal is to determine which evidence is the best available – that is, the most trustworthy. Sometimes, the quality of the evidence available is less than ideal; for example, there may not be any randomised controlled trials on your intervention of interest. But this does not leave us empty-handed. We can look at other studies that are less trustworthy but still go some way to showing cause-and-effect. Indeed, it’s possible that the best available evidence on an important question is the professional experience of a single colleague. However, even this limited evidence can still lead to a better decision than not using it, as long as we are aware of and open about its limitations. A useful maxim is that “the perfect is the enemy of the good”: if you don’t have the ideal evidence, you can still be evidence-based in how you make decisions. Data will never be perfect and it will differ day to day, moment to moment because it changes over time. This means perfection is unlikely, but predictability based on the most accurate data available is the best course of action. There will always be assumptions with data but as long as this is evidenced, validated and disclosed, the data will be deemed more credible.

After we have acquired and critically appraised the different types of evidence, how should we bring them all together? The broad process of knitting together evidence from multiple sources is more craft than science. It should be based on the question you wish to answer and the resources available.

One approach can be summarised as follows:

1. Analysis of people data (eg employee survey results and key performance indicators)
2. Hold workshop with practitioner experts and stakeholders to assess the problem and agree priority issues
3. Refer to evidence review for existing evidence
4. Try out new approach in a pilot
5. Hold workshop to discuss findings
6. Draft recommendations
7. Hold workshop to test recommendations.

In most cases, the answer as to whether to implement a new practice is not a simple yes or no. Questions to consider include the following:

• Does the evidence apply to our organisational context?
• Is the intervention in question the most effective or are others more effective?
• Are the anticipated benefits likely to outweigh any risks or costs resulting from the proposed solution?
• Are there ethical issues to consider; for example, if the benefits aren't evenly distributed among stakeholders?
• Do the costs, necessary resources and timescale fit the organisation’s needs?

The final part of this step concerns how, and in what form, the evidence should be applied. This includes the following possible approaches:

• The 'push' approach: Actively distributing the evidence to the organisation’s relevant stakeholders, often in the form of a protocol, guideline, checklist or standard operating procedure. The push approach is typically used for operational, routine practices (for example, hiring and selection procedures, or how to deal with customer complaints).
• The 'pull' approach: Evidence from multiple sources is actively obtained and succinctly summarised. When it concerns summarising findings from the scientific literature, rapid evidence assessments (REAs) are often used – see Step 2 and the CIPD Evidence review hub. The pull approach is typically used for non-routine decisions that involve making changes to the way an organisation operates (for example, the implementation of an autonomous team, or the introduction of performance feedback).
• The 'learn by doing' approach: Pilot testing and systematically assessing outcomes of the decisions we take in order to identify what works (see also Step 6). This is especially appropriate if there is no other option, for novel or hyper-complex decisions when evidence is often not (yet) available (for example, starting a new business in an emerging market).

The final step of evidence-based practice is assessing the outcome of the decision taken: did the decision (or the implementation of the new practice) deliver the desired results?

Unfortunately, organisations seldom evaluate the outcome of decisions, projects or new practices. Nevertheless, assessing the outcome of our decisions is something we can and always should do. Before we assess the outcome, however, we first need to determine:

• whether the decision or new practice was executed/implemented
• whether it was executed/implemented as planned.

After all, if we don’t know with certainty if a practice was implemented as planned, we don’t know whether a lack of impact is due to poor implementation or the practice itself.

When we assess the outcome of a decision, we are asking whether the decision had an effect on a particular outcome. As discussed in Step 3, for a reliable answer to a cause-and-effect question, we need a control group (preferably randomised), a baseline measurement, and a post-measurement – also referred to as a randomised controlled trial. However, often there is no control group available, which leaves us no other option than to assess the impact of the decision or practice by comparing the baseline with the outcome. This type of assessment is referred to as a before-after measurement.

When we do not (or cannot) obtain a baseline, it is harder to reliably assess the outcome of a new practice or intervention. This is often the case in large-scale interventions or change projects that have multiple objectives. But even in those cases, assessing the outcome retrospectively is still beneficial. For example, a meta-analysis found that retrospective evaluations can increase performance by 25%.

Evidence-based practice is about bringing together complementary evidence from different sources, including research and practice.

A good aim for practitioners themselves is to become a ‘curious consumer’ of research, understanding enough that one can ask probing questions; for example, about the strength of evidence and the size of impacts. This is underpinned by skills in critical thinking, in particular being clear about what questions are really of interest and what evidence will do the best job of answering those questions.

To develop your knowledge and capability, visit the CIPD’s online course on evidence-based practice. 

While it is fair to say that evidence-based practice and HR is still in its infancy compared to some other professions, people professionals can begin with these practical steps to pump prime their decision-making:

• Read research.
• Collect and analyse organisational data.
• Review published evidence.
• Trial new practices.
• Share knowledge.
• Above all, think critically.

People professionals should always consider using the four sources of evidence (professional expertise, scientific literature, organisational data, and stakeholder views and concerns). It can be tempting to rely on professional judgement, prior experience and ‘best practice’ examples, and bow to senior stakeholder views, but injecting evidence from the other sources will greatly reduce the chance of bias and maximise your chances of crafting effective solutions.

Published research is a valuable source of evidence - this often neglected as a source, but when drawing on academic papers, the two principles of critical appraisal (‘not all evidence is equal’) and looking at the broad body of research on a topic (‘one study is not enough’) will then put people professionals in a better position to provide effective advice, guidance and solutions through the systematic appraisal of relevant data from multiple , varied and verifiable sources. This ensures risks such as bias, narrow focus and convenience are 

One final note: Be mindful when applying AI to appraising data - modern AI tools are being used more and more to help aggregate, appraise and extract answers from data. Whilst AI can be very useful, like all tools it is how it is applied and assessed that matters. People professionals should not simply rely on an AI response, the response should be assessed and the source data checked for relevance and accuracy. Like humans, AI is not perfect, but by creating the right prompts, sense checking sources and verifying the data, people professionals are more likely to achieve better data credibility and integrity by doing so.

^1,2 Shadish, W. R. Cook, T. D., & Campbell, D. T. (2002). Experimental and quasi-experimental designs for generalized causal inference. Belmont, CA, Wadsworth Cengage Learning. 

³ Petticrew, M., & Roberts, H. (2006). How to appraise the studies: an introduction to assessing study quality. In: Systematic reviews in the social sciences: A practical guide, pp125-63. Oxford: Blackwell.

⁴ Sawilowsky, S. S. (2009) New Effect Size Rules of Thumb. Journal of Modern Applied Statistical Methods. Vol 8(2), pp 597–599.

This guide is based on a 2023 edition written by Jonny Gifford, Jake Young and Eric Barends.

This appendix explains some important technical aspects of appraising scientific research, which is inevitably the trickiest aspect of evidence-based practice for non-researchers. As we note in this guide, most people professionals won’t need to become researchers themselves, but a sensible aim is to become ‘curious consumers’ of research.

To support this, below we explain four aspects of appraising scientific research:

1. The three conditions that show causal relationships.
2. Common study designs.
3. Assessing methodological appropriateness.
4. Interpreting research findings (in particular effect sizes).

We hope that this assists you in developing enough understanding to be able to ask probing questions and apply research insights.

Three conditions to show causal relationships

In HR, people management and related fields, we are often concerned with questions about ‘what works’ or what’s effective in practice. To answer these questions, we need to get as close as possible to establishing cause-and-effect relationships.

Many will have heard the phrase ‘correlation is not causality’ or ‘correlation does not imply causation’. It means that a statistical association between two measures or observed events is not enough to show that one characteristic or action leads to (or affects, or increases the chances of) a particular outcome. One reason is that statistical relationships can be spurious, meaning two things appear to be directly related, but are not.

For example, there is a statistically solid correlation between the amount of ice-cream consumed and the number of people who drown on a given day. But it does not follow that eating ice-cream makes you more likely to drown. The better explanation is that you’re more likely to both eat ice-cream and go swimming (raising your chances of drowning) on sunny days.

So what evidence is enough to show causality? Three key criteria are needed:¹

• Association: A statistical relationship (such as a correlation) between reliable measures of an intervention or characteristic and an important outcome.
• Temporality or prediction: That one of these comes before the other, rather than the other way round. We obtain this from before-and-after measures to show changes over time.
• Other factors (apart from the intervention or influencer of interest) don’t explain the relationship: We obtain this from various things: studying a control group alongside the treatment group to see what would have happened without the intervention (the counterfactual); randomizing the allocation of people to intervention and control to avoid selection bias, and controlling for other relevant factors in the statistical analysis (for example, age, gender or occupation).

Common study designs

Different study designs do better or worse jobs at explaining causal relationships.

Single studies

• Randomised controlled trials (RCTs): Conducted well, these are the ideal method that meets all three criteria for causality. They are often referred to as the ‘gold standard’ of impact studies.
• Quasi-experimental designs: These are a broad group of studies that go some way towards meeting the criteria. While weaker than RCTs, they are often much more practical or ethical to conduct, and can provide good evidence for cause and effect. One example is single-group before-and-after studies. Because these don’t include control groups, we don’t know whether any improvement observed would have happened anyway, but by virtue of being longitudinal they at least show that one thing happens following another.
• Parallel cohort studies: These compare changes in outcomes over time for two groups who are similar in many ways but treated differently in a way that is of interest. Because people are not randomly allocated to the two groups, there is a risk of ‘confounders’ – that is, factors that explain both the treatment and outcomes, and interfere with the analysis. But these studies are still useful as they show change over time for intervention and control groups.

These research designs go much further to show cause-and-effect or prediction than cross-sectional surveys, which only observe variables at one point in time. In survey analysis, statistical relationships could be spurious or the direction of causality could even be the opposite to what you might suppose. For example, a simple correlation between ‘employee engagement’ and performance could exist because engagement contributes to performance, or because being rated as high-performing makes people feel better.  

Other single study designs include controlled before-after studies (also called 'non-randomized controlled trials' or 'controlled longitudinal studies'), controlled studies with post-test only, and case studies. Case studies often use qualitative methods, such as interviews, focus groups, documentary analysis, narrative analysis, and ethnography or participant observation. Qualitative research is often exploratory, in that it is used to gain an understanding of underlying reasons or opinions and generate new theories. These can then be tested as hypotheses in appropriate quantitative studies.

Systematic reviews and meta-analyses

Systematic reviews and meta-analyses are central to evidence-based practice. Their strength is that they look across the body of research, allowing us to understand the best available evidence on a topic overall. In contrast, even well-conducted single studies can give different results on the same topic, due to differences in context or the research approaches used.

Characteristics of these are as follows:

• Systematic reviews: These are studies that summarise the body of studies on the same topic. They use consistent search terms in different scientific databases, ideally appraise the quality of studies and are explicit about the methods used. The CIPD conducts evidence reviews based on rapid evidence assessments (REAs), a shortened form of the systematic review that follows the same principles.
• Meta-analysis: This is often based on a systematic review. It is a study that uses statistical analysis to combine the results of individual studies to get a more accurate estimate of an effect. It can also be used to analyse what conditions make an effect larger or smaller.

More information on research designs can be found in CEBMa resources.

Assessing methodological appropriateness

When conducting an evidence review, we need to determine which research evidence is ‘best’ (that is, most trustworthy) for the question in hand, so we can prioritise it in our recommendations. At the same time, we assess the quality of research evidence to establish how certain we can be of our recommendations: well-established topics often have a strong body of research, but the evidence on new or emerging topics is often far less than ideal.

This involves appraising the study designs or research methods used. For questions about intervention effectiveness or cause-and-effect, we use tables such as that below to inform a rating of evidence quality. Based on established scientific standards, we can also estimate the trustworthiness of the study. Hypothetically, if you were deciding whether to use a particular intervention based on evidence that was only 50% trustworthy, you would have the same 50/50 chance of success as tossing a coin, so the evidence would be useless. On the other hand, using evidence that was 100% trustworthy would give you certain success. Of course, in reality nothing is 100% certain, but highly trustworthy research can conclusively demonstrate that, in a given context, an intervention has a positive or negative impact on the outcomes that were measured.

Table 1: Methodological appropriateness of effect studies and impact evaluations

Notes: Trustworthiness takes into consideration not only which study design was used but also how well it was applied. Table reproduced from CEBMa (2017), based on the classification system of Shadish, Cook and Campbell (2002)² and Petticrew and Roberts (2006)³.  

There are two important points to note about using such hierarchies of evidence. First, as we discuss in this guide, evidence-based practice involves prioritising the best available evidence. A good mantra here is ‘the perfect is the enemy of the good’: if studies with very robust (highly methodologically appropriate) designs are not available on your topic of interest, look at others. For example, if systematic reviews or randomized controlled studies are not available on your question, you will do well to look at other types of studies, such as those with quasi-experimental designs.

Second, although many questions for managers and people and HR relate to effectiveness or causality, this is by no means always the case. Broadly, types of research questions include the following:

Table 2: Types of research question

Different methods are suited to different types of questions. For example, a cross-sectional survey is a highly appropriate or trustworthy design for questions about association, difference, prevalence, frequency and attitudes. And qualitative research is highly appropriate for questions about experience, perceptions, feelings, needs and exploration and theory building. For more discussion of this, see Petticrew and Roberts (2003).

Effect sizes and interpreting research findings

Even if practitioners wanting to be evidence-based can search for and find relevant research, they are left with another challenge: how to interpret it. Unfortunately, academic research in human resource management is often highly technical, written in inaccessible language and not closely linked to practice. A recent analysis found that in a sample of 324 peer-reviewed articles, half of them dedicated less than 2% of the text to practical implications, and where implications were discussed, this was often obscure and implicit.  

Even if published research does include good discussion of practical implications, it’s helpful and perhaps necessary for practitioners wishing to draw on them to understand the findings. This can be tricky, as they contain fairly technical statistical information.

Statistical significance

There’s an obvious need to simplify the technical findings of quantitative studies. The typical way to try to simplify research findings is to focus on statistical significance, or p-values. Reading through a research paper, this may seem intuitive, as the level of significance is identified with asterisks: typically, * means sufficiently significant and ** or *** means highly significant. However, there is a lot of confusion about what the p-value is – even quantitative scientists struggle to translate it into something meaningful and easy to understand – and a growing number of scientists are arguing that it should be abandoned. What’s more, statistical significance does nothing to help a practitioner who wants to know if a technique or approach is likely to have a meaningful impact – that is, it does not answer the most important practical question of how much difference an intervention makes.

Effect sizes

The good news is that effect sizes do give this information. The information is still technical and can still be hard to understand, as studies often use different statistics for effect sizes. Fortunately, however, we can translate effect sizes into every-day language. A useful tool is 'Cohen’s Rule of Thumb', which matches different statistical measures to small/medium/large categories.  For a table showing different measures of effect sizes according to Cohen’s Rule of Thumb, see CEBMa Guideline for Rapid Evidence Assessments in Management and Organizations, p20.  

According to Cohen:

• a ‘small’ effect is one that is visible only through careful examination – so may not be practically relevant
• a ‘medium’ effect is one that is ‘visible to the naked eye of the careful observer’
• a ‘large’ effect is one that anybody can easily see because it is substantial. An example of a large effect size is the relationship between sex and height: if you walked into a large room full of people in which all the men were on one side and all the women on the other side, you would instantly see a general difference in height.

The rule of thumb has since been extended to account for very small, very large and huge results.⁴

Effect sizes need to be contextualised. For example, a small effect is of huge importance if the outcome is the number of fatalities, or indeed, sales revenue. Compared to this, if the outcome is work motivation (which is likely to affect sales revenue but is certainly not the same thing) even a large effect will be less important. This shows the limits of scientific studies and brings us back to evidence from practitioners and stakeholders, who are well placed to say what outcomes are most important.