Why Incumbent Data in Salary Surveys is Misleading

For many years, employers have used salary surveys to provide market references to manage compensation in their organization and to “price jobs” in the market.  Conventional wisdom suggests a high-quality, reliable survey has the following characteristics:

  • The largest possible group of participants
  • The greatest number of specific jobs
  • The highest number of incumbents reported
  • Survey statistics based on incumbent-weighted averages

In short, bigger is better.

As is often the case, we believe that the conventional wisdom is wrong!

Incumbent-based data is not job-based – it’s personal!

Think about it.

  • Salary ranges represent the range of pay an employer is prepared to offer for position with the same level of contribution to their organization (e.g., the same equivalent worth).
  • All incumbents are placed within the same internal, generic range (employers do not usually have separate salary scales for each occupational group or function).
  • Individual salary levels are not determined by any job factor!  Incumbent salaries are based on personal characteristics such as qualifications, skills, experience and performance, and organizational constraints such as internal policies and guidelines, and internal equity.

There are no job-based factors used to determine placement in the range.  It is easy to conclude using actual incumbent salaries instead of salary ranges is — at best – misleading, if not entirely invalid.  But don’t take our word for it.  Let’s put it to a test.

Salary Range Data is Highly Correlated to Incumbent Values

Birches Group did an analysis of actual incumbent data values and the corresponding salary range values for BG-10 level Senior Working Professional roles in Jamaica.  The dataset included positions from seven different job families.  Twenty-nine employers reported data at this level.

To start the analysis, we examined each discrete position reported by each employer.  In the chart below, the positions are color-coded by occupation (job family), and each employer is represented by a vertical array of dots.  The chart shows all observations, not a specific percentile.

You can see there is a wide variety of values, both within each employer and across different ones.

Next, we examined the salary ranges for these employers, and determined how the incumbent salaries fell into each employers’ salary range.  The chart below shows the ranges.

It’s clear that except for two employers with no formal range defined (the two single dots in the chart circled in red), the rest all have salary ranges defined.  Some of the incumbent values are distributed across the ranges, while some are more clustered, but they are all within the range!  You will also notice that the range spans (the “distance” from minimum to maximum) varies quite a bit – some employers use narrow ranges, others wider ranges, depending on their unique circumstances and requirements.

A common benchmark that many clients use is the 50th percentile or median of the market.  While we could debate the definition of “the market” (and we will in another post), it is possible to measure the median of the salary ranges in a market.

For BG-10 in Jamaica, the subject of this analysis, the 50th percentile salary range is J$ 6,273,150 to J$ 9,576,152.  We calculated these numbers by simply separately calculating the median of all the reported minimum values, and all the reported maximum values. The next chart shows this median range added to the incumbent chart.

Now you can see which incumbent data points are within the 50th percentile range in the market.  It’s important to realize that a percentile value in a salary survey should never be a single number; it should always be a salary range.  If you rely only on incumbent data points when using surveys, you are missing out on what is really happening in the market.

Combining the two prior charts yields the next one, which shows the employer salary ranges against the 50th percentile (shaded blue horizontal bars):

You can observe that just 6 employers (about 21% of the sample) have ranges that are totally outside of the 50th percentile range of the market.  Or stated another way – nearly 80% of employers have a salary range that intersects with the market median range.

We also examined the data by occupation, looking at the mean range for each occupation versus the market.

In the above chart, you can see, aside from Logistics and Program, which were matched by fewer than 8 employers and not truly representative, the rest of the occupational data ranges fall well within the overall 50th percentile.  In other words, the occupation or job family doesn’t impact the going rate for a job very much at all, and it would be very easy to just use the overall data, without any occupational designations, as the basis to determine your market position.

Think about it.

You go crazy trying to match multiple benchmark jobs in surveys.  Then you take that data and apply a secret formula (perhaps weighted averages by incumbent count, for example) to arrive at a “going rate” for your midpoint.  But the data you are combining isn’t really that different; our data shows you could just take the range we report and go from there.  Much easier.

A New Vision for Salary Survey Data

Birches Group believes that salary survey data should be job-based, not personal.  Salary ranges represent for an employer the potential range of salary which the organization is willing to pay for a job at a specific grade level in the organization.  In other words, the range represents the value the organization attaches to all jobs at that grade level, which are deemed to have equivalent worth because they have been evaluated to the same grade.  Salary ranges, not incumbent data, represent job-based values which are appropriate for benchmarking salaries.

By comparing salary ranges instead of incumbent data, employers avoid using personal data, which is volatile and introduces a level of false precision which is misleading when specific jobs or occupations are compared to each other.

Introducing Community™ Market

Community Market is the new name for the Birches Group salary and benefits survey.  The survey format has been streamlined and simplified and includes several new features, including an easy and convenient way to assess your market position at a glance.  Another important change is the way job information will be captured.  We will no longer show separate market values for each benchmark job.  Instead, we will show data aggregated for all jobs at the same Birches Group level.  We will still identify which jobs are in which grade, but as we’ve demonstrated, the occupational differences reported in surveys (including ours) are resulting from personal data, not job-based information, and are not appropriate for benchmarking.

Beginning in October 2019, the new format will be the standard report output for all our surveys.  And starting in April 2020, we will no longer collect incumbent-based data since it will no longer appear in the survey reports.

Community™ Market is part of the Birches Group Community™ platform for integrated HR management.  The platform includes modules for job evaluation (Community™ Jobs), skills assessment (Community™ Skills) and performance management (Community™ Performance) in addition to market surveys.  The job levels established using our Community™ Jobs methodology are used to provide job levels in our surveys, and to assess skills and performance against standards that reflect the same job levels.  It’s the first integrated approach to human resources management.

Learn More

Contact us to learn more about the changes in our survey methodology, or to explore how the Community™ platform can be useful for your organization.

I Have A Salary Survey – Now What?

As a data provider and as compensation consultants, clients often ask us for advice and guidance in formulating policies and processes for using market data to inform and manage their compensation program.  The questions range from “What percentile of the data should I use?” to “How should I take inflation into account?” 

The answer is to refer to your remuneration framework (aka compensation policy).  Your framework should address the key issues you need to sort out, along with practical steps to move from market data to salary scale.  Here are some tips.

Top Ten Features of a Well-Designed Remuneration Framework

Here are the top ten things that form a solid basis for a well-designed remuneration framework:

  1. Cost of Labor vs. Cost of Living. Salary setting is essentially an application of supply and demand to the labor market.  While managers and employees like to think that inflation is important, data shows that there is little or no correlation between labor market increases and inflation.  So stick to cost of labor, not cost of living for your policy.
  2. Credible Market Sources. You need market data to assess your competitive position.  Use a professionally-conducted salary survey.  Resist the temptation to rely on internal mechanisms such as the “call around” to your peers to see what they’re paying. Not only is this time consuming and fraught with pitfalls, in many countries, it’s illegal to share salary information.  A neutral third-party survey provider will insulate you from these risks.
  3. Solid Job Classification. Despite reports to the contrary, job classification is alive and well, and good practice requires its use.  Comparing jobs is not like comparing a can of peas.  Each organization defines roles differently, so job comparisons are hard.  This is true when comparing jobs internally in your organization, and externally in your market.  Job classification is a systematic and objective way to determine which jobs are equivalent to which other ones, along with building a hierarchy of your organization.  There are many good ways to apply job classification, but before you think about that, be sure you have a robust classification standard in place.
  4. Know Your Market. Organizations should define their ideal comparator group to include peer organizations within their sector, and across all sectors, with whom you compete for talent.  Limiting your comparators to your own sector is unwise. In most developing countries, sectors are too small for meaningful sector cuts.  And even when they are possible, comparing across different market strata is a fool’s exercise.  Instead, focus on those employers in any sector, including the international public sector, that target the same strata as your organization.  If you are a leader, you should compare to other leaders. And remember, your sector is not an island.  Even if there is good sector data available, as you will find in more developed markets, it is useful to compare sector data to general market practices to understand the actual differences.
  5. Total Compensation Approach. There are many parts of total compensation, and there are wide variations within countries and from country to country.  The only practical way to determine your market position is to use total compensation.  If your total comp is right, then you can “unscramble the egg” into the components your company chooses to offer.  If you look at things component by component, it will be almost impossible to achieve your desired target.
  6. Annual Updates. Labor markets are dynamic.  In developing countries, where we do most of our work, the amount of change we see during the year is significant, so much so, that we update our survey data three times a year.  Even in more stable, slow growth, developed economies, it’s helpful to look at the market at least once per year.  Remember, markets are not static – there could be new competitors emerging in previously non-existent sectors, for example, which cause market disruption.  Or a big company could open a new facility and try to poach all the key talent from the market.  With access to current data, you are in a stronger position to know what’s happening and how best to react.
  7. Use Local Currency. With just a few exceptions around the world, such as distressed economies or countries where civil unrest is ongoing, staff compensation should be determined in local currency, without any hard links to foreign currency benchmarks (e.g., US dollars, Euros, British Pounds, etc.).  If you don’t believe me, we should talk so I can convince you.  And you should have a Special Measures Policy (see number 10 below).
  8. Go GLOCAL – Global Standards but Local Adaptation. Organizations often formulate policies on a global basis without enough consideration of local practices.  The world is a messy place and compensation practices vary according to many factors, including local culture, level of economic development, historical practices, and a host of other reasons. Reserve some flexibility to address the market differences that exist which require wider pay spans and variable increase percentages between grades depending on the market.  Pay curves are much steeper in developing markets than in more mature ones.  If you acknowledge this, then your global standard formulaic approach won’t deliver the results you need.  In these situations, trust your local experts – they usually know what’s common in their country.
  9. Ageing of Data. Survey data is always a snapshot in time, and always retrospective, not prospective.  There are many approaches to age data forward to anticipate some market movement that is not already reflected in the survey.  It’s a good practice to make these adjustments.
  10. Address Special Circumstances. We believe every employer should develop a policy outlining specifically what steps will be taken if an unforeseen or uncontrollable event occurs.  Whether it’s economic (inflation, devaluation) or non-economic (natural disaster, civil unrest, ongoing conflict, etc.), your managers and employees want to know, when something bad happens externally, what their employer will do. It’s a real competitive advantage to have a policy in place for such circumstances, to provide clear, transparent information to all those affected, and enable your organization to act quickly and lead the market in responding to the crisis.  Yet few employers have taken the time to develop a Special Measures Policy to do this.  You probably have a crisis management plan in place for other functions, but I bet it doesn’t address these fundamental issues that become especially important to staff when a crisis occurs.

So there you have it – our top ten features to include in your compensation policy to help manage it in a market-driven, cost-effective and professional manner.  It takes time and discipline to do this consistently, but don’t be afraid to try.  Of course, professional assistance could increase the capacity of your organization to deliver and fill in any technical gaps you may be experiencing.

Birches Group specializes in the study of work – how work works.  Our Community™ Platform includes job evaluation, labor market data, skills assessment and performance management.  Through a combination of consulting, simple to use software and our focus on jobs as the core element for every employer, we assist organizations in optimizing their workforce design and ensuring their competitive goals are achieved.

We conduct multisector market surveys in 155 developing country markets, including all of Africa.  In addition, we offer a specialized survey for international NGOs and those companies organizations involved in international development in approximately 85 countries.

Birches Group can assist clients through our consulting services in the areas of job evaluation, salary scale design, compensation policy development (including special measures), as well as support for skills development and performance management.

For more information, please contact us.