A competency-based performance appraisal for nurse leaders

Introduction: An employee performance appraisal is used internationally to improve employee performance, productivity, efficiency and effectiveness in nursing care. The purpose of the study was to assess the performance and efficiency of nurse leaders at all levels of leadership, identify deficient areas for each individual leader and propose improvements. Methods: Two surveys were conducted in 15 Slovenian public hospitals. The sample from the first survey included 1311 nurses who assessed the level of competencies of their immediate superiors. The sample from the second survey included 236 nurse leaders. Data envelopment analysis is used in this study to develop a model of practice outputs and inputs to help identify the most efficient nurse leaders. Results: Significant differences exist in the performance appraisal among the leaders in nursing. A total of 203 (86 %) nurse leaders are defined as inefficient and could improve their leadership performance. On average, inefficient leaders could improve their behaviour most in the areas of generic leadership competencies (by 51.7 %) and interprofessional relationships (by 47.7 %). Discussion and conclusion: The areas with the worst performance are generic leadership competencies and interprofessional relationships where also the most significant improvements are possible. Nurse leaders and hospital managers can use the analysis results to determine in which areas the leaders in their organisation should improve the levels of their competencies. IZVLEČEK Uvod: Metode merjenja učinkovitosti zaposlenih se pogosto uporabljajo za izboljšanja učinkovitosti zaposlenih v zdravstvenih organizacijah. Namen tega članka je oceniti uspešnost in učinkovitost vodilnih medicinskih sester na vseh ravneh vodenja, ugotoviti slabše ocenjena področja in predlagati izboljšave. Metode: Dve anketi sta bili izvedeni v 15 slovenskih javnih bolnišnicah. V vzorec prve ankete je bilo vključenih 1.311 medicinskih sester, ki so ocenjevale ravni kompetenc svoje neposredno nadrejene vodje. Vzorec iz druge ankete je obsegal 236 vodilnih medicinskih sester. V raziskavi smo z uporabo metode analize ovojnice podatkov (data envelopment analysis) razvili model ocenjevanja učinkovitosti vodij v katerega smo skušali vključiti čim več vložkov in izložkov v procesu vodenja. Rezultati: Med ocenami učinkovitosti vodij v zdravstveni negi obstajajo pomembne razlike. 203 (86 %) vodij v zdravstveni negi je neučinkovitih in ima priložnost, da izboljša svoje vodenje. Neučinkoviti vodje lahko v povprečju najbolj izboljšajo svoja ravnanja in vedenje na področjih generičnih vodstvenih kompetenc (za 51,7 %) in medpoklicnih kompetenc (za 47,7 %). Diskusija in zaključek: Najslabše sta ocenjeni področji generičnih in medpoklicnih kompetenc, kjer so možne tudi največje izboljšave.Vodilne medicinske sestre in vodstva bolnišnic lahko na osnovi rezultatov učinkovitosti vodenja opredelijo, na katerih področjih naj vodje v njihovi organizaciji izboljšajo raven svojih kompetenc.


Introduction
In most countries nurses comprise the largest professional group within the health care system (World Health Organization, 2015).The professional group of nurses has its own system of leadership in most larger, secondary and tertiary level hospitals.Only efficient leaders are capable of guiding such large groups of employees towards a common goal -to improve quality and safety in healthcare and consequently ensure better health care for the population.Increasing attention is therefore being paid to leadership in nursing (Blaney, 2012;International Council of Nurses, 2015).
Leadership performance is multidimensional (Tonidandel, et al., 2012;Hassan, et al., 2013) and it is empirically difficult to define all of its dimensions.The behaviour of an individual leader is influenced by various factors, which are sometimes hard to measure.Every action by a leader significantly influences their subordinates and, consequently, the entire organisation.It is therefore important to investigate the relationship between the traits and characteristics that are linked to a specific leader and the leader's actions, behaviours or activities.A large number of performance measures must be incorporated into a unified system that enables the consistent classification of leaders (Osman, et al., 2011;Weber, et al., 2015).Such a framework needs to combine the social interactions among the people involved with the structural forces in the environment (Thurston & McNall, 2010).
A variety of methods is employed for measuring leaders' performance (Kotzé & Venter, 2011).One of the most frequently used approaches is a performance appraisal.A performance appraisal is intended to engage, align and coalesce individual and group effort to continually improve overall accomplishment of the organisational mission (Manoharan, et al., 2009).The performance appraisal and the performance appraisal outcomes themselves can have an important influence on employees' reactions to their work, their supervisors, and their organisation as a whole (Thurston & McNall, 2010;Schay & Fisher, 2013).Performance appraisals have long been seen as an indispensable practical tool for raising the quality of health care and advancing professional nursing standards.In the literature on the health care sector, a variety of methods have been reported in the performance appraisal of nurses (Hamilton, et al., 2007).
Competencies and competency models are often used as a method for appraising employee performance.In the competency approach, we compare employees' performance against performance standards or desired behaviours (Whelan, 2006;Rosenfeld, et al., 2012).The development of competency-based performance appraisal systems is especially important for leaders.The complex nature of the relationship between the multiple inputs and multiple outputs involved must be taken into account when developing and applying a performance appraisal.Data envelopment analysis (DEA) is a model that allows multiple inputs and multiple outputs to be used.We can find papers that have addressed this issue, including in the health sector.DEA is used to compare: hospitals (Kirigia & Asbu, 2013), critical access hospitals (Wilson, et al., 2012), trauma facilities (Helton & Langabeer, 2012) or employees (Seljak & Kvas, 2015).

Aim and objectives
This article aims to assess the performance and efficiency of nurse leaders at all levels of leadership, identify deficient areas for each individual leader, propose improvements and set target values.The purpose of this study is to integrate as many different aspects (inputs) and results (outputs) of nurse leaders' work as possible into a comprehensive performance appraisal system.On the basis of this system, we can identify those leaders with the highest and lowest efficiency scores.
Hypothesis 1: Efficiency scores among nurse leaders in Slovenian public hospitals differ.
The measurement of different resources used (traits and characteristics the leaders possess and the characteristics of their job position) with which the most efficient nurse leaders achieve high levels of performance appraisal provides us with useful information to help improve the performance of less efficient nurse leaders.The DEA method not only provides an efficiency score, but also information about competency areas requiring improvements.A performance appraisal system designed in this way would be useful for the preparation of person-specific and organisation-specific training programme proposals.
Hypothesis 2: The competency areas requiring improvements among nurse leaders in Slovenian hospitals differ.

Methods
DEA is one of the most successful methods of operational research today, with a wide range of applications and an extensive bibliography available (Cooper, et al., 2011).DEA is a ''data-oriented' approach for evaluating the performance of a set of peer entities called decision-making units, which convert multiple inputs into multiple outputs (Cooper, et al., 2011).These units may be production units, branches of a bank, police departments, states, hospitals, employees, leaders etc. DEA defines the relative efficiency of each decision-making unit.The basic DEA models assumed that all inputs were discretionary and decision-making units could control them in order to produce the maximum output feasible.However, in practice, the results of production processes also depend on the effect of another type of variables which are beyond the decision-making units' control (Cordero, et al., 2009).These non-discretionary or external inputs affect the transformation of discretionary inputs into outputs (Ruggiero, 2005).
This paper demonstrates how three-stage DEA with non-discretionary inputs can be applied as a fair evaluating and sorting tool.For each leader we obtain a DEA efficiency score.Unlike traditional performance appraisals, DEA searches for the efficient employees who will serve as role models (Sowlati & Paradi, 2004).One of the major advantages of DEA is its ability to show the improvement amounts required to achieve efficiency.DEA can be used to identify employees who are relatively inefficient, measure the magnitude of their inefficiency, and aid in the selection of alternative paths to eliminate inefficiencies (Manoharan, et al., 2009).

Description of the research instrument
This study was part of a larger research project entitled ''Leaders in Health Care'' conducted between 1 July 2010 and 30 April 2011.Two surveys were conducted in the 15 largest Slovenian public hospitals: two university medical centres, six general and seven special hospitals.
The questionnaire in the first survey contained a total of 95 items organised into four groups or competencies: -Generic competencies of leaders in the public sector.Specific principles of work in the public sector characterise all of its subsystems and therefore the behaviours of the leaders managing these subsystems are also similar.Behaviours that are common to all leaders in the public sector are represented through generic competencies (Winterton, et al., 2006).-Ethical competencies are an important component of most leadership competency models and play a particularly significant role in nursing (International Council of Nurses, 2012).-Interprofessional relationships.One of the key competencies every health care professional should possess is the ability to efficiently participate in health care teams.Interprofessional collaboration is now commonly accepted as a priority and widely pursued in many health systems (Ovijač, 2012).-A positive attitude towards knowledge and education.The basis for the nursing profession and nurses' knowledge is a good education system that must be supplemented with continuing education following graduation (Collins, 2013).
The full list of competencies for nurse leaders are defined in the competency model for leadership in nursing that emerged in Slovenia between 2009 and 2011 (Kvas, et al., 2013).A pilot study was not performed as the questionnaire has already been tested in the population of nurses and nurse leaders in Slovenia (Kvas, et al., 2013).Respondents indicated to what extent each of 95 different behaviours or actions was characteristic of their immediate superiors.The reliability of the measuring instrument was assessed for each of the four groups of competencies.Cronbach's alpha ranged between 0.77 and 0.97.The value indicated a high level of reliability of the measuring instrument.
Factor analysis was applied to determine the construct validity for each of the four groups of competencies.The Kaiser-Meyer-Olkin (KMO) measure of sampling adequacy was between 0.65 and 0.98 and indicated that the factor analysis was appropriate.Bartlett's test of sampling adequacy was significant (p-value less than 0.005) (Munro, 2005).The Principal Component Analysis (PCA) method was applied to the extraction of components.A varimax rotation was applied in order to optimise the loading factor of each item on the extracted components (Blaikie, 2003).All factor loadings were greater than 0.40 (between 0.48 and 0.91).This indicates good construct validity (Rattray & Jones, 2007).
In the questionnaire in the second survey, nurse leaders were asked to provide information regarding their education and job experience, and assess their knowledge, job satisfaction, organisational commitment and levels of interpersonal and interprofessional relationships.Leadership performance is influenced by factors directly linked to a specific leader (defined as discretionary inputs) and other factors (that we attempted to define as non-discretionary inputs).Discretionary inputs must reflect the resources used (Drake & Simper, 2002).All discretionary inputs are (or could be) influenced by individual leaders and/or the management of the hospital where they are employed.We defined six indicators that indicate which traits and characteristics the leaders possess and which they presumably utilised in their efficient leadership: -The three inputs in the area of knowledge/education were: (1) level of formal education; (2) skills and experience (total years of employment); and (3) self-assessment of knowledge (self-assessment of eight areas important for work in health care).
A formal education and job experience should presumably have a positive effect on leadership success.Self-assessment of knowledge is another contributing factor for it reflects an understanding of the various aspects of leadership as well as one's level of confidence and belief in one's potential (MacPhee, et al., 2012).
-Job satisfaction (satisfaction with one's job position, work, pay and life) has also been proven to significantly influence performance and productivity.Improving job satisfaction and work environments holds the promise of better quality patient care (Lorber, et al., 2015).-Communication and interpersonal relationships.
Efficient cooperation enables an unrestricted exchange of ideas, closer relations among team members, trust, mutual respect and personal satisfaction of health care providers (Yoder-Wise, 2013).-Organisational commitment and commitment to the profession have also been found to be significant motivational elements in achieving the required performance (Yurumezoglu & Kocaman, 2012).
The non-discretionary inputs are not directly linked to an individual but to the characteristics of the hospital and their job position: -job position, i.e. the type (level) of leadership position: (1) team leaders; (2) senior nurses or small department heads; (3) leaders of wards, clinical departments, operational blocks, hospital units, outpatient clinic groups; (4) head nurses of clinics/hospital centres; (5) head nurses of hospitals, assistant directors of nursing; -the leader's organisation, i.e. the type of hospital: (1) small specialist hospitals; (2) middle-sized general hospitals; (3) large university medical centres; -the number of direct subordinates; and -the number of health care employees in the organisation.
The interconnection of the data from the first and second surveys was ensured with a coding system.The data for the non-discretionary inputs were obtained from hospital records.

Description of a sample
All 24 Slovenian hospitals were invited to participate; the 15 hospitals that participated account for 87 % of all hospital health care employees in Slovenia.All nurses and nurse leaders in 13 general and special hospitals were invited to join in the study.At the University Medical Centre (UKC) Maribor, questionnaires were given to all nurses and nurse leaders in one clinic (out of five) and at the UKC Ljubljana to all nurses and nurse leaders in three clinics (out of 18).The sample from the first survey included 3,938 nurses, 1,311 (response rate 33.3 %) of whom assessed the level of competencies of their immediate superiors.The average age of nurses was 38.5 years.59.2 % had a secondary education, while 34.4 % had completed a postsecondary or professional college, and 2.7 % a bachelor's degree or higher.Men accounted for 12.1 % of the 1,311 nurses (Table 1).The sample from the second survey included nurse leaders in participating organisations.The participating institutions employ 526 nurse leaders, 236 of whom (44.9 % of the sample) answered the questionnaire and were assessed by at least one subordinate (in the first survey).Women represented 92.8 % of the 236 nurse leaders.The average age of a nurse leader was 42.3 years, 88.6 % had completed a postsecondary education or professional college, and 11.4 % a bachelor's degree or higher.

Description of the research procedure and data analysis
The data were analysed using Frontier analyst 4.1.0and SPSS 19.0.Descriptive statistics were used to describe the sample.Based on the results for individual behaviours, a simple arithmetic mean was used to calculate values for four groups of competencies.A significance level of alpha = 0.05 was used for all statistical tests.
The study was approved by the Honorary Court of Arbitration of the Nurses and Midwives Association of Slovenia and the managements of individual hospitals.Participants were assured there was no risk from participating in the study and that their responses would be treated confidentially.

Results
In the competency approach, the assessments of each leader and their behaviour provided us with information regarding the degree to which they meet the norm defined in the competency profile.Values above 100 signify that the assessed leader exceeds the norm in an individual area, while values below 100 signify a failure to meet the norm (Table 2).On average, nurse leaders achieved a relatively high score in the competency area of ethics, but scored significantly lower in the area of generic leadership competencies.
Our performance appraisal is based on a large number of outputs, discretionary and nondiscretionary inputs.We used three-stage DEA to combine them and obtain a DEA efficiency score for each nurse leader (Table 3).Significant differences exist in the performance appraisal among leaders in nursing (confirming the first hypothesis).Thirty-three leaders achieved an efficiency score of 100, which means they are efficient and represent a role model for peers displaying lower assessments.All efficient leaders are female.Their average age is 38.9 years.They are excellent in all competency areas, but the highest output value is in the competency area of ethics (122.9).A total of 203 (86 %) leaders could improve their leadership performance based on their potential.However, great differences appeared for 16 (6.8%) of them who scored below 55 in their performance appraisal, meaning they could significantly improve their performance.The average age of the least efficient leaders is 47.1 years.Two of the sixteen (12.5 %) are men.
On average, the 16 lower-scoring leaders achieved a relatively high score in ethics, but scored significantly lower in other areas.In a comparison with peers, those leaders' greatest potential for improvement lies in the areas of generic competencies (148.8 %) and interprofessional relationships (131.0 %) (confirming the second hypothesis).
Table 4 shows the leadership performance appraisals of the five lowest-scoring leaders.Similar representations can be compiled for each of the 203 leaders performing below requirements.The focus of the model is to enhance the output, rather than decrease the inputs.Therefore, the performance appraisal values achieved by the leaders and the possibility for improvement are shown for each of the four groups of competencies (outputs).
The lowest-ranking leader (leader no. 1 -anonymous for data protection reasons) has a peer within the same hospital (hospital no.25) with whom to compare themselves against and learn from and exchange experience with.In comparison with the peer, leader no.1's greatest potential for improvement lies in the area of generic competencies.The second poorest score was achieved by a leader whose assessment showed the lowest results in ethics.Leader no. 2 also has a peer within their hospital that could help improve leader no.2's outputs.
Leaders in the lowest-ranking hospital (hospital no.4) achieved an average efficiency score of 69.9.Hospital no.4's greatest potential for improvement lies in the area of generic competencies.The third poorest score was achieved by leaders in hospital no. 7. Hospital no.7's greatest potential for improvement is in the area of interprofessional relationships.

Discussion
There are differences in efficiency scores among nurse leaders in Slovenia.The differences concerning the efficiency of employees are normal, but we have to analyse them to find areas where improvements are necessary (Rosenfeld, et al., 2012;Westphal, 2012;Vesterinen, et al., 2013).The greatest differences are in the areas of generic leadership competencies, where also the greatest improvements are possible.Generic competencies represent the characteristics of all leaders and the low ratings of nurse leaders in most hospitals reflect a deficiency in the leadership attitudes of the professional group as a whole.Nurse leaders should be aware of the importance of their role as leaders and of the importance of achieving high levels of individual leadership competencies (Lorber & Skela-Savič, 2011).Leadership training should begin upon the completion of formal education and continue systematically throughout the entire process of a nurse's professional development (Paterson, et al., 2010), and no longer be random depending on the needs of the organisation.Only in this way can the appropriate generic leadership competencies and their implementation in leadership in nursing be developed in individuals.
Great differences are also seen in the area of interprofessional relationships.For four of the sixteen lowest-scoring leaders, the biggest potential improvements lie in this competency area.Competencies involving good management of interprofessional relationships are particularly important for leaders in nursing because they continuously cooperate with other professional groups and because they must ensure that nursing plays an appropriate role in patient treatment (Yoder-Wise, 2013).Nurses in leadership positions must possess competencies which will enable them to achieve organisational goals while developing and sustaining healthy work environments for all employees (Fennimore & Wolf, 2011).
A competency-based performance appraisal can provide information for identifying areas requiring improvements for two stakeholders in the health care leadership process: employees in hospitals and nurses' professional associations.
Based on the efficiency scores, individual leaders can find peers within the hospital to serve as role models.The assessment also shows the degree to which they would need to improve based on the existing inputs, i.e. what they already possess (education, knowledge, networks, satisfaction, etc.) to achieve the performance level of their role model (Manoharan, et al., 2009).Nurse leaders can use the analysis results as a guideline regarding in which areas they need to improve their competencies.Less efficient leaders and their role models could form study groups in which they could exchange ideas and learn.The primary aim of these study groups should be to improve leadership behaviours, while later they could be used as a base for educational programme proposals (Griscti & Jacono, 2006).Less efficient leaders could be given an option of more training to improve their behaviours and actions or enrich their job by adding more responsibilities and increasing outputs while keeping the same level of inputs.Hospital managers can use these results to determine which areas of competencies the leaders in their organisation should improve.A special educational programme should be prepared for each of these areas of competencies.Nurse leaders would be accordingly acquainted with individual competencies and educated about proper actions and behaviour (Weber, et al., 2015).
Nurses' professional associations should ensure appropriate professional continuing education for nurses (Yoder-Wise, 2013).A variety of oriented study programmes should be developed.Most education programmes for nurses are organised for narrow professional fields of nurses' work (cardiology, psychology etc.).There is a need for interdisciplinary education programmes that would enable nurse leaders to obtain knowledge in various fields related to leadership in health care and the broader environment in which nurses work (Kvas & Seljak, 2013).In the future, the main task of professional associations should be to draft programmes that expand beyond the borders of individual professional fields.Nurses are a professional group bound to lifelong education due to the characteristics of their profession, as well as legal requirements (Gallagher, 2007).Therefore, it is an essential task of both professional associations and the education system for health care to equip nurses with leadership knowledge for only those professional groups with adequate leadership can be successful.Professional associations should promote better leadership in nursing and organise training for nurse leaders in areas showing poorer scores and promote exchanges of good practices among hospitals.

Limitations of the study
DEA and a performance appraisal are most commonly used for assessing the efficiency of a homogeneous group of employees (Manoharan, et al., 2009;Osman, et al., 2011).Most of those studies assess efficiency within a single organisation.Tasks performed by nurse leaders in different hospitals are similar, as are their cultural and working environments.However, performance appraisals are also influenced by other types of variables which are beyond the nurse leaders' control (Cordero, et al., 2009).In such conditions, it is difficult to ensure the homogeneity of a larger group of leaders from different organisations.Therefore, this analysis includes four non-discretionary inputs representing influences related to the specific characteristics of a job position and the organisation of employment.The influence of these variables on leadership performance is moderately significant.Arguably, there are also other influences which call for additional research that would test the influence of other variables.

Conclusion
The performance appraisals were based on the leadership competency model in nursing, which comprises two parts: generic and professionallyspecific aspects (ethics, inter-professional relationships, education).The main aim of leadership competency models is to incorporate all attitudes and behaviours that characterise good leaders.But since this is difficult to achieve in practice, human resources management must combine different methods to incorporate the remaining dimensions of the complex nature of leadership.This article presents a quantitative method for developing human resources in nursing care, which in practice must be complemented with qualitative methods.Nevertheless, it can be used as an effective initial indicator of areas requiring additional research.
It should be emphasised that a single best method of selecting, assessing and training leaders does not exist.Only a combination of several methods can lead to successful results.Human resources management must take into account as many variables from different sources as possible and combine several different methods.The DEA method provides us with results in a form that enables inefficient leaders to find peers within their organisations to help them improve their leadership skills.This is an individualised approach to leader development and, as such, it is only possible in environments willing to adopt new and modern forms of peer-based training and leader development through an exchange of good practices.

Table 1 :
Demographic data on the sample of survey respondents Tabela 1: Demografski podatki o vzorcu anketiranih