http://www.city.grande-prairie.ab.ca/perform.htm#IssuesAndPrinciples
PERFORMANCE MEASUREMENT
- ISSUES AND PRINCIPLES
The following list of issues
and principles are excerpted from papers presented at the February 95 conference
"Performance Measurement and Monitoring in Government" sponsored
by the Institute for International Research, and from articles on Performance
Measurement in the periodical "Public Management," Vol. 76, No.
9, Sept. 94. Because this is a collection, some concepts appear in more than
one context.
KEY PRINCIPLES:
·
Acceptance of the measurement process is essential to its success as a performance
improvement tool. Like Strategic and
Business Planning, the process by which you determine what to measure, how
to measure, and how to utilize the measures is more important than the actual
product itself.
·
The audience/user and purpose must be clearly defined. Who are the customers and end-users for the
measurement system? What are their
requirements? What do they feel they
need from measurement to help them do a better job managing, problem-solving, and decision-making?
·
The greater the participation in the process of creating a performance measurement
system, the greater the resulting performance change, and the greater the
ease of implementation of future changes based upon performance measurement.
(Morris, 1979). This participation includes Employees, Management,
Senior Administration, Council and the Electorate.
·
Measures must be seen to have value well beyond the task performance level.
Performance measurement and reporting thus becomes not only an accountability
tool, but also an advocacy tool. (Allen, 1985).
·
Measurement is hard and complex. Once
we accept this, measurement can become less difficult.
·
A complete and effective system of performance measurement will require
years of consistent, incremental work to achieve.
·
One of the reasons performance measurement is difficult is that these measures
were not available historically; and the resulting uncertainty dampens enthusiasm
substantially for some individuals and groups.
·
Further, it is complicated by the fact that there is no generally accepted
"bottom line" in civic government because there is no scientific
or analytical measurement that indicates the relative benefit to society of,
for example, more hospital beds, less toxic waste, less family violence, more
reforestation or better public transit. (Allen, 1985).
·
Measurement of any kind will affect the behavior of individuals within the
organization [for better or for worse]. It
has nearly universal capacity to focus attention. Management needs to recognize their obligation
to monitor and direct the resulting changes in focus. Reporting performance measures will also affect
the behavior of Senior Administration, Council and the Electorate.
·
One suggestion is to adopt an experimental approach to measurement systems
development in order to drive out fear. Eliminate
the use of numerical goals, work standards and quotas. Numerical measures are often achieved, even
when improvement is not. (Deming, 1986).
Deming, of course, is W. Edwards Deming, a page and extensive
references for whom can be found at the Clemson Continuous Improvement Website
at: http://deming.eng.clemson.edu/pub/psci/. Also on-line, the Deming Electronic Network
is "a volunteer-based, non-commercial electronic communications resource
available internationally to individuals and organizations interested in the
past, present, and future of Dr. W. Edwards Deming's System of Profound Knowledge
and related philosophies."
·
Central Principle: It is NOT "the right measures." Instead, it
is a process and culture for choosing, using and revising measures to assist
employees in focusing on achieving continuous improvement over the long run.
·
Do not make a commitment to measurement, benchmarking, TQM or any other
process or program, nor to the reporting of heroic results, nor to assigning
the blame. Instead, make a commitment
to service delivery improvement [the latter implies changing the way we do
things (usually cooperatively and in a spirit of good will), adding new activities
and deleting old (sometimes favorite) activities].
·
David Osborne and Peter Plastrik, in Banishing
Bureaucracy: the five strategies for reinventing government, in the chapter
on the "Consequences Strategy," say: "We have not listed performance
measurement as an approach because we do not believe
that the act of measuring itself has enough power to force fundamental change
in most public organizations. It is a critical competence organizations need
if they are to use the consequences strategy. But some public organizations have measured
performance for years, with virtually no impact." [P. 132, their emphasis.]
·
Aim to improve the things which will make a difference (those with large
costs, large customer value, substantial consequences, etc.).
·
Measure what employees can translate into direct corrective action. Measuring global hunger is interesting and is
of monumental importance, but few can apply any direct correction. "For metrics to be motivational, those
they apply to must be able to see what to do.
There must be a "line of sight" between the actions employees
can take and what shows on the measure. Being
held accountable for measures with no clear means to affect them is demotivating
at any level." [The foregoing quote is an excerpt from an article
"Performance Metrics: How to Use Them and How
to Get More Leverage" by Bob Frost.]
·
Good metrics (according to Ian Macdonald's "Performance Indicators"
paper) need to be:
directional - to confirm that you are on track to reach the
goals,
quantitative - to show what has been achieved and how much
more is to be done,
worthwhile - adding more value to the business than they
cost to collect and use.
·
Measure what's important strategically (or what is of value to customers),
not just what is easy to measure, or already being measured.
·
Measure group and team outputs, not individual outputs. Individual performance cannot be measured except
over the very long run. (Deming). Deming,
of course, is W. Edwards Deming, a page and extensive references for whom
can be found at the Clemson Continuous Improvement Website at: http://deming.eng.clemson.edu/pub/psci/.
·
Have the team that produces the result develop
the measures, perform the measurements and report the results.
CAVEATS:
Remember: Measurement at its best only
tells you something about the history of your performance.
Remember: no matter how well an employee's
work is planned, managed and measured, the outcome will depend much more on
how passionate the employee is concerning the work.
Remember: the word entrepreneurial doesn't
mean outsourcing; it means continuously shifting activities away from those
with lower returns and toward those with higher returns.
Remember: problems related to an organization's
output are much more often related to poor management of the systems than
to poor performance. An excellent manager's
superior interpersonal and leadership skills have much greater potential to
foster continuous improvement than does performance
measurement.
Remember: it is easy to measure the trivial.
It is much more difficult to measure what is truly important, and in
an objective way.
Remember: increased focus on individual
employee performance produces decreased focus on responsibility to the team
and the organization. Intense focus on individual performance encourages competition
at the expense of cooperation. ["Equipment and files just disappeared,
sometimes for weeks at a time, when individuals hoarded them for their projects.
They made their gains; but the team and the organization lost much more."]
Remember: Outstanding performance has
to make a difference (in recognition). If
everybody is treated the same when it is all said and done, then in the long
run performance will never become outstanding.
Remember: that nowadays successful management:
(a) completes the right task on time and within budget, (b) builds and strengthens
the team, (c) develops the individual and (d) fosters continuous improvement.
Remember: that outcomes (or results)
are what paid the bills last quarter. What
positions you to produce results in the coming quarter is the knowledge and
experience your workers gained executing (well planned and managed) processes
in the current quarter. For additional
information on Knowledge Management, see the series of Knowledge Management
articles in the Archive of the CyberCity
·
The goal is to design, develop and successfully implement measurement systems
that share information such that continuous performance improvement is supported
and enhanced.
·
The measurement system must clearly fit into the management process and
be acknowledged as decision-making and problem-solving aimed at performance
improvement support.
·
An effective measurement system must build upon consistent and well understood
operational definitions for the seven performance criteria
·
effectiveness,
·
efficiency,
·
total quality,
·
productivity,
quality of work life,
·
innovation, and
·
profitability/budgetability
·
The unit of analysis/target system must clearly be defined in order for
measurement to succeed. An input/output analysis is a necessary precondition.
·
The process of measurement must clearly be separated from the process of
evaluation.
·
Measure intermediate-term outcomes as well as longer-term outcomes. Set
dates for revision of intermediate-term outcomes to reinforce the perception
of their temporary status.
·
Do not stop with measurement. Take time to analyze results of measurement,
changes in behavior, etc. Take time to communicate results, propose improvements,
persuade others of the value of the proposed improvements.
Reward sound employee initiatives which are not adopted as well as those which
are. Remember, it is the processes and the culture and the structures for
focusing attention on continuous improvement which contain the long-term big
payoffs we are seeking (not just the immediate improved results for a specific
case).
·
Take time to analyze the real causes of the improvements. Adjust effectiveness measures accordingly.
·
Balance the degree of management control over the processes being measured
with the desirability of measuring outcomes (rather than outputs or inputs).
Generally, the measurement of items purely under management control (usually
only inputs) or pure global outcomes (usually substantially beyond management
control) are not nearly as useful as measures which strike a balance.
·
Encourage comment/analysis in measurement reporting. Special circumstances
need to be visible along with measures of performance (especially where the
measures anticipated only the routine case).
·
Control-oriented measurement systems often hinder continuous improvement
efforts.
·
Measurement is often resisted due to perceptions (real or imagined) of negative
consequences. Visibility of good performance leads to diminished resources.
Visibility of bad performance leads, initially, to more resources, but eventually
to punishment. Visibility of performance therefore often leads to crisis catering,
more measurement, micro-management [and little lasting improvement].
·
What is needed is not a standard set of measurements created by experts
or obtained from a "shopping list" that are imposed on the organization,
but rather a method by which measurement teams and their various clienteles
create and continually modify performance measurement systems suited to their
own inevitably special needs and circumstances. (Morris, 1975).
·
Any system should result in a vector of performance measures, not in attempting
to achieve a single measure. Much of the controversy and lack of acceptance
stems from attempts to make a very complex problem appear too simple. (Morris,
1975). [A corollary here is that the simplistic measure just causes people
to focus more on the simplistic view of the problem, usually causing an imbalance
of effort that produces results that are worse than the results obtained before
the measurement was introduced.]
·
A performance measurement system must not appear to those involved as simply
a passing fad. (Morris, 1975).
·
A useful system must be seen by those whose behaviors and performances are
being assessed as being non-manipulative and non-gamed.
An Informal Bibliography of
Sources of Performance Measurement Principles and Precepts.
The
above list of performance issues and principles came mainly from a case study
at the City of Tacoma, Washington by William Larkin entitled Establish Performance
Measurement Within a Municipal Government, given at the February 1995 conference
Performance Measurement and Monitoring in Government sponsored by the Institute
for International Research (708 Third Avenue, 4th Floor, New York, N.Y. 10017-4103;
Tel: 800+345-8016 or 212+661-8740; Fax: 212+661-6677), and from a series of
articles entitled Benchmarks of Performance, from the periodical Public Management,
Volume 76, Number 9, September 1994. Public Management is published monthly
by the International City/County Management Association (ICMA), 777 North
Capitol Street, N.E., Washington, D.C. 20002-4201 (Tel: 202+962-3620).
For
further interest we include the titles of the papers from the Performance
Measurement and Monitoring in Government conference as follows (other sources,
some with links may be seen below):
If
You Don't Know Where You Are Going, Any Path Will Do, ... or, The Case Against
Business-as-usual, B. Smith, Former Deputy Director of Defense Information,
Office of the U.S. Secretary of Defense, 22 pp., mainly overheads.
Prepare
for Performance Measurement, J. Hill, U.S. General Accounting Office, 10 pp.,
mainly overheads.
Using
Surveys to Improve Human Resource Management, M. Dole, U.S. Department of
the Interior, 42 pp., mainly overheads, but includes example survey questions.
Performance
Measures in
Design
Measurement Systems Based on "Customer Value", W. C. Parr, Director,
Center for Advancement of Organizational Effectiveness,
Create
an Environment for Measuring Performance - The Human Side of Quality, D. Joseph
and D. Glass, U.S. Air Force, 38 pp., mainly overheads.
Establish
Performance Measurement Within a Municipal Government,
W. Larkin,
Implementing Strategic Planning and Performance Measurement
in Arizona State Government, P. O'Sullivan-Kachel,
Governor's Office of Strategic Planning and Budgeting, State of
Analyzing
and Integrating Large-scale Organizational Processes, T. Gulledge,
Professor,
Measuring Program Outcomes and Impacts, M. Marsh, Oregon
State Department of Administrative Services, 22 pp., mainly overheads.
Evaluative
Criteria for Auditing a Performance Measurement System, G. Silva, City Auditor,
City of San Jose, California, 10 pp., mainly overheads.
The
Uses of Performance Measurement, R. Allen, Former Performance Measurement
System Manager, Province of Ontario, 14 pp., mainly overheads. John R. Allen
Voice: 416 + 769-2272; Fax: 416 + 769-2310; 61
Strategic
Management: Using the Strategic Plan to Build Organizational Budgets, D. Well
& L, Doherty, Department of the U.S. Navy Total Quality Leadership Office,
et al., 66 pp., mainly overheads.
Other
sources of Performance Measurement information include the following:
A
Performance Measurement Addendum page is available containing examples, including
those shown below. In both cases, energy is diverted from program effectivenss improvement into making reported measures look
good.
Performance
Measurement Addendum 1
This
comment deals with a side-effect which does not (and perhaps cannot) contribute
to program improvement.
Performance
Measurement Addendum 2
Goal Shifting and Staff Behavior in Perf
Measurement - Jerry Winston.
These
examples show how measurements confirm improved performance, while actual
effectiveness is not improved.
In
the overview of the MIT Learning History Research Project, Dr. George Roth
describes the use of "Learning Histories" (informal papers by participants
describing their view of learning efforts) aimed at avoiding the adverse effects
of assessments while facilitating the transfer of understanding among and
between groups. Excerpt from the overview: "All efforts to transform
organizations sooner or later run up against the challenge of proving their
value. Yet traditional "assessment" approaches, reacting to everyday
pressures, can easily undermine the original learning effort. As people become
aware of being judged and measured, they seek to satisfy the evaluation criteria
instead of improving their capabilities. The intrinsic motivation which drives
learning is then supplanted by the desire to look successful. Yet evaluation
is vital to learning as a feedback process that provides guidance and support.
"Learning Histories" were invented in response to this dilemma"
(our emphasis). Roth also provides a useful field manual and describes future
aspects of the research.
"Four
Measurement Traps" - a paper by Jim Clemmer
of the Clemmer Group. "Measuring performance
can cut both ways. It can play a valuable role in improving organizations
- or it can stand in the way of necessary change." (1)
Trying to manage results - that's history; (2) Measuring inside-out; (3) The
measurement stick; and (4) Confusing knowing and doing. In his book,
Pathways to Performance, Clemmer identifies (Chapter
19, Measurement and Feedback) five core measurement areas that provide balanced
feedback for assessing and improving organizational performance:
In
his book Firing on all Cylinders: The Service/Quality System for High- Powered
Corporate Performance Jim Clemmer of the Clemmer Group and Barry Sheehy report
on two broad categories of service quality used by Hewlett-Packard (chapter
18, "Standards and Measures"):
The
first category deals with the quality of the product:
Functionality
- the feature set, capabilities, compatibility, and security.
Usability
- the human factors, consistency and documentation of the product.
Reliability
- the frequency and severity of failures, the predictability and accuracy
of the product.
Performance
- the speed and efficiency of the product as well as the resource consumption.
Supportability
- maintainability and serviceability of the product, along with its ability
to be installed.
The second category deals with the quality of the relationship with the
customer:
Anticipation
- the ability to identify, understand and help solve customer needs before
they become problems.
Availability
- the degree to which our products and services provide for uninterrupted
usage at full functionality.
Responsiveness
- the ability to provide timely, accurate and complete information and/or
solutions to customer initiated requests for help.
Transitions
- the ease of initial startup and of ongoing changes as individual products
and services evolve and conform to new needs and technologies.
The
CyberCity Mailing List (Selected Current Topic Indexes) article
"The Death of Industrial Age Organizations," in the digest of
Robert
H. Kent of the Mansis Development Corporation posts
"Why You Should Think Twice About 360 Degree
Performance Appraisal." In the article,
The
Governing.com magazine for state and local government officials is hosting
the Government Performance Project which provides report cards for the states
(scroll down to "State Report Cards") and about three dozen cities
(scroll down again to "City Report Cards"). Among the cities, of
course,
The
The
Centre for Business Performance at the Cranfield
School of Management,
Zigon Performance Group posts an extensive page of links to
reviewed Performance Measurement Related Websites, including sections on Performance
Measurement, Performance Management, Performance Management Software, and
many others. Their page of "Articles by Jack Zigon" provides links to a dozen or more articles, including
"Measuring the Hard Stuff: Teams and Other Hard-to-Measure Work."
and "How to Measure White-Collar Employee Performance." And they
also sell performance measurement manuals, including one with hundreds of
example measures.
The
Cabinet Office of the British government has posted an "Access Checklist"
which is a first step in providing guidance for departments in improving access
to government services. They have focused on seven areas:
The
Cabinet Office also posts the "Performance and Innovation" website,
which
The
British National Audit Office lists among its publications a report on Good
Practice in Performance Reporting, (850 Kb; 66 pp; .PDF format) an Executive
Summary of which (95 Kb; 9 pp; .PDF format) is also available. For those without
access to the Adobe Acrobat file format, the Archive of the CyberCity Mailing List (Selected Current Topic Indexes) contains
an article "Converting Web Pages from PDF to HTML Format" in the
digest of
Kevin
Bounds' paper on the balanced scorecard (Kevin is Finance Director at Nationwide
Life) "The Role of Performance Measures and the Impact of Corporate Structure"
from the Foundation for Performance Measurement (about the foundation).
"Score
It a Hit" [cookies (cookie caution)] a November 1998 article in the
The Related Links page from the performance measurement
systems toolkit at the Department of Industrial and Systems Engineering at
Virginia Tech.
Performance Indicators, by Ian Macdonald. This paper compares the Balanced
Scorecard (Kaplan and Norton), the Performance Pyramid (Lynch and Cross) and
the Stakeholder Model (Curtis and Kastner).
The
International City/County Management Association (ICMA) posts some example
"Performance Indicators" for police and fire services, neighborhood
services and for support services.
Perform
Magazine is published several times a year in hard copy for a fee with some
articles on its website. The spring 1999 edition contains an article "Performance
Metrics: How to Use Them and How to Get More Leverage"
by Bob Frost. It emphasizes reporting results using graphs and with consistency
(of data collection and measurement) and reporting comparative information.
"Your
metrics need anchor points for comparison, ... [but]
think carefully about which comparatives will lead you to valid conclusions
and sensible action."
"If
your employees know that you value metrics and track the entire organization's
performance, an amazing thing happens: The culture of your organization changes.
Whether mentally or on paper, employees begin to track how their own performance
contributes to enterprise performance. And a results-tracking culture is one
of the most powerful competitive advantages your enterprise can have."
Fred
Nickols of The Distance
Consulting Company posts several dozen articles, including "Human Performance
Technology: The End of An Era," which provides a brief overview of his
contrarian view. He gets more specific with "Don't
Redesign Your Company's Performance Appraisal System, Scrap It!" and
"Now What?: What to do after you scrap your
performance appraisal system." He makes a number of very good points
to be considered in any kind of performance review or management initiative.
The
Australian state of
The
Local Government and Shires Associations' page "Towards Best Practice"
records and links information relating to benchmarking and best practice relevant
to Local Government throughout
The
U.S. General Services Administration [cookies (cookie caution)] "Office
of Information Technology" [cookies (cookie caution)] has posted a page
"Performance Pathways" [more cookies (cookie caution)] which provides
a good list of performance measurement materials and links [more cookies (cookie
caution)], including guides; sample performance measures; legislation, regulations
and policy; training sources; etc. Their "Performance-Based Management"
[yet more cookies (cookie caution)] report (table of contents) [cookies (cookie
caution)] provides a comprehensive discussion of the "eight steps to
develop and use IT performance measures effectively." It also includes
a good three page "Executive Summary" [cookies (cookie caution)].
In their finance area [more cookies (cookie caution)], they also post
The
U.S. National Performance Review (brief history) has issued some relevant
reports and papers:
An
annnouncement of the NPR "Managing for Results - Initiatives"
website appeared in the CyberCity mailing list article
"Managing for Results at U.S. Federal Gov't" which appeared in the digest of 4 March 1997.
Managing Results: Initiatives in Select American Cities.
Their
On-line Highlights lists new and interesting resources made available in the
last several months.
Their
News Room contains newsletters, press releases, and speeches.
Their
Library contains agency agreements, customer service improvements, executive
orders, performance measurement initiatives, privatization, regulatory reform
guidelines, etc.
The
Sunsite edition of selected National Performance Review links.
The
U.S. Department of Commerce Acquisition Community has posted a comprehensive
"Guide to a Balanced Scorecard: Performance Management Methodology,"
(Table of Contents) describing application of performance management in a
government agency.
"The
most successful performance measurement systems are not "gotcha"
systems, but learning systems that help the organization identify what works--and
what does not--so as to continue with and improve on what is working and repair
or replace what is not working."
The
The
Congressional Institute (a
The
FinanceNet has posted a page "GPRA - Integrating Performance
Measurement into the Budget Process." It contains sections on "How
to Do It: a Developing Cookbook on Integration" and "Challenges
and Considerations to Overcome Them." Most of
the pages are in PDF ( portable document format),
but they are clearly marked. Note: For those without access to the Adobe Acrobat
file format, the Archive of the CyberCity Mailing
List (Selected Current Topic Indexes) contains an article "Converting
Web Pages from PDF to HTML Format" in the digest of
The
Government Executive Magazine posts "The Results Report" They provide
a "Results Act primer for everyone trying to help their agencies get
their plans together as well as regular coverage of Results Act hearings,
workshops and news."
FinanceNet posts several other useful pages Government Performance
Measures/Benchmarking page; U.S. Government department Performance Plans (many
of the latter in PDF format, unfortunately); and Archive of Performance Measures
Mailing List Messages.
The
U.S. General Accounting Office (GAO) has posted a text page: "Executive
Guide - Effectively Implementing The Government Performance
And Results Act" (109 Kb) that is also available in Portable Document
Format (caution: the latter is a 60 page document). It points out that "every
major federal agency must now ask itself some basic questions: What is our
mission? What are our goals and how will we achieve them? How can we measure
our performance? How will we use that information to make improvements?"
Florida
Office of Program Policy Analysis and Government Accountabilty
posts "Florida Monitor," containing reviews of various performance-based
program budgeting measures and standards.
Overcoming the Inadequacies of Performance Measurement
in Local Government: The Case of Libraries and Leisure Services (abstract).
The
American Evaluation Association at the
OECD's page on Key Performance Management Issues.
The
US Department of Health and Human Services has embarked on a process to establish
performance measures for their public health programs. This is a good site,
with lots of health care related links.
Database of