Checklist Questions
- What needs to be achieved through the OD intervention?
The laboratory needs to become self-sufficient and preserve capabilities necessary for national security and defense. The OD intervention, or rather series of interventions, will move the organization toward this ultimate goal. No doubt, this will be a multi-year process with many coordinated interventions. Because of the breadth and length of the OD initiative, evaluation plays a critical role in making sure that the organization is making progress toward its goal.
- What would the desired change state look like?
The desired change state is that the organization generates enough revenue to support existing programs, avert layoffs, and maintain facilities and equipment in a state of readiness. An even more desired state would be growth – additional employees and growth in revenue, because this would directly reflect on the organization’s ability to meet its customers’ requirements.
- What metric will be used to measure success for the desired change after the intervention?
In the summative evaluation phase (Park, J., 2018), we will be measuring our success using annual revenue, number of employees, and condition of facilities and equipment. These lagging measures will tell us whether our overall initiative was successful. Along the way, in the formative evaluation phase, we will assess the intervention using leading measures and KPIs such as customer satisfaction scores, number of patent applications, time to complete a project, employee satisfaction, percentage of employees accessing training, safety record, and facility/machinery downtime.
- Who/which are the targets of change (e.g. people/group/units)? What level of change do we need?
In this change initiative, which is a whole system change, it is safe to say that all employees, managers, and even customers are targets of change as they adapt to new cultural expectations and adopt new processes and practices. The change is taking place at all levels of the organization and across all divisions of the organization. Specifically, this change initiative includes the following categories of workers:
- What are the specific evaluation expectations at each level?
It is expected that we will evaluate each level using the same assessment criteria – metrics for financial, behavioral, and skills appropriate for their level of the organization. We will use the annual personnel assessment process to identify and track individual progress in these areas. However, we expect that each level of the organization will be involved in coming up with metrics and a process to measure their success. Additionally, responsibility for collecting data and conducting the evaluation will be pushed down the lowest levels of the organization. Ideally evaluation sponsors and change agents are identified at the lowest levels of the organizations to ensure buy in to the initiative from the employees who are doing the work.
- Are the evaluation expectations aligned with the aims and levels of the change intervention?
We expect that evaluation expectations will be appropriately sized to the level of leadership the individual holds in the organization. For instance, the director will be evaluated on whether she met her $300 million organizational revenue goal, while an individual team lead may be held accountable for only a $2 million revenue goal. Similarly, behavioral change is also aligned with the levels of the change intervention. Leaders are expected to model certain entrepreneurial behaviors and employees modeling behaviors aligned with good customer service.
- Are you in agreement with each level of targets, with regard to evaluation goals and objectives?
Employees will participate in development of the evaluation plan and in performing the evaluation, so agreement with evaluation goals and objectives on the part of each level of targets is essential.
Influential Factors of Evaluation Strategy Planning
- How will you secure time for designing, data gathering, analyzing, and reporting the evaluation outcomes? Include who you will network with to get these resources
This OD initiative is crucial to the organization’s survival and as such already has the support of all senior leaders in the organization who hold the purse strings. One person at the senior leader level will be identified as the overall champion of the evaluation initiative. He or she will be supported by contractors.
- How much time will each employee or groups of employees need to participate in the evaluation elements you have defined to date? What will they need to do in these events – what is their role?
The evaluation will require the attention of employees up and down the organizational hierarchy. It is expected that specific evaluation metrics will be identified and tracked by each team, with reporting occurring at each team meeting.
- What other departments will need to be included in order for your project (and the evaluation plan) to be successful? What is their role in approval/chain of command/resource allocation/general support etc?
The Chief Financial Officer, Chief of Human Capital, and Director of Business Processes are key leaders in the evaluation initiative. They will be instrumental in managing and tracking major components of the evaluation plan and ensuring employee participation. For this project they will report to the senior leader assigned as champion, who in turn reports to the director of the laboratory. Additionally, Team leaders, Branch chiefs, and Division directors all play significant roles because they command resources in the organization. In addition to ensuring that their employees take necessary training, these leaders will play an enormous role in modeling the new behaviors expected in an entrepreneurial organization. Thus, these leaders will be among the first to be trained and evaluated.
- Will you need to purchase expertise – consultant or other help to complete the project AND should those additions be included in the evaluation plan – if so what would you evaluate (stick to the outcomes rather than evaluating the person who is performing the tasks).
The evaluation plan will be supported by a team of contractors working to help implement the OD change initiative. Their support will be necessary to help set up data collection and analysis systems throughout the organization. They will also provide the ongoing data collection work. They will report to the senior leader and champion of the project. Their work is evaluated by the project officer and the contract officer assigned to monitor their work.
- Is there a need to acquire any technology? Use current technology in a new way? Request technology help?
We will create a dashboard that integrates evaluation plan metrics and displays progress toward goals and objectives. This will be created by in-house programmers who have the capability of tying this dashboard into existing data collection systems.
- What are the opportunity costs for doing the project and not doing the project AND for planning and implementing an evaluation plan and for doing no evaluation?
The laboratory is in an existential battle for its survival. The opportunity cost for not doing the project is clear – the organization will cease to exist and be absorbed into other laboratories. This change initiative is our strategic goal and success in this endeavor is necessary for the lab to continue serving the country. The opportunity for not doing the evaluation is that we will not know whether the interventions are effective, efficient, and having the desired impact. As mentioned previously, this is a long-term change initiative and progress will be marked incrementally. Showing progress, no matter how small, will be important to sustaining interest and motivation over several years. This can only be accomplished by identifying and tracking meaningful metrics across the organization.
Reference:
Park, J.W., (2018). Planning the Evaluation. In Jones, M. C., & Rothwell, W. J. (Eds.). Evaluating Organization Development: How to Ensure and Sustain the Successful Transformation (1st ed., pp. 75-88). Productivity Press.