GAO - Program Evaluation: Experienced Agencies Follow a Similar Model for Prioritizing Research. Amid efforts to improve performance and constrain spending, federal agencies are being asked to expand the use of rigorous program evaluation in decision- making. In addition to performance data, indepth program evaluation studies are often needed for assessing program impact or designing improvements. Agencies can also use their evaluation resources to provide information needed for effective management and legislative oversight. GAO was asked to study federal agencies with mature evaluation capacity to examine (1) the criteria, policies, and procedures they use to determine programs to review, and (2) the influences on their choices. GAO reviewed agency materials and interviewed officials on evaluation planning in four agencies in three departments with extensive evaluation experience: Education, Health and Human Services (HHS), and Housing and Urban Development (HUD). HHS and HUD agreed with the description of how they plan evaluations. HHS noted that the optimal location of evaluation units will vary with the circumstances and purpose of evaluations. HUD felt the draft report did not emphasize enough the influence of the appropriations process. GAO has added text to note its influence on evaluation planning. Education provided technical comments. Although no agency GAO reviewed had a formal policy describing evaluation planning, all followed a generally similar model for developing and selecting evaluation proposals. Agencies usually planned an evaluation agenda over several months in the context of preparing spending plans for the coming fiscal year. Evaluation staff typically began by consulting with a variety of stakeholders to identify policy priorities and program concerns. Then with program office staff they identified the key questions and concerns and developed initial proposals. Generally, the agencies reviewed and selected proposals in two steps: develop ideas to obtain initial feedback from senior officials and develop full- scale evaluation proposals for review and approval. The four general criteria these mature agencies use to plan evaluations were remarkably similar: (1) strategic priorities representing major program or policy area concerns or new initiatives, (2) program- level problems or opportunities, (3) critical unanswered questions or evidence gaps, and (4) the feasibility of conducting a valid study. The agencies' procedures differed on some points. In any case, the Government reserves the right to make award(s). We use always use the standards of the American Evaluation Association. However, for inspections and accountability studies related to government programs, we also.External parties' participation in evaluation planning may reflect these agencies' common reliance on nonfederal program partners. Only the offices GAO reviewed in HHS' Centers for Disease Control and Prevention held formal competitions to rank- order proposals before submitting them for approval; in other agencies, senior officials assessed proposals in a series of discussions. When evaluation authority and funds are tied to the program, evaluators generally choose not which programs to evaluate but which research questions to answer. Sometimes this resulted in a program's never being evaluated. Evaluation units at higher organizational levels conducted a wider range of analytic activities, consulted more formally with program offices, and had less control over approvals. The Congress influences an agency's program evaluation choices through legislating evaluation authority, mandating studies, making appropriations, and conducting oversight. GAO concludes that (1) all four criteria appear key to setting an effective evaluation agenda that provides credible, timely answers to important questions; (2) most agencies could probably apply the general model in which professional evaluators iteratively identify key questions in consultation with stakeholders and then scrutinize and vet research proposals; (3) agencies could adapt the model and decide where to locate evaluation units to meet their own organizational and financial circumstances and authorities; and (4) agencies' reaching out to key program and congressional stakeholders in advance of developing proposals could help ensure that their evaluations will be used effectively in management and legislative oversight. GAO makes no recommendations. Assessment & Evaluation. Developing senior leaders in the U.S. Government through Leadership for a Democratic Society. Amid efforts to improve performance and constrain spending, federal agencies are being asked to expand the use of rigorous program evaluation in decision-making. An evaluation of TravelSmart Local Government program was. Government are outlined in the ACT Government Evaluation Policy and Guidelines (the Policy). It is important to acknowledge that program evaluation.
0 Comments
Leave a Reply. |
AuthorWrite something about yourself. No need to be fancy, just an overview. Archives
January 2017
Categories |