Outcomes

Legal Aid Society of Eastern Virginia: Zoning In on Housing Justice

sierra-chacona@lsc.gov — Thu, 12 Sep 2024 16:18:14 +0000

Legal Aid Society of Eastern Virginia: Zoning In on Housing Justice sierra-chacona… Thu, 09/12/2024 - 12:18

In this project brief, learn how the Legal Aid Society of Eastern Virginia leveraged Housing Justice Program Fellows to provide focused legal assistance to housing communities in southeastern Virginia.

Legal Aid Society of Eastern Virginia: Zoning In on Housing Justice

Grantees will learn how the Legal Aid Society of Eastern Virginia (LASEV):

1.    Identified neighborhoods with the highest rates of eviction filings;
2.    Prioritized direct community connections to build trust with residents in these neighborhoods; and
3.    Advocated to improve living conditions and reduce evictions.

Hear from LASEV Deputy Director of Litigation Sarah Black and Housing Justice Fellow Brandon Ballard on how they increased trust in LASEV, preserved valuable housing subsidies, and improved rent repayment plans.

Read Project Brief

“By participating as a host site for [the Housing Justice Program], LASEV was able to focus much-needed time and effort on addressing the most pressing issues facing residents."

Excerpt from the Project Brief

More on Providing Legal Services

Visit this MPI topic page to explore providing legal services resources.

Explore Providing Legal Services Resources

More Model Practices & Innovations

Visit this page to explore additional Model Practices & Innovations Resources.

Explore Model Practices & Innovations Resources

Data Outcomes Housing / Evictions

Idaho

A Culture of Systems Change

sierra-chacona@lsc.gov — Tue, 27 Aug 2024 16:17:08 +0000

A Culture of Systems Change sierra-chacona… Tue, 08/27/2024 - 12:17

Learn about Legal Services NYC's litigation process and considerations for identifying potential impact litigation cases in this infographic.

Legal Services NYC’s Impact Advocacy: A Culture of Systems Change

Grantees will learn how Legal Services NYC (LSNYC):

1. Maximized resources to achieve the greatest possible benefits and systemic solutions;

2. Partnered in and out of the court systems to achieve the best possible outcome; and

3. Coordinated impact advocacy across five programs.

Discover how LSCNYC outlined the program’s litigation process and the factors to consider when identifying potential impact litigation cases.

Read Infographic

“LSNYC understands that potential impact litigation cases can come from any practice area, as well as outside the organization.”

Christine Clarke

Chief of Litigation and Advocacy, Legal Services NYC

More on Strategic Planning

Visit this MPI topic page to explore strategic- and succession- planning resources.

Explore Strategic Planning Resources

More Model Practices & Innovations

Visit this page to explore additional Model Practices & Innovations Resources

Explore Model Practices & Innovations Resources

Leading and Managing an Organization Outcomes Strategic Planning

New York

Neighborhood Advocacy Program

sierra-chacona@lsc.gov — Tue, 20 Aug 2024 15:38:34 +0000

Neighborhood Advocacy Program sierra-chacona… Tue, 08/20/2024 - 11:38

Learn how the Legal Services of Eastern Missouri's neighborhood Advocacy Program helps prevent property vacancy abandonment and promotes neighborhood redevelopment in this project brief.

Legal Services of Eastern Missouri: Neighborhood Advocacy Program

Grantees will learn how Legal Services of Eastern Missouri (LSEM):

1. Recruited and formalized partnerships between St. Louis-area law firms and place-based community organizations;

2. Reduce neighborhood property vacancy by organizing private investments; and

3. Preserved homeownership through title clearance and estate planning.

Learn about how LSEM used the Pro Bono Innovation Fund to expand a neighborhood-based pro bono project to prevent property abandonment and promote the redevelopment of long-abandoned properties.

Read the Project Brief

"Neighborhood Advocacy’s significant accomplishment was its ability to quickly create and implement a trusted, well-respected, neighborhood-based pro bono project in an area with no pre-existing legal support system.”

Excerpt from the Project Brief

More on Providing Legal Services

Visit this MPI topic page to explore resources on effective legal service delivery models.

Explore Providing Legal Services Resources

More Model Practices & Innovations

Visit this page to explore additional Model Practices & Innovations Resources

Explore Model Practices & Innovations Resources

Grant Program Housing / Evictions Outcomes Pro Bono Innovation Fund (PBIF)

Missouri

Cultivating Diversity to Retain Talent and Increase Inclusion

sierra-chacona@lsc.gov — Wed, 31 Jul 2024 19:14:21 +0000

Cultivating Diversity to Retain Talent and Increase Inclusion sierra-chacona… Wed, 07/31/2024 - 15:14

Learn about the Legal Action of Wisconsin's efforts to establish diversity, equity, and inclusion within the organization by enhancing staff retention, morale, and hiring practices in this case study.

Legal Action of Wisconsin: Cultivating Diversity to Retain Talent and Increase Inclusion

Grantees will learn how Legal Action of Wisconsin (LAW):

1. Integrated diversity, equity, and inclusion (DEI) values within the organization’s culture;
2. Enhanced staff retention and morale; and
3. Expanded diversity and inclusion for clients.

Hear from Jennifer L. Johnson, Director of Diversity & Inclusion at LAW, about the DEI Committee crafting LAW's vision, values, and mission and overcoming challenges to incorporate DEI values into the organization.

View the Case Study

”Staff opinions and feedback are crucial to incorporating DEI into organizations, as is ensuring the staff feel valued and heard when leadership solicits their feedback.”

Jennifer L. Johson

Director of Diversity & Inclusion, Legal Action of Wisconsin

More on Management

Visit this MPI topic page to explore management resources.

Explore Management Resources

More Model Practices and Innovations

Visit this page to explore additional Model Practices and Innovations resources.

Explore Model Practices and Innovations Resources

COVID-19 Intake Leadership Leading and Managing an Organization Outcomes Strategic Planning

Wisconsin

From Courtroom to Community: How Legal Services NYC Uses Impact Advocacy and Litigation to Expand Access to Justice

sierra-chacona@lsc.gov — Wed, 31 Jul 2024 18:36:32 +0000

From Courtroom to Community: How Legal Services NYC Uses Impact Advocacy and Litigation to Expand Access to Justice sierra-chacona… Wed, 07/31/2024 - 14:36

Learn about Legal Services NYC's tips, tricks, and strategies for conducting impact advocacy in this case study.

From Courtroom to Community: How Legal Services NYC Uses Impact Advocacy and Litigation to Expand Access to Justice

Grantees will learn how Legal Services NYC (LSNYC):

1. Conducted impact advocacy through strategic litigation;

2. Allocated resources and support for high-impact cases; and

3. Managed the team structure and pro bono partnerships.

Hear from Christine Clarke, Chief of Litigation and Advocacy at LSNYC, about key takeaways and lessons learned from some of the organization’s most impactful casework over the years.

View Case Study

“An important aspect of conducting sustainable impact work is shifting organizational culture to see legal aid work not just as solving the same problem a thousand times, but as addressing inequality on a systemic level.”

Christine Clarke

Chief of Litigation and Advocacy, Legal Services NYC

More on Strategic Planning

Visit this MPI topic page to explore strategic planning resources.

Explore Strategic Planning Resources

More Model Practices and Innovations

Visit this page to explore additional Model Practices and Innovations resources.

Explore Model Practices and Innovations Resources

Intake Leading and Managing an Organization Outcomes Strategic Planning

New York

Shaping Stories through Impact Advocacy and Litigation

sierra-chacona@lsc.gov — Wed, 03 Jul 2024 15:25:54 +0000

Shaping Stories through Impact Advocacy and Litigation sierra-chacona… Wed, 07/03/2024 - 11:25

Learn Colorado Legal Services’ tips, tricks, and strategies for conducting impact advocacy through strategic litigation, handling related cases, and filing amicus briefs.

Colorado Legal Services: Shaping Stories through Impact Advocacy and Litigation

Grantees will learn how Colorado Legal Services (CLS):

1. Conducted impact advocacy through strategic litigation;

2. Handled high volumes of impactful cases; and

3. Filed amicus briefs to advocate for expanding civil protections.

Hear from Matt Baca, Executive Director of CLS, about key takeaways and lessons learned from some of the organization’s most impactful casework over the years.

View Case Study

“Day-to-day legal aid cases do not always make headlines, but they still provide clients with peace of mind, justice, and dignity in the most dire moments of their lives.”

Matt Baca

Executive Director, Colorado Legal Services

More on Strategic Planning

Visit this MPI topic page to explore strategic and succession planning resources.

Explore Strategic Planning Resources

More Model Practices and Innovations

Visit this page to explore additional Model Practices and Innovations resources.

Explore Model Practices and Innovations Resources

Intake Leading and Managing an Organization Outcomes Strategic Planning

Colorado

Technology Initiative Grant Awards: TIG Projects Funded by Year

admin — Mon, 17 May 2021 20:27:17 +0000

Technology Initiative Grant Awards: TIG Projects Funded by Year admin Mon, 05/17/2021 - 16:27

2023 Technology Initiative Grant Awards

In the 2023 cycle, LSC awarded 33 Technology Initiative Grants (TIGs) to 29 legal services organizations totaling over $5 million. This year’s TIG recipients are taking several innovative approaches to leveraging technology to strengthen their organizations and expand access to legal services. Click here to see the 2023 TIG Awards Press Release.

2021 TIG Awards Project Descriptions

Previous TIG Awards by Year

2022 TIG Awards
2021 TIG Awards
2020 TIG Awards
2019 TIG Awards
2018 TIG Awards
2017 TIG Awards
TIG Awards Archive

In 2022, LSC awarded Technology Initiative Grants (TIGs) to 33 legal services organizations totaling $4,679,135. Grant recipients have used this funding to enhance cybersecurity, build educational platforms, strengthen program capacity and support the work of pro bono attorneys. Successful TIG projects are often replicated by organizations around the country, creating wide-reaching impacts. Click here to view the press release for the 2022 TIG award recipients.

2022 TIG Award Recipients

In 2021, LSC awarded Technology Initiative Grants (TIGs) to 26 legal services organizations totaling $4,268,938. These funded projects provided greater access to high-quality legal assistance for low-income Americans and enhanced the operations of legal aid organizations around the country. Click here to view the press release for the 2021 TIG award recipients.

2021 TIG Award Recipients

In 2020, LSC awarded Technology Initiative Grants (TIGs) to 26 legal services organizations totaling $3,941,298.00. LSC's grant awards include projects that use technology to improve self-help resources, enhance internal efficiency, and improve access for vulnerable populations. Click here to view the press release for the 2020 TIG award recipients.

2020 TIG Award Recipients

In 2019, LSC awarded Technology Initiative Grants (TIGs) to 30 legal services organizations totaling $4,230,718. LSC’s grant awards include projects that use technology to improve self-help resources, enhance internal efficiency, and improve access for vulnerable populations. Click here to see the press release for the 2019 TIG award recipients.

2019 TIG Award Recipients

LSC awarded 26 TIG grants to 24 legal services organizations in 21 states for a total of $3,884,257. Among the 26 funded initiatives are projects that enhance online self-help resources, expand services to rural communities, and facilitate clients’ ability to seek legal help via text messaging services and voice assistants like Google’s Siri and Amazon’s Alexa. Click here to see the press release for the 2018 TIG award recipients.

2018 TIG Award Recipients

LSC awarded nearly $4 million in TIG funds for 22 legal services organizations in 18 states. Among the 25 funded initiatives are projects to create interactive legal forms for self-represented litigants, to enhance an online statewide “access portal” with a chatbot feature, and to produce online resources for veterans and military families confronting legal problems. Click here to see the press release for the 2017 TIG award recipients.

2017 TIG Award Recipients

This report describes TIG funds awarded from 2000-2009.

TIG Awards 2000-2009

Outcomes Grant Program Technology Initiative Grant (TIG)

small

richardsons@lsc.gov

Resources

fanlundk@lsc.gov — Mon, 17 May 2021 19:14:51 +0000

Resources fanlundk@lsc.gov Mon, 05/17/2021 - 15:14

Resources and Information Specifically Focused on the Use of Outcomes Data by Legal Services Programs

��˶��:

Model Practices & Innovations. The “Evaluations & Adjustments” section of LSC’s Best Practices Corner has a selection of resources regarding the use of outcomes measures by LSC grantees and other legal services programs.
LSC Summit on Performance Measures: Assessing Quality and Measuring Results. Materials from LSC’s Outcomes Summit II on June 24-26, 2004.” This summit was convened by LSC, the Hale and Dorr Legal Services Clinic, and the Bellows-Sacks Access to Civil Legal Services Project at Harvard Law School. Materials include a summit report as well as papers prepared for the event.

The Resource for Great Programs provides strategic support and technical assistance to civil legal services programs and other networks of community-based organizations. It supported the establishment of the outcomes reporting systems used by IOLTA funders in Virginia and New York, and has worked with other state funders of civil legal aid and legal services programs around the country on program evaluation, the economic impact of legal aid, and other issues.

Like Virginia, the California State Bar has a set of outcomes reporting requirements for IOLTA-funded programs in the state.

Jeffrey Selbin, Jeanne Charn, Anthony Alfieri, and Stephen Wizner, Service Delivery, Resource Allocation, and Access to Justice: Greiner and Pattanayak and The Research Imperative, 122 YALE L. J. ONLINE 45 (2012). The authors call for legal services lawyers and clinical law professors to conduct extensive empirical research concerning the efficacy of various “access to justice” strategies. They argue that this research can inform service delivery approaches and resource allocation within legal aid programs, providing crucial information at a time when demand for legal services is growing, while supply is shrinking.

Ken Smith, Barbara Finkelstein, and Christopher O’Malley, “Economic Impacts of Legal Aid” (Management Information Exchange Journal, Fall 2011). This article discusses the economic benefits that legal services representation provides the general public, profiles the ways three New York legal services programs used these economic benefits data to expand their funding at a time when state and local government budgets were badly strained, and suggests strategies for measuring the economic effects of legal aid outcomes and presenting this information persuasively to potential funders.

Resources on evaluation and outcomes measurement for nonprofit organizations

The Bureau of Justice Assistance Center for Program Evaluation and Performance Measurement provides “user-friendly online evaluation and performance measurement tool designed to assist state and local criminal justice planners, practitioners, State Administrative Agencies, researchers, and evaluators in: 1) conducting evaluations and performance measurement that will address the effectiveness and efficiency of their projects and 2) using evaluation information to improve program planning and implementation.” Its resources include basic outcome evaluation and comparative outcome evaluation.

Authenticity Consulting, LLC. has published very useful online evaluation tools and other management resources for nonprofit organizations. These include:

Basic Guide to Program Evaluation (Including Outcomes Evaluation), which provides guidance for planning and implementing evaluations.
Basic Guide to Outcomes-Based Evaluation for Nonprofit Organizations with Very Limited Resources, which provides guidance for basic planning and implementation of an outcomes-based evaluation process for nonprofit organizations (including those with very limited resources).
Corporation for National and Community Service, Performance Measurement Core Curriculum. This web portal provides a variety of resources designed to improve grantees’ understanding of key performance measurement concepts and evaluation, including: Performance measurement basics, theory of change, evidence, performance measures, and data collection and instruments.

The Innovation Network provides training, technical assistance, and other services to enhance nonprofit organizations’ program effectiveness. Its website contains a variety of resources regarding evaluation, including the use of outcomes measures.

Joint Center for Housing Studies at Harvard University conducts research and education to increase the understanding of and inform policy about housing issues. “Demystifying Outcome Measurement in Community Development” describes outcome measurements, why they are important, and the costs and benefits of undertaking outcome measurements for a community development organization (CDO). Although it focuses on CDOs, the information it contains can be useful for legal services programs.

The Impact Foundry seeks to enhance the resources and improve the management of nonprofit organizations. Its website includes a variety of very useful materials regarding evaluation and the use of outcomes measures.

The United Way of America (UWA) was among the first funders to require its grantees—including legal services providers—to report outcomes data. UWA has produced many publications about the use of outcomes measures, including:

“Measuring Outcomes of United Way-Funded Programs: Expectations and Reality,” which provides a discussion and assessment of “the strengths and limitations of the distinguishing features of the UWA approach, efforts to disseminate the approach, implementation by local United Ways, and actual outcome measurement by local agencies.”
“Connecting Program Outcome Measurement to Community Impact,” which provides information about using program outcome measurement in the work of a community impact. It provides insights into the ways UWA and local United Ways use outcomes.

The Urban Institute has three web portals with resources and information in using outcomes.

Outcome Indicators Project. Provides information regarding frameworks, outcomes, and outcome indicators to help nonprofit organizations analyze and improve program performance.
A useful resource is “Building a Common Outcome Framework To Measure Nonprofit Performance.”
Performance Measurement-Non Profits has a series of “six outcome management guidebooks to assist nonprofit organizations in implementing the development or improvement of a process for tracking the outcomes of their work.” The introductory guide, “Key Steps In Outcome Management,” provides an overview of the outcome management process, identifies specific steps and provides guidance for using outcomes information.
PerformWell, a joint initiative of the Urban Institute, Child Trends, and Social Solutions, is a web portal with outcomes and effective practices resources to help nonprofit practitioners identify and use quality performance indicators to improve program performance.

The Nonprofit Technology Network did a year-long study of how nonprofits collect and use data, and produced “Collected Voices: Data-Informed Nonprofits,” an eBook of advice and case studies.

Idealware, a nonprofit dedicated to helping other nonprofits make good use of technology, has many articles and resources on the topic of outcomes management:

“Nonprofit Performance Management: Using Data to Measure and Improve Programs” is an in-depth study of how 10 nonprofits are tracking their outcomes.
“Beyond Dashboards: Business Intelligence Tools For Program Analysis and Reporting” outlines a number of tools that can be used to work with and dynamically present outcomes data.

Outcomes

small

California Evaluation and Main Benefits Handbook

Breeglej@lsc.gov — Mon, 17 May 2021 18:50:10 +0000

California Evaluation and Main Benefits Handbook Breeglej@lsc.gov Mon, 05/17/2021 - 14:50

While not a case study, this handbook from the California State Bar describes their methodology for outcomes tracking and contains numerous examples of forms and reports.

California Evaluation and Main Benefits Handbook (2015)

Outcomes

small

Cleveland Legal Aid's Use of Outcomes

Breeglej@lsc.gov — Mon, 17 May 2021 18:37:22 +0000

Cleveland Legal Aid's Use of Outcomes Breeglej@lsc.gov Mon, 05/17/2021 - 14:37

The Cleveland Legal Aid Society uses outcomes measures to more effectively and efficiently meet clients' needs. Key components of its outcome measures system are highlighted in the following sections:

Why Cleveland Uses Outcomes Measures
How Cleveland Developed Its Outcomes Data System
Major Outcomes Categories and Metrics Cleveland Uses
Systems Cleveland Uses to Collect, Compile, and Analyze Outcomes Data
Examples of the Ways Cleveland Uses Outcomes Measures
Cleveland’s Lessons Learned ��˶�� Using Outcomes Data

Why Cleveland Uses Outcomes Measures

Prior to 2007, the data Cleveland collected was limited to outputs data such as LSC Case Services Reporting (CSR) data, LSC other services (matters) data, and similar information required by specific funders. Very few funders—most notably United Way—required outcomes data at the time. Cleveland did collect some outcome data for its own purposes, but their value was limited because they were not compiled in a consistent fashion and were not the most significant outcomes of the program’s client services.

In 2007, Cleveland's management determined that the data they were collecting was inadequate because it did not allow the program to meaningfully analyze and assess the results of its work. CSR case closing data and similar output information documented the volume and levels of the program’s work in different substantive law areas or community engagement and partnerships, but the program could not show the ways and extent to which the program’s work benefited the client community.

These limitations led the program to develop a more comprehensive outcomes measures system. The program did not replace CSR-type data and other outputs and descriptive data with outcomes data. Instead, it uses the combination of this data.

Cleveland uses outcomes data to better accomplish four fundamental, interrelated objectives.

Objective 1: Assess the program’s success in achieving the program’s mission and strategic goals

Cleveland’s core mission is to secure justice and resolve fundamental problems for low-income and vulnerable people by providing high-quality legal services and working for systemic solutions.

Its strategic goals—the desired end results—for client services are to:

Improve safety and health
Promote education and financial stability
Secure decent, affordable housing
Ensure that the justice system and government entities are accountable and accessible

Cleveland determined that data about the results of its work—outcomes data—were essential to effectively assess whether and to what extent it was accomplishing its mission and goals. And without outcomes data, the program could not determine how well its work was aligned with its mission and goals, nor could it identify how its work might need to be changed to ensure it was most effectively focused on the mission and goals.

Objective 2: Allocate resources and developing strategies

Cleveland also found that the outcomes data it collects allows the program to gain a more complete understanding of the results and impact of its work, which in turn enables the program to more efficiently allocate its limited resources. For example, the program needed outcomes data to assess whether its allocation of resources among different substantive law areas or among different regions within the service area were most appropriate. Similarly, outcomes data were essential to assess the effectiveness of advocacy strategies and to develop the strategies that would provide the greatest benefits to the client community.

Objective 3: Fundraising and “telling our story”

The program determined that CSR data and other outputs data did not demonstrate the value of its work or convince other potential funders that supporting civil legal aid is a good investment, especially when the program was competing with many other social services organizations for limited resources. In fact, some funders had a “So what?” reaction to proposals with deliverables that were limited to CSR or similar outputs data. Funders wanted to know what impact their investment would make in clients’ lives.

The use of CSR and other outputs data had similar limitations when the program tried to tell its story to the local media or community partners. Anecdotes about the ways the program helped individual clients could be very compelling, but the program needed more comprehensive data to effectively demonstrate that the benefits for individual clients were examples of the broad benefits it provided the community.

Objective 4: Improve organizational performance and staff engagement

Cleveland staffers were skeptical when the program’s leadership first began discussing outcome measures. They expressed concerns that the use of outcomes would divert resources from client services to bureaucratic record keeping, drive the program to “cherry-pick” cases that were easy, have limited benefits, or be used as an inappropriate staff evaluation tool.

Program management successfully addressed these concerns in the development and implementation of the system. (The outcomes data is not used to evaluate individual employees’ performance.)

Staff have come to embrace and value the outcome measures system.

Staff were integrally involved in the system’s development and its ongoing improvements.
They have found the data collection process to be efficient and easy to use.
Advocates consider the data essential for ensuring the efficient allocation of resources and for developing and improving advocacy strategies.
By demonstrating the impact and value of their work, the system has strengthened morale.
The system’s benefits have fostered increased staff engagement in program planning.
The outcome system data have enabled staff to identify ways to enhance the impact and effectiveness of their work.

How Cleveland Developed Its Outcomes Measures System

Cleveland spent about a year developing its outcomes measures system. It had to develop the system’s substantive and technical components and ensure that these components were effectively integrated. The program recognized that the staff had to be integrally involved in the development process. The board was also interested in the program developing more meaningful measurements of achievement, and supported the development of the new system.

The development process was not as linear as the description below implies. Instead, it was an ongoing iterative process in which proposals would be developed, analyzed by staff stakeholders, and then revised.

Outcomes are collected only for extended services cases.

Phase 1: Articulate the vision and engage staff

The program’s executive director and deputy director led and coordinated the effort. The first phase in the process was for management to articulate to the staff its vision for the system as clearly and as frequently as possible. Management focused on how the system would enhance the program’s effectiveness in serving clients as well as how the system would help staff identify and improve their own work. Staff expressed their views about the potential system’s benefits and pitfalls and also made suggestions about how the system could be most effectively developed and implemented. The concerns and ideas of all program staff were essential to the successful development and implementation of the system.

Phase 2: Identify substantive outcomes

After management set out its vision and actively solicited feedback and input from the staff, it turned to members of the program’s intake unit and four substantive law practice groups:

Consumer, including Foreclosure (LSC CSR problem codes 01-09)
Housing, excluding Foreclosure (problem codes 60-69)
Family (problem code 30-39), and
Health, Education, Work, Income, and Immigration (problem codes 11-19, 21-29, 41-49, 51-59, 71-79, and 81-89)

The practice groups were tasked with developing the substantive elements of the system. Specifically, the group was charged with identifying the outcome measures and indicators they thought would enable them to best assess and demonstrate the benefits of their work to clients.

The practice groups developed their outcome measures using the same framework. Outcomes needed to be defined from the client’s perspective. The key questions they asked were:

Why are we doing this work?
What do we want to accomplish in each substantive law area?
What data would we need (and can obtain) that would enable us to know what we were accomplishing?
What do we need to report to funders?

In addition, the outcomes needed to be based on the LSC CSR substantive issue categories (“problem codes”)—e.g., Consumer/Finance, Family, Health, and Housing. Outcome questions are specifically associated with each of these groups, so that the number of questions required for any case is only a small subset of the entire list of questions.

A major challenge was developing a list of outcomes (and associated indicators) that would not be too long and detailed, but at the same time would provide enough information to effectively assess and demonstrate the impact of the advocates’ work. The outcome measures the groups ultimately developed are identified in the "Major outcomes categories and metrics" section below.

This process took about six months.

Phase 3: Develop the technical requirements

The third phase of the process involved developing the technical requirements for collecting the outcomes measures and indicators. The program’s technology staff led this effort. They had to develop the functionalities in the case management system (CMS) and other data systems that would: (a) enable advocates to easily and efficiently input the results of their casework; (b) provide for effective data compilation and analysis; and (c) produce the reports the program needed to assess and show the impact of its work to different audiences. And, as indicated above, these functionalities needed to be based on and integrated with the LSC CSR problem codes.

The program’s CMS lacked the necessary functionalities. Therefore, Cleveland staff worked with staff of the Ohio Legal Assistance Foundation to develop these functionalities. These are discussed in the "Systems to collect, compile and analyze outcomes data" section below.

This process took about four months.

Phase 4: Staff training

The fourth phase of the process was to train staff on how to use the outcome measures and the CMS. All staff using the system were trained. The training was coordinated by management, IT staff, and the practice group leaders. Rather than formal training, small group conversations were conducted with members of different practice groups. After an overview of the system functions and content, IT staff and managers engaged with staff to respond to questions, provide necessary guidance on specific issues, identify areas for additional system improvements and further assistance, etc. An initial training period of about two months was followed by an ongoing process of assessment and adjustment of the system to enhance its effectiveness.

This process took about two months.

Major Outcomes Categories and Metrics

This section identifies the major outcomes categories Cleveland uses as well as the types of indicators and methods on which these measures are based. The outcomes correspond to three of Cleveland’s strategic goals:

Improving safety and health
Promoting education and economic stability
Securing decent, affordable housing

Cleveland is able to track their achievements related to these goals by aggregating (combining) the outcomes data from different substantive law case types. See Cleveland's Strategic Goal Outcomes for the list of outcomes that are combined to calculate the total outcomes for each of these goals.

There are two basic outcome categories:

Category 1: Outcomes that describe the result of the case or matter

These outcomes are associated with a particular CSR problem codes (e.g., Consumer/Finance, Housing, Family). For example, the outcomes for the Consumer/Finance area include, but are not limited to:

“Obtained monetary claim”
“Reduced/avoided debt”
“Increased assets"
“Obtained/restored utilities"

Each of these potential outcomes require the advocate to indicate "yes," "no," or "not applicable." That way, Cleveland can track the percentage of success for each outcome indicator. Requiring responses to every question also helps ensure data integrity. Cleveland Legal Aid does not have to wonder whether the advocate only picked the first outcomes on the list and did not record the most important or all of the outcomes achieved.

View the complete lists of outcomes collected for each of the substantive law areas.

Category 2: Financial outcomes secured for the client (where relevant)

These include data regarding changes in the amounts of a client’s monthly income, assets, or debts that can be attributed to Cleveland’s work. The two basic questions for including and calculating financial data are: “If Legal Aid had not been involved, what would the client's [value/amount of income/asset/debt] be at the time the case was closed?” and “What was the client's [value/amount of income/asset/debt] at the time the case was closed?”

Here is the form an advocate completes at case closing that shows the financial and non-financial outcomes for a consumer case:

Systems Cleveland Uses to Compile, Analyze and Report Outcomes Data

Cleveland LAS expanded the functionalities of its case management system (CMS) in order to compile, analyze, and report outcomes data effectively and efficiently.

In-house staff and staff of Ohio Legal Assistance Foundation (OLAF) made the necessary CMS changes. The original development, testing, and refinements of the system took about four months. Given the programming and other technology tools developed since then, this process would be much easier and quicker today. (A skilled programmer could probably complete the requisite tasks in about two weeks.)

Functionalities that were added to the CMS include:

The ability for case handlers to record the appropriate outcomes data for closed cases. On the front end, case handlers would see the questions to which they needed to respond to input the data when they closed the case. Here is a screenshot with the questions for Consumer cases.
Contain these answers in fields related to other CMS fields so that the outcomes can be linked to type of case, level of service, and other key fields.
Develop the report formats that are most useful for staff, board, management and funders.
Develop the Crystal Reports required to create those reports.
Develop the Excel spreadsheets required to present the data in charts and graphs.

Cleveland uses Crystal Reports to compile data and generate a variety of reports with outcomes data. A sample of these reports are highlighted below.

Some reports are automatically generated and routed to appropriate staff (e.g., executive director, practice group managers) on a monthly basis. The data in these reports can also be readily compiled for different time periods (e.g., quarterly, annually). Examples of these reports include:

Reports that provide outcomes data for each of the LSC CSR problem code areas as well as financial outcomes data for the entire program. Here is the report with annual data for 2013.
Three-page reports that provide the same type of information for each of the program’s four substantive law practice groups and the key outcome data for each practice group. These include data regarding case volume and case staffing (page 1), client data (page 2) and outcomes data (page 3). See an example of this report (with annual data) for the housing group. Here is the same report, combining the data for all of the practice groups.

Other reports combine different outcomes and other data.

Reports with outcomes and other data corresponding to the program’s strategic goals. Here is a report that highlights data regarding the Health, Education, Work, Income, and Immigration (HEWII) practice group:
Reports comparing clients’ and advocates' assessments of case results:
Reports with historical trend data showing the success rates for different case types and the amount of financial benefits provided clients:
Funder reports that combine CSR-type data, outcomes data, and quantitative and descriptive data about the program’s outreach and other community work. See an example: Cleveland Evidence of Success-St. Lukes.

Examples of the Ways Cleveland Uses Outcomes Measures to Improve Its Services

There are many ways that Cleveland uses its outcome measures to enhance its organizational performance and client services.

Resource allocation. At the height of the foreclosure crisis, case outcomes data showed that a percentage of clients were unable to keep their homes. Many clients would inevitably lose their home because of their inadequate resources. Data analysis indicated that unless a client’s income was at least 75 percent of the poverty line, they would likely lose their home. Therefore, the program decided to stop taking foreclosure cases where the client’s income was less than 75 percent of the poverty line. Instead, the program concentrated its resources on helping those clients who would be able to keep their homes if they had the program’s assistance. Cleveland LAS could not help all of the clients with incomes above 75 percent of the poverty line who needed the program’s help. However, a high percentage of clients—about 70 percent—the program did help were able to keep their homes.
Resource development. The use of outcomes data significantly increased Cleveland’s fundraising success. By using outcome data in their grant proposals and reports, Cleveland was able to translate the impact of its work into a language that funders understand. Most funders do not fund lawyers or access to justice. They fund programs that, for example, provide housing, stabilize families, or increase access to healthcare. By combining outcomes data with CSR-type data and information regarding the program’s work with community groups, Cleveland was able to concretely demonstrate the impact legal advice and counsel could have on achieving those outcomes. View an example of a funder report that incorporates all of these data types: Cleveland's Evidence of Success-St. Lukes.
Staff morale. Cleveland's leadership did not initiate the use of outcomes to improve staff morale. However, that has been a very important and valuable unintended consequence. The outcomes data profiled in the various reports generated by the program demonstrate to the staff the value and benefits of their work to clients.

Cleveland’s Lessons Learned ��˶�� Using Outcomes Data

Since it began using its outcomes system, Cleveland has learned a range of lessons about how outcome measures can be most useful. The most significant are discussed below.

Many important aspects of the program’s work cannot be quantified. The program’s management and staff conclude that critical aspects of its work cannot be meaningfully quantified. These include the quality of an advocate’s casework, the impact of the program’s work on a client’s sense of well-being (e.g., from avoiding homelessness, escaping from an abusive partner) or empowerment, the impact of the program’s work with community organizations, and if and how its work increases the accountability and accessibility of the justice system and government entities to the low-income population.

Every measure is imperfect. The program recognizes that the outcome measures it uses reflect assumptions and practices that can limit their validity and value. Questions include: Are they measuring the right things? Are the definitions of success appropriate? Are the measures based on the appropriate indicators? With regard to the financial outcomes, is the assessment about what the clients’ finances would have been without the program’s intervention valid? Is the method valid that is used to combine (“roll-up”) various outcomes to determine success in achieving strategic goals? Are the advocates accurately reporting the information case outcomes? The program addresses these issues by recognizing and trying to make refinements to overcome these limitations and being transparent about them.

Consistency is important. Because there may be different legitimate answers to each outcome question, it is important that staff members use consistent definitions and assessments in responding.

Outcome measures are necessary, but not the only tool to assess effectiveness. Outcomes provide valuable insights about the impact and value of Cleveland’s work. However, outcomes must be used in combination with a wide range of other assessment tools and datasets to effectively analyze and maximize the effectiveness of program operations and benefits for clients. These include CSR data, Census data and data from other federal and state agencies about the client’s population characteristics (e.g. geographic location, economic and demographic data, education and employment status), ongoing engagement with client and community groups, surveys and interviews with staff and community members, and GPS or similar data to align services with need.

Ongoing evaluation and improvement of the system is essential. The program’s management and staff engage in a continuous assessment of the outcomes system to identify ways it is—and is not—working. A key aspect of this process is to ensure that the system does not distort the program’s work, such as by driving it to “cherry-pick” cases that can be easily measured and result in high success rates.

Outcomes

small

Outcomes

Legal Aid Society of Eastern Virginia: Zoning In on Housing Justice

Legal Aid Society of Eastern Virginia: Zoning In on Housing Justice

“By participating as a host site for [the Housing Justice Program], LASEV was able to focus much-needed time and effort on addressing the most pressing issues facing residents."

More on Providing Legal Services

More Model Practices & Innovations

A Culture of Systems Change

Legal Services NYC’s Impact Advocacy: A Culture of Systems Change

“LSNYC understands that potential impact litigation cases can come from any practice area, as well as outside the organization.”

More on Strategic Planning

More Model Practices & Innovations

Neighborhood Advocacy Program

Legal Services of Eastern Missouri: Neighborhood Advocacy Program

"Neighborhood Advocacy’s significant accomplishment was its ability to quickly create and implement a trusted, well-respected, neighborhood-based pro bono project in an area with no pre-existing legal support system.”

More on Providing Legal Services

More Model Practices & Innovations

Cultivating Diversity to Retain Talent and Increase Inclusion

Legal Action of Wisconsin: Cultivating Diversity to Retain Talent and Increase Inclusion

”Staff opinions and feedback are crucial to incorporating DEI into organizations, as is ensuring the staff feel valued and heard when leadership solicits their feedback.”

More on Management

More Model Practices and Innovations

From Courtroom to Community: How Legal Services NYC Uses Impact Advocacy and Litigation to Expand Access to Justice

From Courtroom to Community: How Legal Services NYC Uses Impact Advocacy and Litigation to Expand Access to Justice

“An important aspect of conducting sustainable impact work is shifting organizational culture to see legal aid work not just as solving the same problem a thousand times, but as addressing inequality on a systemic level.”

More on Strategic Planning

More Model Practices and Innovations

Shaping Stories through Impact Advocacy and Litigation

Colorado Legal Services: Shaping Stories through Impact Advocacy and Litigation

“Day-to-day legal aid cases do not always make headlines, but they still provide clients with peace of mind, justice, and dignity in the most dire moments of their lives.”

More on Strategic Planning

More Model Practices and Innovations

Technology Initiative Grant Awards: TIG Projects Funded by Year

2023 Technology Initiative Grant Awards

Previous TIG Awards by Year

Resources

Resources and Information Specifically Focused on the Use of Outcomes Data by Legal Services Programs

Resources on evaluation and outcomes measurement for nonprofit organizations

California Evaluation and Main Benefits Handbook

Cleveland Legal Aid's Use of Outcomes

Why Cleveland Uses Outcomes Measures

Objective 1: Assess the program’s success in achieving the program’s mission and strategic goals

Objective 2: Allocate resources and developing strategies

Objective 3: Fundraising and “telling our story”

Objective 4: Improve organizational performance and staff engagement

How Cleveland Developed Its Outcomes Measures System

Phase 1: Articulate the vision and engage staff

Phase 2: Identify substantive outcomes

Phase 3: Develop the technical requirements

Phase 4: Staff training

Major Outcomes Categories and Metrics

Category 1: Outcomes that describe the result of the case or matter

Category 2: Financial outcomes secured for the client (where relevant)

Systems Cleveland Uses to Compile, Analyze and Report Outcomes Data

Examples of the Ways Cleveland Uses Outcomes Measures to Improve Its Services

Cleveland’s Lessons Learned ���˶��� Using Outcomes Data

Cleveland’s Lessons Learned ��˶�� Using Outcomes Data