Social work programs collect end-of-practicum evaluations every year because accreditation requires it. Practicum Evaluation Insights turns that data into agency profiles, review flags, trend views, and a dashboard for field education leadership, helping teams review agency patterns more clearly over time.
As a staff member in a field education program, I worked with Lumivero’s practicum placement software every day. It did exactly what it was built to do. It housed active placement sites, managed forms, and handled the administrative backend work.
What it did not do was help the team review agency patterns over time. After each practicum cycle, student evaluations were reviewed, what worked and what did not was summarized, and the results were filed away. There was no clear way to compare patterns across agencies, no trend view, and no early sign that a placement site was starting to decline.
That became harder to ignore through two parts of my work. One was reading through the most difficult student evaluations. The other was speaking with an alumna whose practicum experience nearly pushed her away from the profession entirely.
While working on our program's accreditation review, I found something important that focused the direction of this project. We were meeting CSWE benchmarks across the competencies. But I kept coming back to the same question. What does that actually tell us about what students are experiencing at their practicum sites?
The evaluation process provided useful information, but the review structure did not clearly track agency patterns over time. This workflow was built to address that.
"She told me her practicum had been so discouraging it nearly deterred her from wanting to practice social work at all."
An alumna of the programEnd-of-practicum surveys collected at the close of each placement cycle.
Highlights from the strongest and weakest experiences are identified manually.
There's no multi-year data compilation, no agency-level patterns, and the upcoming cycle lacks historical insight.
Data was being collected, but there was no system in place for ongoing reviews.
Field education is the signature pedagogy of social work. Students complete at least 450 hours at a placement site, usually around 15 to 20 hours each week. A strong placement can build confidence and professional judgment. A weaker one can leave students underused, underprepared, or questioning whether they want to stay in the profession.
Evaluations are collected and filed, but there is no consistent process for comparing patterns across agencies and over time. Each cycle begins with limited visibility into what happened before.
A site may begin to decline across multiple student cohorts, and that pattern can remain difficult to see because there is no structured way to track it over time.
Concerns often become clear only after the placement has ended, when students have already completed the experience, and nothing about that placement can be changed.
The effort to collect and review evaluations was there. What did not exist was the structure needed to turn that information into agency-level monitoring. This project builds that structure.
CSWE requires programs to assess student competence, which is important. But competency success and placement quality measure different things, and neither can substitute for the other.
The University of Montana School of Social Work has collected end-of-practicum evaluations from students each year. Before building the pipeline, the dataset needed to be reviewed, cleaned, and profiled so the thresholds and review signals reflected real patterns in the data rather than arbitrary cutoffs.
The cleaned dataset covers academic years 2014-2015 through 2025-2026. It includes both structured Likert-rated survey responses and open-ended student feedback. Each row represents one student evaluation of one agency in one academic year.
| Item | Value |
|---|---|
| Total evaluations | 2,258 |
| Practicum agencies | 222 |
| Agencies with sufficient data | 217 |
| Academic years covered | 12 (2014–2026) |
| BSW evaluations | 856 (37.9%) |
| MSW evaluations | 1,402 (62.1%) |
| Mean Placement Quality Score | 3.83 (SD 0.46) |
| Score range | 2.49 to 4.93 |
Most agencies scored above 3.5. The lower end of the distribution had more review flags.
MSW students rated placements higher on all six measures. Three of those differences were statistically significant using Welch’s t-test at p < 0.05.
Running
python run_all.py
executes the full pipeline in order. The initial setup takes less than an hour. Annual updates take less
than five minutes.
The pipeline is documented clearly enough that a staff member with no prior Python experience can replicate it using the public GitHub repository or run it interactively in Google Colab . No local installation is required for the Colab version.
Requiring two or more overlapping signals helps reduce false alarms and keeps the review focused where concern is more likely to reflect a real pattern. A single signal on its own could be noise, such as a small cohort or an unusual year. When multiple signals overlap, the workflow is identifying something worth a closer look.
29.5% of the 217 agencies with sufficient data had two or more simultaneous concern signals active. All
thresholds are stored in
config.py
and can be adjusted without modifying the pipeline logic.
A measure built from Likert items covering supervision, learning goals, and practice readiness. This marks the lower portion of the distribution.
The share of students who said yes when asked whether they would recommend the agency to a future student.
VADER sentiment analysis applied to open-ended comment fields. This captures near-neutral or negative language in professional evaluative writing.
A theme tag is applied when students describe paperwork, shadowing, or limited direct practice as the dominant experience.
An agency’s recent Placement Quality Score dropped more than 0.30 points below its own historical average.
The analysis combined structured ratings, recommendation data, sentiment scores, and open-ended themes from 2,258 responses. Three findings shaped the workflow and the way the results are reviewed.
Maroon triangles show flagged agencies. Gray dots show unflagged agencies. The shaded area indicates agencies where both the Placement Quality Score and the recommendation rate fall below the review thresholds.
Agencies with lower Placement Quality Scores also tended to be agencies where fewer students would recommend the site to a future student. When both measures were weaker simultaneously, review flags were more likely to appear.
p = 0.89 across 217 agencies
Knowing an agency’s competency score tells you almost nothing about how students rated the placement experience. Both measures matter, but they do not measure the same thing.
Students described stronger placements in terms of supervision, learning, and real practice. They described weaker placements more often in terms of administrative overload, weaker structure, and limited client contact
2025-2026: first time below the 3.5 concern threshold in 12 years of data. The program had no way to know until this pipeline ran.
The workflow revealed a pattern that wasn't clear before. In 2025-2026, the Placement Quality Score fell to 3.41, the first time in 12 years it dropped below 3.5.
Select any flagged agency from the dropdown to view the full profile produced by the workflow. Each field comes directly from the output, including scores, review signals, theme breakdown, trend direction, and representative student language.
Six views, sidebar filters, agency profiles, trend monitoring, language patterns, and benchmark alignment are all embedded here for review.
For the fully interactive dashboard with the most current data from the latest workflow run, open the live Streamlit dashboard directly
SSW Placement Quality Dashboard ↗
python run_all.py
, and the dashboard reflects the latest evaluation data automatically.
Embedded preview above shows selected views. For full functionality including all 222 agencies and live filtering, open the SSW Placement Quality Dashboard ↗
The pipeline was built using data from a single program. But the issue it addresses is not limited to one program, or even to social work.
CSWE requires programs to evaluate the effectiveness of field-based settings. Most collect this data every year. Very few have a structured way to use it for agency-level review.
Nursing, counseling, education, and public health all face the same problem. Evaluation data is collected each year, but patterns remain difficult to see without a monitoring process.
These platforms house the agencies and manage placements. However, they currently do not track placement quality over time. They need a pipeline to monitor this quality, which can be added to their existing setup instead of running it separately.
The pipeline runs as a standalone workflow for any program that exports evaluation data. It can also be integrated into existing placement management software as a monitoring feature, turning what programs are already required to collect into something they can actually act on.
If your program collects end-of-practicum evaluations with Likert-rated items, open-ended student comments, and agency identifiers, this workflow can be adapted to your data format. The logic does not depend on UMSSW-specific fields. It depends on having structured ratings, narrative text, and a way to identify which agency each response came from.
One configuration file. One-time setup. No code changes needed.
The first full pipeline run processes all historical evaluation files.
Adjust the review thresholds to match your distribution. Stored in config.
Add a new evaluation file and rerun the workflow. The dashboard refreshes automatically.
This workflow supports review. It does not replace the professional knowledge, relationships, or judgment that field education leadership brings to agency decisions.
Agencies with fewer than three evaluations in a given year are excluded from that year’s analysis. Scores based on very small cohorts can shift dramatically because of individual student fit rather than a broader agency pattern.
Agencies that have gone through mergers, staffing transitions, or other organizational changes may show score declines that reflect disruption rather than a lasting change in placement quality. Review flags should be interpreted with that context in mind.
A review flag means multiple signals were active at the same time. It does not explain why. The Field Education Team’s history with an agency, including site visits, supervisor relationships, and student feedback, is what turns a signal into a meaningful judgment.
Some agencies attract students who arrive more prepared because they offer pay, qualify as employment-based placements, or are easier to access. Strong competency scores at those sites may reflect student preparation as much as placement quality. The pipeline shows the pattern. It cannot determine the cause.
Agency profiles are not rankings, and they are not public. They are structured inputs for a leadership review conversation guided by the Field Education Team. The goal is to help programs ask more focused questions earlier, not to automate decisions that belong to people.
This project grew out of my role as a staff member in a field education program, specifically from watching evaluation data collected, summarized, and filed away each year without a way to build on it year to year.
The method used here, a multi-signal review pipeline applied to structured evaluation data, is not limited to social work. It can be adapted to any program that collects structured evaluations of practicum or internship sites.
The full code and documentation are publicly available. The pipeline is designed to be handed off. Any staff member with the evaluation export file and access to the repository can replicate or update the analysis.