Pennsylvania Department of Labor & Industry

The Pennsylvania Department of Labor and Industry (the Department) was awarded a United State Department of Labor (USDOL) Workforce Innovation Fund (WIF) grant in 2015 to implement the “Micro-credentials: Opportunity Through Stackable Achievements” project. The project united seven community colleges and seven workforce development boards (partnerships) with the goal of creating bite-sized educational opportunities for students with barriers to education and employment. The idea of micro-credentials differs from traditional educational models where the degree (credential) is not received until a series of competencies are displayed via requisite courses. Micro-credentials are designed to allow students to earn credentials in a shorter timeframe, helping address the needs of the individuals to quickly gain skills and become employed. The Department contracted TPMA to conduct the robust evaluation required by USDOL.

TPMA conducted a three-pronged rigorous evaluation, including an implementation evaluation with a curriculum study, an outcomes evaluation with predictive analyses, and a cost evaluation. The implementation evaluation tracked grant implementation, including curriculum development, through regular update calls, site visits, surveys, and a curriculum rubric.

For the outcomes evaluation, TPMA created a data tracking system, with common intake forms, to ensure participants were tracked consistently across partnerships. The outcomes evaluation measured enrollment, persistence, and completion of micro-credentials and pathways, and measured changes in participants’ employment status and wages. The cost evaluation assessed the cost per student outcome, the cost effectiveness of implementation, and the cost allocation of grant funds. TPMA partnered with Next Step Associates for the curriculum study and the Indiana Statistical Consulting Center for the predictive analysis.

Through the site visits and surveys, TPMA engaged upward of 50 people at each partnership, from college presidents to students and industry leaders. TPMA provided annual recommendations to partnerships and the Department based on findings from the various methods of data collection. TPMA provided training and technical assistance throughout the grant period to assist with data collection processes and updated partnership data files as changes were made to program structures. TPMA developed a data sharing agreement with the Pennsylvania Center for Workforce Information Analysis (CWIA) to obtain participant employment and wage data from Unemployment Compensation records.

The final summative evaluation report detailed the key findings and methods from the “Micro-credentials: Opportunity Through Stackable Achievements” program. A total of 700 individuals enrolled in the 19 micro-credential pathways offered across the seven partnerships. The outcomes evaluation found that participants who completed at least one micro-credential, and were employed both 12 months pre-program and six months post-program, had an average increase of $801.50 per quarter, a highly significant increase (p = 0.0178). The implementation evaluation found that themes of success included grant and programmatic flexibility and partner engagement, while themes of challenges included grant structure and requirements and serving the target populations. Effects of the program are anticipated to continue beyond the grant, including sustainment of micro-credential programs; establishing additional partnerships with employers and community; sustained collaboration between workforce boards and community colleges; and continued modifications and improvements to the programs.