Increasing the Power of Transparency by Assuring Quality of Credentials
There have been calls for a focus on outcomes in postsecondary education and training for decades. But these conversations have happened largely between similar providers or programs. The frameworks and metrics of quality and definitions of “outcomes” haven’t been consistent or comparable. Education and training providers use inconsistent and opaque systems to report outcomes, which makes it almost impossible to gauge quality.
This is changing, however. Recently, several organizations, including Education Strategy Group, the Education Quality Outcomes Standard Board (EQOS), and National Skills Coalition (NSC) have been developing frameworks to contribute to the conversations around credentials’ value and quality including:
- Market alignment: Is the credential relied upon or prioritized by employers hiring for in-demand, high-wage roles? Is it industry recognized?
- Equity: Are there gaps in or credential attainment or other outcomes by race, gender, or other student characteristics?
- Outcomes: Are outcomes connected to a specific credential comparable and reliable? Are the data related to credentials around job placement and earnings defined using the same metrics and consistent definitions? Is the credential attainment data reported reliable? Valid? Audited by a third party?
These three organizations are collaborating with Credential Engine to map their quality assurance frameworks to the Credential Transparency Description Language (CTDL). This will ensure that no matter which framework is used by a state, it will be available in a consistent, common language. States will also be able to more easily highlight credentials of value and compare their results through the Credential Registry.
Quality Assurance: Why and How
States do not lack for education and workforce data; every state has implemented a longitudinal data system which captures data from early childhood through education and workforce development (often referred to as a P20W data system). Many states are already collecting data on credentials from degree-granting and non-degree granting providers. But most state leaders desire quality information to improve decision making form the State Capitol to the classroom to the kitchen table of individual opportunity seekers. Quality information is timely, comparable, reliable, and useful. Decision makers at all levels require quality data around job opportunities, skill and knowledge demands, and which credentials and pathways to pursue.
Credentials begin to make sense of the plethora of data sitting in states’ P20W data systems by connecting the dots between various data silos. Efforts to make this connected data more transparent through a shared, common open space and using a shared language are a critical step, but the value of this credential transparency rises exponentially when it includes assurances of quality.
Providing decision makers at all levels with frameworks to describe the attributes of a quality credential is one that requires partnership across state policymakers and with other stakeholders such as employers, education and training providers, advocates, and accreditors. The four organizations authoring this brief offer an emerging description of the quality assurance process so that state policymakers better understand the opportunities to support quality information about credentials of value. The components captured in the descriptions and graphic below are illustrative of the phases that organizations working with states have identified as necessary to develop a shared definition of quality credentials and the policies to nurture an effective and efficient labor market.
State policymakers, working together with critical stakeholders, can lead efforts to prioritize high-quality and high-value credentials through the following actions:
Describe: Universal descriptions make it possible to compare data across various providers and credentials. To support assurances of quality credentials, there needs to be consistent definitions and descriptions of credentials themselves, the outcomes of those earning a credential, and its labor market value.
- Description of Credentials: The skills, competencies and knowledge signaled by attainment of the credential are described through a linked, open data structure that allows for interoperability, portability, and comparability of all credentials.
- Description of Outcomes: The outcomes of programs granting credentials need to be defined in the same way and measured using agreed upon common metrics in order for them to be comparable. This is critical, for example, to ensure that all programs are describing and measuring job placement of people earning the same credential in the same way to ensure comparisons across programs are reliable and valid.
- Description of High-Value Credentials: The attributes of a credential of value include validation by employers, recognized labor market need, and has measurable positive employment and earning outcomes. These consensus descriptions of quality make it possible to compare credentials within or across states.
Assess: While the first step of quality assurance is to articulate and adopt definitions of high-quality, high-value credentials, the impact emerges only after states establish benchmarks to capture it, and develop frameworks for gauging value, relevancy and quality that can be applied to both credentials and providers of credentials. As greater numbers of states adopt and implement a shared common framework that assesses quality and value, the more comparability there is across the field, which leads to greater insights and transparency.
Incent: Quality assurance standards and frameworks created and adopted by states, employers, job platforms/boards, and other labor market aggregators can help differentiate and highlight credentials of quality and value. This makes it easier to also create and implement incentives that prioritize learners’ pursuing recognized credentials. This differentiation by quality and value is also critical to the state’s role to protect consumers against predatory or non-quality programs.
Connect: State leaders are critical to ensuring the quality assurance process is a coordinated and aligned system that serves all learners well. Policymakers, education and training institutions, state agencies, philanthropy, and advocates can all collaborate to ensure the players, programs and providers are working together to create a system that works for the opportunity seeker and employer.
This Quality Assurance process has emerged through the efforts of thought leaders, policy leaders and advocates who care about ensuring that critical information about education and training programs and credentials is available to everyone. The graphic and these descriptions of the “phases” of the process will continue to evolve as more organizations and states join the discussion.
National Skills Coalitions (NSC) developed a consensus definition of quality non-degree credentials with four components:
- Evidence of substantial job opportunities related to a credential, and employers must confirm that the credential is used in hiring, retention or promotion decisions.
- Evidence that credential-holders master competencies. Seat time isn’t as important as whether the learner has the shills needed to do the job.
- Availability of information, including the employment and earnings outcomes of individuals after obtaining a credential–disaggregated by race, ethnicity, gender and other characteristics when possible.
- Proof that the credential is one step along a career pathway. Non-degree credentials should be embedded in or lead to additional educational and training opportunities.
In 2020-21, the NSC held its Quality Postsecondary Credential Policy Academy to help states adopt a cross-sector definition of quality non-degree credentials; develop a policy agenda to increase the number of residents with quality credentials; and advance data policies to support such efforts. Alabama, Colorado, Louisiana, New Jersey, Oregon and Virginia participated.
Education Equality Outcomes Standards Board (EQOS) is a nonprofit that develops universal quality outcomes standards and their indicators. The organization’s mission is to establish a consistent process for collecting and reporting results data across all postsecondary education and training programs so that learners can compare them.
EQOS outlines basic definitions, metrics and guidelines for postsecondary providers to track and evaluate student success through its quality assurance framework. The framework provides universal guidelines for all postsecondary providers to assess their students’ learning, completion, placement, earnings and satisfaction.
The framework has five metrics around provider outcomes:
- Learning: Pre- and post-program assessments that delineate expected learning outcomes; assess student proficiency; ascertain credential attainment; and evaluate student competency.
- Completion: The percentage of learners who graduated within the expected time.
- Placement: A breakdown of students eligible to receive a credential in the past 36 months, including the percent who moved to relevant positions within 180 days completion.
- Earnings: The median annual earnings the first full calendar year post-graduation, as well as the change over time during the next five years.
- Satisfaction: Pre- and post-program assessments gauging students’ purpose for attending, preliminary expectations and the likelihood of recommending the experience. It also includes a survey gauging the likelihood of employers to rehire graduates.
EQOS offers toolkits, templates, and technical assistance to education and training providers as they collect and report their data. It offers an EQOS Certification of Quality for providers that complete the process and have their outcomes data audited by a third-party source.
Education Strategy Group (ESG) convened a cross-sector expert workgroup in 2018 to develop strategic recommendations to help states and communities identify and define high-value credentials; determine policy-based incentives to drive attainment of those credentials; and develop strategies to collect and report on credential attainment. They define high-value non-degree credentials as those that are required or preferred for in-demand roles that pay a family-sustaining wage and that “stack” through continued education and training to additional credentialing opportunities.
In 2019, ESG built a corresponding toolkit–Building Credential Currency. It offers a start-to-finish process that can be modified and adapted to identify in-demand, high-skill, family-sustaining wage occupations and the credentials associated with them; validates the importance of those credentials with their employer community; and incentivizes learner attainment.
Since then, ESG has led its Credentials of Value Institute to help Florida, North Carolina, Massachusetts, Kentucky, Washington and Ohio define and identify high-value credentials. It also helped the Hawaii Chamber of Commerce and Hawaii P-20 identify priority credentials, which they’re using to better align to promising credentials and re-envision their data systems to track and bolster equitable credential attainment.
