FacebookTwitterEmail

Young people engage with multiple institutions and organizations as they grow up. Most youth attend school and interact with public and private health and recreational organizations; many youth connect with social service institutions of various stripes; some youth find themselves involved with the criminal justice system. However, these diverse institutions rarely coordinate their service strategies or goals with one another to better serve youth, and because of institutional silos, they often fail to capitalize on the close interrelationships among youth’s social, emotional, cognitive, and physical development. As a result, they fail to provide the “web of support” policymakers and practitioners intend. This institutional isolation can also lead to misallocated funding and disappointing outcomes because services do not cover all youth needs or are unnecessarily duplicated. Ultimately, this can result in a failure to provide the necessary investments to support positive development for all young people.

A better alternative is to foster the establishment of holistic “youth sector” collaboratives, where myriad agencies can find opportunities to work collectively toward shared goals. A shared data framework can hold these collaboratives accountable to reaching those goals. This essay examines possibilities for using shared data systems to support and sustain youth sector collaboratives through the experience of the Youth Data Archive (YDA), an initiative of the John W. Gardner Center for Youth and Their Communities at Stanford University. The YDA links data from schools, public agencies, and community-based organizations to help ask and answer key questions about the youth the partnering organizations serve. Participating agencies collectively identify shared questions that no single agency can answer alone. Ultimately, the YDA supports partners in understanding the resulting analyses and in making data-driven policy and program decisions to improve outcomes for youth.

What Is a Youth Sector?

A youth sector includes all the public and private organizations addressing a complex youth-related issue, such as child and infant health, social and emotional development, or college access. Some communities recognize how the missions of these agencies overlap and look beyond single institutions or existing policy systems to identify resources and relationships that will promote common goals for youth across sectors (e.g., health and education). In most communities, however, much of the work often happens in silos, with each agency independently pursuing its own goals with little knowledge of the work of other agencies serving some of the same children and youth. For instance, schools and district personnel are charged with the difficult task of educating all students and meeting federal and state standards to show their progress. They operate under shrinking budgets and unfunded mandates, leaving little room to focus on anything other than educational outcomes. Yet, research shows—and most educators know—that healthy students are more successful students. By participating in a youth sector collaborative, a school district could access information, and possibly resources, from partners to achieve both health and education goals. A collaborative approach reframes the policy space from a set of disconnected institutional policies and programs, each focused on a specific aspect of youth policy, to an approach in which youth initiatives are collectively developed, implemented, and evaluated. By considering how various service strategies are mutually reinforcing, youth sector initiatives can invest in youth development more effectively and efficiently. In addition, a youth-sector frame supports communities in tackling complex and entrenched social issues that no institution can address alone.

At their core, youth sector initiatives require sustained cross-sector collaboration undergirded by data sharing and strategic alignment, but can look different depending on a collaborative’s particular goals. The Promise Neighborhoods initiative, for instance, funded by the U.S. Department of Education and based on the Harlem Children’s Zone project, promotes coordination among youth-serving organizations from many sectors. The initiative has a strong commitment to using real-time data to measure progress and hold partners accountable to the program goals of improving educational and developmental outcomes of children and youth in distressed communities, and in the process transforming those communities.[1] Locally, many communities are forming their own youth sector collaboratives. For instance, Redwood City 2020 is a collaboration among multiple city and county agencies, school districts, nonprofit organizations, and business partners focused on improving youth outcomes through data-sharing, collaborative planning, and joint funding.[2] One scaffolding for these types of collaboratives is “collective impact,” which emphasizes shared measurement as a way to move partners toward common goals.[3]

Creating an Effective Youth Sector Collaborative

Although collaboration and collective action among youth-serving institutions makes intuitive sense in terms of maximizing a very limited set of resources available to serve youth, in practice, this is not an easy undertaking. Collaboration among institutions within a single sector (such as health or education), let alone among sectors, has been notoriously difficult to achieve or sustain. Competition for scarce resources, different models for addressing problems, and incompatible policy frames and accountability requirements can often interfere.[4] Similarly, differing financial resources, personnel or time constraints, or lack of political or community support can frustrate collaboration and threaten the trust-building that is essential to a cohesive youth sector.

Experience suggests that meaningful and sustained collaboration depends on at least four factors. The first factor is time. It takes time for partners to establish the relationships, trust, and new routines fundamental to this approach. A second factor is a shared sense of urgency to address a specific problem, to think beyond business-as-usual responses. Successful collaborations are forged when a critical concern—such as poor educational outcomes or increased youth violence—motivates joint action.

A third factor is the presence of an independent, neutral convener. This entity provides stability in the face of institutional churn and other challenges, and offers a neutral stance on data and findings. Such conveners also function as collective capacity builders. They manage the opportunities and issues that emerge in a collaborative setting and integrate knowledge and resources among participants.

The final requirement essential to success is integrated data; that is, data collected by each participating organization and linked across agencies. An integrated database can foster a shared mission and collective responsibility while providing a reason to continue collaborating. By themselves, good will and a shared goal of improving youth outcomes are insufficient to create the new relationships required to engage in joint action to tackle complex issues. Also needed are an agreed-upon set of outcomes and cross-institutional data to inform both the urgency and the outcomes of the collaboration. Promise Neighborhoods initiatives, for example, are required to implement a shared service-provider data system to allow providers to see the services used by participating families across service providers. Likewise, the Next Generation Afterschool System Building Initiative supports the creation of citywide data systems to track program participation and improve quality.[5] Linking data across institutions allows actors and agencies from different sectors to consider youth outcomes in broad terms, align investments, and identify shortfalls. Shared data become the glue that binds and deepens the relationships among partner organizations.

The Youth Data Archive: Using Shared Data to Support Common Goals

The Youth Data Archive (YDA) houses longitudinal data collected by public and nonprofit youth-serving agencies and organizations in a community, harnessed by partners to ask and answer questions that no one agency could answer alone.[6] At its core, the archive creates actionable knowledge that communities can use to support improved policies and practices for all children and youth. The YDA contains data from a wide range of agencies, including data such as student attendance and test scores, afterschool program participation, child welfare involvement, health indicators, and surveys on youth attitudes. It currently covers three California counties (San Mateo, Orange, and San Francisco) and is in development in Santa Clara and Alameda counties.

Researchers at the Gardner Center work with community partners to obtain and link data, which are maintained on a secure server at Stanford University. Partners retain ownership of their data and give consent for their data to be used for analysis. They must also approve any resulting reports or publications before they are disseminated. The YDA supports only research; partners do not have access to other organizations’ data for case management or service provision.

Collaboratives that have used the YDA vary considerably, and many of the efforts have included broad partnerships among county health and human services agencies, county First 5 Commissions,[7] school districts, community college districts, city police and parks and recreation departments, city mayor’s offices, afterschool providers, providers of counseling services, and others.[8] In one project, multiple school districts and the county human services agency engaged the YDA to examine the causes, consequences, and correlates of truancy and chronic absenteeism in an effort to help affected students. In another project, the County Office of Education, First 5 Commission, and a local school district partnered to understand how student participation in Preschool for All programs supported their transition to elementary school, with particular attention to the needs of low-income and minority students. County and district personnel use the research to support subsidized high-quality preschool and the alignment of preschool and elementary school standards and goals.

At the heart of the YDA is a university-community partnership, but this arrangement is not essential to its success. What is essential is identifying a host agency with the capacity and infrastructure to support the storage and analysis of shared data. In the YDA case, the data analysts have a place at the table when partners are building their capacity to understand and interpret the data. The researchers are neutral supporters of the collaborative, which can be particularly helpful in contexts where partner organizations are more accustomed to blaming one another than working together.

Creating a Data-Supported Youth Sector Collaborative

Developing an archive of shared data that can inform cross-institutional action is not necessarily a linear process, but conditions build on each other in mutually reinforcing, or inhibiting, ways. Six factors help ensure success.

Establishing Trust

Establishing trust among partners is critical.[9] Previous competition among partners for scarce community resources, lack of experience with cross-sector interactions, and concerns that someone outside the organization could misuse or misreport data can derail any data-sharing efforts. The YDA addresses some of these concerns by using written agreements that data will not be shared with any other parties and that findings will be released only with permission from all who have contributed data. These assurances make partners more comfortable with sharing data and less concerned that they will be taken by surprise in seeing findings from their own data presented in a public forum.

Strong Leadership

Strong, committed leadership is also key to success. Agency heads unaccustomed to thinking of their work in the context of the broader youth sector will need to become comfortable with new norms of practice and organizational relationships. Although members of a youth sector collaborative will have committed to a joint agenda for supporting children and youth, they are still individually accountable for specific outcomes to funders and governing agencies. Negotiating this dual responsibility requires the agency leader to understand the importance of placing his or her work in community context and extending this understanding to operations of the organization.

Establishing a Shared Research Agenda

In coming to agreement on a research agenda, a key step involves selecting shared outcomes to track. When community organizations come together for common purpose, it is often to achieve a lofty goal, such as reducing childhood obesity or improving graduation rates. The factors underlying such problems are highly complex and not rooted in the system charged with addressing the problem (e.g., a health department or school district), and thus are ripe for a cross-sector approach. However, rates of obesity or high school graduation change slowly, and even if the community is making some progress in addressing the problem, it may take a decade or longer before success is evident. It is important, then, to consider progress on interim outcomes. These can include tracking whether communities are providing opportunities for youth to engage in physical activities and access healthy foods, or creating culturally responsive school environments that enable and encourage youth to stay in school.

Even with shared outcomes in mind, agreeing on research questions can be challenging for agency heads who are accustomed to asking questions about their own programs or populations but not about how those programs or populations intersect with others or about opportunities for joint approaches to complex problems. A good research agenda should have three characteristics:[10]

  • Aligned: Does the question represent or support a core goal of the collaborative?
  • Answerable: Can the question be answered with data that partners collect?
  • Actionable: Will partners be able to take action when they have results?

Questions should first be aligned with the mission of the collaborative. If the mission is to improve college attendance rates for inner-city high school students, the collaborative should specifically consider the process leading to high school completion and college enrollment, as well as students’ early college experiences. This may require educational leaders focused on K-12 to extend their traditional commitment to students beyond graduation and postsecondary leaders to consider factors that influence student success even before they enter college. Data could be combined from various agencies to understand students’ supports and experiences during high school, although it is not essential that data from every collaborative member be used in each analysis.

The questions asked must also be answerable with data collected by contributing organizations. Administrative data—data collected by agencies or service providers as part of their daily operations —help agencies track the number of youth they serve, the types of services received, or selected outcomes. Organizations may collect information on young people’s experiences to comply with state and federal government regulations or to satisfy foundation requirements. However, administrative data may not capture the measures most appropriate for a particular analysis. For instance, many agencies are focused on social and emotional development. However, no single method for measuring social and emotional development exists, and partners are most likely not collecting this information routinely. If the collaborative determines that providing positive developmental experiences is a shared goal, agency partners will need to find or develop new metrics or use different types of data to measure current conditions and progress.

Answers to partners’ questions may require other types of data collection, such as student or parent surveys, interviews, or focus groups. Parent or survey data that include identifiers can be linked to other administrative records. In contrast, interview or focus group data, which may involve few respondents or group-level data (in the case of focus groups) would not be integrated into the overall database but can still be important to understanding questions about program implementation or respondents’ perceptions.

The most challenging of these three criteria is whether the research is actionable. How will partners know in advance whether the findings will lead them to act? There is no way to know in advance whether the findings will help leaders to act, but if the analysis is relevant to the decision-making process, even a decision for the status quo may be ‘action.’ The YDA research team has adopted a definition of action that takes many forms along a continuum, including any change in policy, practice, or programming; continuing with existing efforts; or the intention to use research to discuss making changes or continuing with the status quo.[11] Partners may take action on a given set of findings immediately or in the future. Several YDA analyses have gained the attention of state and local policymakers, which broadens the potential for action beyond the contributing partners’ use of the findings.

Building Capacity

The process of establishing a youth sector collaborative and sharing data to support a shared mission creates opportunities to build capacity for the partners to use cross-sector data for their shared endeavor. It also allows partners to identify gaps in their own data collection or in the entire collaborative’s data collection process. Collaborators can build on the strengths of existing data to collect more robust and helpful information about the youth they jointly serve.

Asking cross-sector questions across agencies in support of positive youth outcomes is new for many agencies. Agreeing on a shared research agenda with common research questions, using data to respond to these questions, and coming together to make decisions in support of youth based on the findings can be, at first, challenging for partner organizations and may require a research partner. As agency partners become more familiar with these processes, they will be able to ask and answer increasingly more relevant and complex questions.    

Sharing Data

Establishing a shared data archive also requires collaborating partners to: (1) reliably collect the necessary data in a useable format; (2) agree to share information among agencies and sectors for common purposes; (3) have a technological platform for sharing information either with one another or an external partner; (4) understand and abide by federal and state regulations that govern the sharing of confidential data; and (5) agree on a common agenda for accessing and using the data to make changes to policy and practice.[12]

However, many community organizations do not have sufficient financial resources or time to accomplish these steps. It is a leap of faith to believe that with shared goals and actions come pooled resources and support, yet organizations need to make this leap. In some cases, external funders can help (e.g., Promise Neighborhoods). In other cases, collaboratives may be able to harness seed funding for their work by offering a voice at the table to local businesses, community foundations, health care foundations, and others. They can write joint funding proposals or find funds within each organization to support ongoing work. These are not simple solutions, and funding for this type of work is not easily won. In the case of the YDA, funding comes from various sources, including partners’ own contributions and grants from external funders, and resources from the Gardner Center itself.

Another common impediment to success is that many community agencies have not invested in collecting high-quality, consistent, and wide-ranging data on the youth they serve. With missing or inaccurate data on student names, birth dates, and addresses, it is difficult to link individual records from disparate databases. Additionally, if agencies have not populated their database with the kinds of information needed to answer the questions generated, the analysis will stall out. For instance, if collaborative partners want to understand the effects of regular afterschool program participation on students’ academic achievement, the participating afterschool provider would need to keep ongoing records on students’ attendance in its program. Yet many providers do not keep these types of records. Participation in a collaborative provides an opportunity for organizations to learn about others’ data collection and improve and expand their own data in the process.

Agreements for how to share and store data, all the while ensuring compliance with regulations on data security and privacy, are critical to a cross-institutional data-sharing initiative. Agreements to share data require consideration of who may access the data and how permissions to use and report the data will be managed. As mentioned earlier, the YDA operates on the principle that contributing agencies retain full ownership of their data. Partners report that this aspect of the YDA is critical for them to feel comfortable contributing their confidential data. Data security is also essential. The YDA’s data platform meets the highest standards for secure data (e.g., not linked to the internet, only accessible from encrypted computers) and is operated and maintained by Stanford University. Other options include using a host partner’s own server or purchasing external server space to store the data. The process of linking and analyzing data requires technical expertise, which may be available in-house at partner agencies.

Privacy and other legal restrictions pose another difficulty. Educational organizations, for example, are governed by the Family Educational Rights Privacy Act (FERPA), and health organizations are guided by the Health Insurance Portability and Accountability Act (HIPAA). The restrictions imposed by these or other regulations can be difficult to navigate, so partners should consider their options carefully when deciding how to proceed. Use of external research partners (or neutral third parties) can aid efforts considerably because regulations often allow for data exchange for research purposes if individual-level information is not shared publicly.[13]

Measuring Progress

How do partners in the youth sector know that their collaboration is making a difference for children and youth—and, ultimately, for the community? Progress can take time and can be difficult to assess. A key marker of success is the type of action—considerations or changes to policy or practice—that results from collective planning. Tracking the action resulting from the youth sector collaborative is critical, but our experience with the YDA suggests that community partners may not be able to act on findings immediately. The release of findings may not be timed to partners’ decision-making cycles, acting may require additional funds, or partners may simply be busy with other pressing priorities. But there is no statute of limitations on action; partners may refer to research that is several years old to advance an agenda that was not possible when the research was performed.

Conclusion

We have made the case that collaboration among the youth-serving organizations in different sectors creates opportunities for enhancing the policies that govern programs and services for young people. The collaboration among typically siloed agencies can be greatly enhanced by shared data, allowing the collaboration members to examine and build on the synergies that exist among their varied efforts.

Research conducted using shared data allows members of a collaborative to see how well their service strategies align with those of other members in the community to support positive outcomes, or how they fall short of goals. It also allows collaborative members to see how families take advantage of multiple programs to support individual children and youth.

Private and public funders are beginning to recognize the importance of establishing youth sector collaboratives. To succeed, these collaboratives will need to embrace the essential role of linked, cross-institutional data in supporting their goals. They will need to build internal capacity for asking cross-agency questions, use their data in new ways to answer them, and commit to using the analysis findings to create policy and programmatic improvements in support of positive youth outcomes.

[1]   U.S. Department of Education, “Promise Neighborhoods and the Urban Institute: Measuring Performance—A Guidance Document for Promise Neighborhoods on Collecting Data and Reporting Results” (Washington, DC: US Department of Education, 2014), www2.ed.gov/programs/promiseneighborhoods/index.html.

[2]   Redwood City 2020. www.rwc2020.org/.

[3]   J. Kania and M. Kramer, “Collective Impact,” Stanford Social Innovation Review (Winter 2011). www.ssireview.org/articles/entry/collective_impact.

[4]   R. Friedland and R. Alford, “Bringing Society Back In: Symbols, Practices and Institutional Contradictions.” In The New Institutionalism in Organizational Analysis, edited by W. Powell and P. Dimaggio (Chicago: University of Chicago Press, 1991); E. Weber and A. Khademian, “Wicked Problems, Knowledge Challenges, and Collaborative Capacity Builders in Network Settings,” Public Administration Review 68 (2) (2008): 334−49.

[5]   Next Generation initiative is a Wallace Foundation effort to strengthen systems for coordinating local afterschool opportunities in grantee cities. See National League of Cities, “Communities Learning in Partnership”: http://www.nlc.org/find-city-solutions/institute-for-youth-education-and-families/education/higher-education/communities-learning-in-partnership.

[6]   M. McLaughlin and R. London, eds., From Data to Action: A Community Approach to Improving Youth Outcomes (Cambridge, MA: Harvard Education Press, 2013).

[7]   First 5 California distributes funds through county First 5 County Commissions to support education, health services, childcare, and other programs for children ages 0 to 5 and their parents and caregivers. See http://www.ccfc.ca.gov/.

[8]   See John W. Gardner Center, “The Youth Data Archive,” http://gardnercenter.stanford.edu/our_work/yda.html.

[9]   I. Nelson, R. London, and K. Strobel, “Reinventing the Role of the University Researcher” (Stanford, CA: John W. Gardner Center for Youth and Their Communities, 2013).

[10]  John W. Gardner Center for Youth and Their Communities, “From Data Collection to Data Utilization: Presentation to St. Paul Next Generation Afterschool Initiative” (Stanford, CA: John W. Gardner Center, 2013).

[11]  K. Dukakis and R. London, “What Makes the Youth Data Archive Actionable?” In From Data to Action: A Community Approach to Improving Youth Outcomes, edited by M. McLaughlin and R. London (Cambridge, MA: Harvard Education Press, 2013).

[12]  McLaughlin and London, From Data to Action.

[13]  See McLaughlin and London, From Data to Action, appendix 1, for more information.