Category Archives: Network Evaluation

Network Evaluation , The Network Building Field

Network Evolution Assessment

May 6, 2015 Peter Plastrik

When Maggie Ullman was asked to assess the condition of eight regional networks of city government sustainability directors around the U.S., we designed an assessment framework that would allow her to map each network’s evolution and also put the networks side by side to form a composite picture of their condition. The assessment, based on interviews, survey data, observations, and written materials, has two axes:

Stages of Evolution –from Emerging to Developing to Near Mature to Mature.
Network Conditions –Connectivity, Leadership, Activities, Communications, Coordination, Member Satisfaction, and Resources.

In each Stage the specifics of each Condition are different, they evolve, as can be seen in the matrix below. Note that the specific “rubric” in each cell is somewhat tailored to the type of networks that were being assessed–but you can adjust these for your own use.

NETWORK ASSESSMENT
STAGE	Connectivity	Leadership	Activity	Communication	Coordination	Member Satisfaction	Resources
Emerging	Small hub of 3-5 connected members	Two or more committed founders	Members connect and exchange information	Face to face meetings with no communications between meetings	Ad hoc capacity by small group of member volunteers	Most members see opportunity but network not yet delivering	No and/or in-kind resources for network activities
Developing	Larger hub of 6-10 strongly connected members	Established steering committee (SC) of 3-5 members	Members align and spread shared ideas such as policy or project needs	Informal communication between face to face meetings between some members	Coordination by member volunteers and minimal third party support	Network beginning to deliver satisfaction for hub of group	Minimal and/or in-kind resources for annual meeting or network coordination or collaborations
Near Mature	Multiple hubs of strongly connected members	SC of 6+ members who create annual strategic plans	Members collaboration on their first project	Formal network wide communication between face to face meetings	Coordination by third party	Network delivering satisfaction for most members	Resources for annual meeting, network coordination, and member collaborations
Mature	Multiple hubs that intertwine of 20+ strongly connected	Second generation of SC leaders who create annual strategic plans	Members collaborate and produce outcomes on multiple projects	Multiple, network wide communications between face to face meetings with high member participation	Consistent, effective coordination by third party	Broad, high level of member satisfaction by a majority of the members	Consistent resources for annual meeting, network coordination, and member collaborations

Network Evaluation

Watching Networks Evolve

May 4, 2015 Peter Plastrik

We emphasized in Connecting that networks are dynamic and evolve, sometimes rapidly, in several dimensions. The structure of their members’ connections and their members’ value propositions may evolve. Their capabilities may evolve, from connecting people to aligning them to helping them collaborate on producing various outputs. Whether a network is evolving and how it is evolving is one way of understanding what the network’s condition and potential are.

So when Maggie Ullman was asked to assess the condition of eight regional networks of city government sustainability directors around the U.S., we designed an assessment framework that would allow her to map each network’s evolution and also put the networks side by side to form a composite picture of their condition. This assessment, recently completed and based on interviews, observations, and written materials, is becoming the basis for customized support for the networks and new philanthropic funding to help them further evolve and generate greater impact.

The assessment framework has two axes:

Stages of Evolution –from Emerging to Developing to Near Mature to Mature.
Network Conditions –Connectivity, Leadership, Activities, Communications, Coordination, Member Satisfaction, and Resources.

In each Stage the specifics of each Condition are different, they evolve. For instance:

Leadership evolves from “Two or more committed founders” (Emerging Stage) to “Second generation of leaders who create annual strategic plans” (Mature Stage).
Member Satisfaction evolves from “Most members see opportunity but network not yet delivering.” (Emerging Stage) to “Broad, high level of member satisfaction by a majority of members” (Mature Stage).

When Maggie assembled a composite picture of the eight networks using the assessment matrix, it showed quite clearly that (a) mostly the networks were in the Developing Stage on nearly all of the conditions, except Activities and Communication, and that they were lagging in the Resources condition, “stuck” as it were in Emerging and Developing Stages. This mapping resulted in a set of recommendations that Maggie developed for the networks.

Here’s the Network Evolution Assessment tool.

Network Evaluation

Profiling a Network

October 27, 2014 John Cleveland

We’re often asked to check out an existing social-impact network and to provide advice about what its next steps might be. Sometimes a potential funder wants to know this, sometimes network leaders are asking for the assessment.

In our book’s chapter 6, “Know Your Condition: Taking a Network’s Pulse,” we describe the key elements of an in-depth assessment: understanding the network’s connectivity, health, and impact. But often a network doesn’t have much of the data that would help you to conduct such a probing evaluation, or what’s being asked for is a “quick (and cheap)” assessment.

One tool we’ve been developing is a “Network Profile”—the information you’d gather step-by-step that would help you to take a “snap shot” of a network’s situation and start a good dialogue with the network’s leadership. We think the Profile can be especially useful for a funder who’s looking at multiple networks and trying to figure out which ones to invest in.

Category	Information	Details
*Network Background*	Network Name	Formal name of the network, if it has one
	Origins	How the network came to be formed Who the original network organizers were What motivated them to create the network
	Start	What year the network started
	Stage of Development	Emerging Developing Mature
	Network Type	Trade Association Professional Association Peer Learning Network Alliance Campaign
*Network Description*	Purpose	What is the Network’s reason for being?
	Membership	Who is eligible to be a member? What are the requirements of participation, once you are a member? How many members are there? Get a membership list. What is the desired end size of the network?
	Value Propositions	What are the benefits of membership? What is the collective value proposition? Are there specific projects or initiatives the network is sponsoring? (Get descriptive materials.)
	Operating Principles	Are there specific principles/rules that guide the network’s culture?
	Network Coordination	Who manages the network? How does the network support interaction between members? What are the dominant forms of interaction? o Connecting o Aligning o Producing
	Resources	What is the network’s funding model?
	Governance	Who decides what the network will do? How do they make decisions?
	External Relationships	Are there specific organizations with which the network maintains strategic relationships? Who manages these relationships?
	Assessment	How does the network monitor its condition and performance?
*Network Opportunities*	Market Positioning	How does this network fit into the overall field of urban climate resilience? Are there specific aspects of field development it can most contribute to?
	Projects	Are there specific projects that the network is seeking support on?
	Partnerships	Are there opportunities to connect this network with other networks to maximize impact?

Network Evaluation

How Do You Know When You Have a “Generative” Network?

October 8, 2014 John Cleveland

Our book is focused on the design, development and management of “generative social impact networks”. It is pretty easy to know if a network is focused on social impact or not – their mission, values, and activities tell you whether the strategy is targeted on social good or some other kind of outcome. But figuring out whether it has the genetics of a “generative” network is much more difficult.

We are engaged in a current project with a national foundation to do a network mapping exercise for them in a few targeted areas of urban sustainability. Part of the question they are trying to answer is: where are the generative networks that can be supported to advance this field? We started with a list of over 100 networks and have been narrowing down the list. Since we have worked with some many “generative” networks, we assumed that it would be easy to tell which fell into that category – we had the “I’ll know it when I see it” attitude. But it has been much more difficult than we expected.

There are some types of networks that are pretty easy to screen out – your traditional trade association, professional association and issue campaigns as an example. As well as the “group activities masquerading as a network” – the NGO-driven distribution systems with relatively little member-to-member interaction. But beyond this, it turns out you need to know a fair amount about the network to figure out which ones have generative genetics. So we created a list of criteria for the analysis. This is our working list at this point in time:

Shared purpose. Members share a collective purpose that is focused on creating social good.
Complex problem. The problem/opportunity being focused on is a complex one with no simple solutions and the network is committed to working on it from multiple angles.
Member-driven. Members set the mission and goals, and “make the rules” and members do most of the work.
Relationship-focused. Members develop deep and enduring relationships with each other; the focus of relationship building is member to member, not staff or other provides to members
Flexible. The structure is flexible and adaptable and changes as needed.
Internal markets for value. Value is determined by whether members engage in the activity or not – members “vote with their feet.”
Decentralized. Decisions are highly decentralized.

Ultimately, it all boils down to understanding the member experience in the networks – what the culture of the network is. (OK, so the Collins and Porras “Built to Last” hypothesis applies here too!) And for that, eventually you have talk to some of the members.

Network Evaluation

What Makes Networks Generative?

September 30, 2014 Madeleine Taylor

Connecting to Change the World builds on an earlier resource that Pete and I developed called Net Gains. This latest collaboration with John Cleveland includes examples and lessons that have emerged from our work with social impact networks over the last decade or so. During that time, we’ve been introduced to many new networks and deepened our work with others. As a consequence, we have a better understanding of what makes some networks highly “generative.” By generative, we mean networks with a renewable collaborative capacity to generate numerous activities simultaneously. These are networks that activate members’ connections on an emergent basis as need and opportunities arise. Examples in the book include RE AMP – more than 165 nonprofit organizations and foundations in eight Midwestern states working together on climate change and energy policies, Reboot- a network of young Jewish American “cultural creatives” who are exploring and redefining Jewish identify and community in the U.S. and the United Kingdom, ten regional networks of state agencies and nonprofit providers that have organized to end homelessness in Massachusetts, and five regional and two national networks of rural-based organizations that are promoting public policies that benefit rural communities in the U.S. In all of these networks, members have been very deliberate about creating, strengthening and maintaining network ties in order to establish a base of connections from which many activities can arise at the same time or over time. This foundation is the starting point for the progression from connecting to aligning to production or joint action that we also discuss in the book.

Network Evaluation

Evaluating Networks

August 13, 2014 Madeleine Taylor

Changemakers everywhere are investing in and building new networks—from large-scale, cross-sector, “collective impact” initiatives to more targeted efforts to promote policy change, spark social innovation, or improve the delivery of services. All of this activity has made evaluation a priority for funders and practitioners who want to learn more about what networks can achieve and how to build networks more effectively. Network assessment—systematic, practical, and focused on a network’s unique capabilities—is coming into its own.

In response to increasing interest from grantmakers, network builders, and evaluators, Network Impact partnered with the Center for Evaluation Innovation (CEI) to develop a “State of Network Evaluation” framing paper about network evaluation and an accompanying network evaluation casebook, as well as a convening for funders to discuss practical questions about designing and funding network assessments. Supported by the David and Lucile Packard Foundation, the W.K. Kellogg Foundation, the Jim Joseph Foundation, and The California Endowment, our three-pronged network evaluation project aimed to gather and share information with evaluators and leaders in the network development field about evaluation approaches, methodologies, and tools that are particularly suited to the assessment of networks.

We began the project with an extensive document and literature review of different types of networks and their evaluations, both domestically and internationally, as well as interviews with funders, network practitioners, and evaluation experts. This led us to some of the real-life examples of funder-supported evaluations that we’ve included in the completed casebook “Evaluating Networks for Social Change: A Casebook.” Producing a casebook with lots of “how to” detail was a priority of the project: it’s one thing to have a good general understanding of how to design a network evaluation and another to gather the necessary resources and expertise to implement one on the ground.

Four of the networks in the casebook—Reboot, Urban Sustainability Directors Network, RE-AMP and Massachusetts Networks to End Homelessness—also appear in Connecting to Change the World: Harnessing the Power of Networks for Social Impact , a new book co-authored with Peter Plastrik and John Cleveland, that goes deeper on network development strategy, with more information about how to build and manage high-performing social impact networks.

Three Pillars

The nine networks profiled in the casebook represent a variety of network types (networks at different stages of development, of different sizes and issue focus), illustrate a range of network evaluation methodologies, and are organized to reflect three basic areas of focus for a network evaluation: network connectivity, network health, and network results. In our state of the field framing paper, we call these the Three Pillars of Network Evaluation and offer guidance on the types of evaluation questions that can be pursued under each of the pillars and their sub-dimensions. A single evaluation may focus on one or more of the pillars at the same time, and several evaluations in the casebook do.

The framing paper also includes a guide to evaluation questions that come up at different stages in a network’s development. We built this guide based on a resource for funders created by the Monitor Institute.

What’s Next?

The collaboration between Network Impact and the Center for Evaluation Innovation is continuing as part of our shared commitment to grow the field and increase the practice of network evaluation. In coming months, we plan to explore the intersection of network evaluation and systems mapping and look in greater detail at the evaluation of informal (small “n”) networks. This was an area for further investigation identified by funders at the April 2014 network evaluation convening: how to assess the progress of networks that many grantmakers hope to catalyze by convening or connecting their grantees.