Knowledge Co-production in Contested Spaces : An Evaluation of the North Slope Borough – Shell Baseline Studies Program

Supporting the development of trusted and usable science remains a key challenge in contested spaces. This paper evaluates a collaborative research agreement between the North Slope Borough of Alaska and Shell Exploration and Production Company—an agreement that was designed to improve collection of information and management of issues associated with the potential impacts of oil and gas development in the Arctic. The evaluation is based on six categories of knowledge co-production indicators: external factors, inputs, processes, outputs, outcomes, and impacts. Two sources of data were used to assess the indicators: interviews with steering committee members and external science managers (n = 16) and a review of steering committee minutes. Interpretation of the output and outcome indicators suggests that the Baseline Studies Program supported a broad range of research, though there were differences in how groups perceived the relevance and legitimacy of project outcomes. Several input, process, and external variables enabled the co-production of trusted science in an emergent boundary organization and contested space; these variables included governance arrangements, leveraged capacities, and the inclusion of traditional knowledge. Challenges to knowledge co-production on the North Slope include logistics, differences in cultures and decision contexts, and balancing trade-offs among perceived credibility, legitimacy, and relevance. Reinforced lessons learned included providing time to foster trust, developing adaptive governance approaches, and building capacity among scientists to translate community concerns into research questions.


INTRODUCTION
The risk governance of socio-ecological systems is increasingly characterized by complexity and uncertainty (van Asselt and Renn, 2011).Developing place-based processes and institutions to coordinate the co-production of relevant, salient, and credible knowledge for riskbased decisions remains a core challenge for science policy research, especially within contested spaces (NASEM, 2016).Such processes require communication and deliberation among a host of individuals involved in research and decision making about risks, including representatives from multiple sectors and levels of governance, who often have disparate values and epistemologies (Ostrom, 2010).
Risk governance is particularly challenging in the Arctic where decision makers-including those involved in oil and gas development, coastal planning, and subsistence-have particular priorities, needs, and objectives (USCG, 2013;Huebert, 2016).Decision makers require information on baseline conditions and cumulative effects of development on marine resources and human health; however, there are significant data gaps that impede decision making for coastal communities and industry (Knapp and Trainor, 2015).Historical legacies also complicate the production of trusted and legitimate science; these include the disregard for traditional knowledge and limited opportunities for local stakeholders to shape research agendas (NRC, 2014).Further, some research is perceived to be tied to vested interests and public policy preferences (Oreskes, 2004).Understanding how to support the co-production of trusted and relevant science in ways that support development in the Arctic without compromising Indigenous cultures and environments remains a key priority (Clement et al., 2013).
This study evaluates the Baseline Studies Program, an organization created by the North Slope Borough of Alaska and Shell Exploration and Production Company to improve the collection of information and management of issues associated with the potential impacts of oil and gas development on marine ecosystems and coastal communities.The evaluation seeks to understand what factors contributed to knowledge co-production within an emergent boundary organization in the context of a contested political environment.This research is grounded in an understanding of the Baseline Studies Program as a boundary organization that exists at the frontier of science and decision making.

KNOWLEDGE CO-PRODUCTION AND BOUNDARY ORGANIZATIONS
Recommended approaches for creating usable science for decision making increasingly highlight the need for collaboration with stakeholders (Moss et al., 2014).Several terms are used to describe these collaborative approaches, including knowledge co-production, user-driven science, actionable science, and transdisciplinary research (Dilling and Lemos, 2011;Jahn et al., 2012;Beier et al., 2017).This paper uses the term knowledge co-production to refer to research approaches that are highly collaborative and community oriented (Bremer and Meisch, 2017;Miller and Wyborn, in press).Knowledge co-production is defined here as the process of creating more usable science through iterative interactions between researchers and stakeholders.
The concept of boundary organizations has emerged as a cross-cutting theme within knowledge co-production (Bremer and Meisch, 2017).Scholarship on boundary organizations is rooted in the demarcation of science and a desire to separate science from non-scientific activities and protect research against undue political interference (Popper, 1965;Gieryn, 1983).Boundary organizations have three defining features: involvement of individuals across the science-practice interface, production of boundary objects that provide a common currency for communication, and existence at the nexus of science and practice with distinct lines of responsibility (Guston, 2001).The third feature is based on principal-agent theory, whereby boundary organizations can mediate conflict among principals and agents who are involved in the production and use of science, respectively (Eisenhardt, 1989).Since the term was coined, the concept of boundary organizations has expanded to include diverse organizations, including co-management boards such as the Intergovernmental Panel on Climate Change and the Arctic Council (Berkes, 2009;Clark et al., 2016).
There are three streams of research on boundary organizations.One stream focuses on how individual boundary organizations can improve the production of credible, salient, and legitimate information (Cash et al., 2003).A second stream focuses on how to improve boundary spanning processes and the transfer of knowledge to action (Buizer et al., 2016).Central to these processes are iterative, sustained, and interactive forms of communication to improve knowledge usability (Lemos et al., 2012).A third stream focuses on how networking among boundary organizations increases the usability of research more efficiently (Lemos et al., 2014).These streams highlight multiple roles and functions of boundary organizations, including brokering and facilitating the co-production of knowledge, translating information, convening dialogue, and providing a space for communication, conf lict resolution, and knowledge integration (Armitage et al., 2015;Buizer et al., 2016).These functions are more effective when governance arrangements facilitate shared accountability across the boundary (Clark et al., 2016).
Boundary organizations may be especially effective in producing usable science in politically contested spaces, where there are polarizations of power, high commitments to value and knowledge claims, and histories of antagonism and distrust (Bridge, 2004;O'Mahony and Bechky, 2008;Armitage et al., 2015).For example, the Health Effects Institute-a partnership between the U.S. automobile industry and the U.S. Environmental Protection Agency (EPA)-was created to support relevant and legitimate research on vehicle emissions at a time of antagonism and distrust between industry and federal regulators (NRC, 1993).Boundary objects, such as shared datasets and maps and other outputs from such partnerships, can provide a trusted currency for bargaining and deliberation (Affolderbach et al., 2012).Some research has examined the conditions under which boundary organizations form in contested spaces, including compelling circumstances and leadership, existing networks, and incentives (Affolderbach et al., 2012).Enabling success involves building legitimacy via demonstrating early accomplishments, managing future conflict, and committing resources (Guston, 2001;Leith et al., 2016).Keating (2001) suggests that the co-production of research is more likely to be successful in contested spaces when boundary organizations focus on quality control, provide competitive review processes for research proposals, and are informed by expert advisory committees on both sides of the boundary.Much of this research has occurred in industrialized countries (Clark et al., 2016).
There is a limited understanding of factors that facilitate knowledge co-production in boundary organizations within Indigenous cultures in the context of contested spaces, though an extensive literature deals with bringing together Western and traditional knowledge, knowledge exchange, and knowledge application in co-management (Berkes, 2009;Dale and Armitage, 2011;Robinson and Wallington, 2012).This literature emphasizes the importance of openness and respect for local and traditional knowledge in management decisions as a way to increase legitimacy and credibility of resolutions and demonstrates how knowledge co-production can trigger adaptation and learning (Armitage et al., 2011;Taylor and de Loe, 2012).Conditions for facilitating innovation include periods of institutional change, willingness to take risks, leadership, and time for reflexive learning (Kofinas et al., 2007).Kruse et al. (1998) find that the frequent and long-term presence of trusted scientists in villages made support for and cooperation with co-management strategies more likely.Characteristics important to creating and sustaining knowledge co-production within boundary organizations will probably vary across contexts because the specific roles, functions, and strategies used depend on the type of boundary work, sources of knowledge, and level of formalization (Crona and Parker, 2012;Clark et al., 2016;van Enst et al., 2016).
Interest in developing metrics for evaluating knowledge co-production, including boundary organizations, has increased in the past decade (NRC, 2005;Fazey et al., 2014;Meadow et al., 2015;Wall et al., 2017).A synthesis of knowledge co-production theory and metrics used by federal agencies highlighted the importance of understanding the input capacities to support co-production, the processes through which activities are undertaken, material outputs, expected and unexpected outcomes for both scientists and decision makers, and the means by which external factors (context) shape outcomes and impacts (Wall et al., 2017).Contested spaces require additional metrics to assess knowledge co-production, including conflict mediation and time to develop trust (Blackstock et al., 2007).Specific indicators (such as historical relationships among tribes, federal and state agencies, and scientists) are also required for evaluations involving rural Indigenous cultures in Alaska (Caldwell et al., 2005).Challenges in evaluating knowledge co-production include establishing attribution and assessing impacts that can take years to emerge (Blackstock et al., 2007;Bell et al., 2011).

THE NORTH SLOPE
The North Slope of Alaska is situated between the Brooks Range and the Chukchi and Beaufort Seas (Fig. 1).Approximately 9600 people live in the North Slope Borough (NSB), most of whom live in Anaktuvuk Pass, Atqasuk, Kaktovik, Nuiqsut, Point Hope, Point Lay, Utqiaġvik (Borough seat, formerly Barrow), and Wainwright (U.S. Census Bureau, 2010).The largest racial population is the Iñupiat, who rely on the local environment and family ties to sustain traditional ways of life (Kruse, 1991).Local and traditional knowledge of the environment have enabled Alaska Natives to thrive in the Arctic.
North Slope communities have changed significantly in the past century in response to military investments, environmental regulations on subsistence, oil and gas development, technological change, and climate change (Knapp and Morehouse, 1991;Chapin et al., 2014).The Alaska Native Claims Settlement Act reshaped governance by creating local and regional Native corporations, which contributed to the incorporation of Western economic structures and establishment of business relations with industry (Case and Voluck, 2012).It also led to the formation of the NSB to support Indigenous traditions, levy taxes, and enact regulations (Knapp and Morehouse, 1991).Taxes and revenues generated from oil and gas development have led to capital improvements and supported several services, including the Department of Wildlife Management (DWM), which conducts marine and terrestrial research on several species relevant for subsistence.
There are trade-offs in the development of oil and gas resources for residents of the North Slope.On one hand, oil and gas development creates risks for wildlife and marine ecosystems that North Slope residents depend on for their livelihoods (Braund and Associates, 2009).Community concerns include potential impacts of seismic drilling and air traffic on marine mammals and wildlife, toxic contamination of plants and animals, air pollution from flares, increased risks from oil spills, and a host of other social and economic impacts (NRC, 2003;AMAP, 2010).At the same time, oil and gas development provides Native corporation dividends, employment and training, and search and rescue assistance (Braund and Associates, 2009).

Applied Science on the North Slope
The earliest efforts to understand the North Slope include Iñupiat people and their predecessors, who learned, shared, and passed along traditional knowledge of the region (Chance, 2002).Early investigations of Western science include several mapping expeditions, anthropological studies, and the research associated with the first International Polar Year, which provided geophysical observations across the Arctic (Jenness, 1957;Bockstoce, 1988).The discovery of substantial economically recoverable oil reserves contributed to the development of significant research programs, infrastructure, and logistic support centered in Utqiaġvik, including the Naval Arctic Research Laboratory (NARL), which is currently operated by the Ukpeaġvik Iñupiat Corporation (UIC), UIC Science, and the Barrow Arctic Science Consortium.A wide range of agencies and organizations, oil and gas industries, and academic institutions from multiple countries have conducted applied research on the North Slope, which has provided volumes of baseline data, sustained observations, and publications on terrestrial and marine geophysical processes, Arctic ecology, and social and environmental impact assessments (NSSI, 2018).These efforts represent billions of dollars invested in research on the North Slope over the past several decades.
Several boundary organizations operating on the North Slope, such as the Arctic Landscape Conservation Cooperative and North Slope Science Initiative, aim to enhance applied research coordination among federal, state, local, and tribal governments, identify science-related information needs, facilitate communication between scientists and decision makers, and reduce duplication of research (Streever et al., 2011;NSSI, 2014).Applied science research priorities linked to management, which vary across stakeholders, include topics related to weather and climate, sea ice, permafrost and hydrology, terrestrial and marine ecosystems, subsistence, coastal erosion, and potential impacts of oil and gas development (Streever et al., 2011;Arctic LCC, 2013).Several programs fund research related to these management needs on the North Slope (NPRB, 2015;Arctic LCC, 2016).Anecdotal evidence provides some insight into the effectiveness of and lessons learned from boundary organizations in supporting knowledge co-production on the North Slope, though no program-level evaluations were identified (Blair et al., 2014).
Applied science is used for policy advocacy, regulation, and decision making on the North Slope, especially in connection with oil and gas development and subsistence (Reiss, 2012).However, North Slope communities and oil and gas industries have several concerns about the credibility and legitimacy of applied research, especially given its potential implications for oil and gas development and subsistence.For example, North Slope residents perceive some research as less credible or relevant because of its close ties to industry and limited opportunities for communities to shape funding priorities (Kintisch, 2015).This is further compounded by inadequate inclusion of traditional knowledge in past research and decisions as to the allocation of permits, though some programs are increasingly providing opportunities for dialogue and engagement (Williams, 2012;NOAA NMFS, 2013;Blair et al., 2014).Oil and gas industries likewise desire highquality science to inform operational, strategic, and planning decisions.

The North Slope Borough-Shell Baseline Studies Program
Shell signaled a major return to Alaska in 2005 when the company purchased 137 offshore leases in the Beaufort Sea.In 2008, Shell successfully bid on 275 leases in the Chukchi Sea with the intent to drill exploration wells.Exploration and development activities require understanding Arctic ecosystems to operate safely, minimize potential impacts, and obtain and comply with permits.Shell also needed to regain trust, as previous drilling activities and plans were not adapted adequately to address community concerns (Reiss, 2012).
In 2007, Mayor Itta of the NSB had to decide whether or not to join litigation against Shell's offshore drilling in the Arctic.Support for drilling was nearly split among NSB residents.Itta recognized the significant potential economic benefits to the NSB and shareholders from local and regional Native corporations; he desired more precautionary standards for oil and gas development (Reiss, 2012).As part of his negotiation strategy, Itta wrote Shell to indicate that the NSB would not oppose offshore drilling if Shell could commit resources to fund an applied research program in Utqiaġvik.Shell was interested in the idea but unable to negotiate an agreement on short notice, and the NSB joined the litigation.Over the next two years Shell and the NSB negotiated the structure and function of a shared science program.During this time, the NSB passed a resolution to establish a proactive policy opposing conventional offshore oil and gas exploration and support actions that ensure responsible development (NSB, 2008).This resolution emphasized the importance of subsistence activities, the high sensitivity of the Arctic marine environment to disruption, and the need for baseline research to inform planning, mitigation, and advocacy efforts.
In 2010 the NSB and Shell entered into a collaborative research agreement to advance baseline studies research on Arctic ecosystems relevant to understanding the impacts of oil and gas exploration on marine ecosystems and resources that are linked to Iñupiat culture (NSB, 2010).This agreement provided the foundation for the NSB-Shell Baseline Studies Program (BSP), which was later renamed the Collaborative Alaskan Arctic Studies Program.Shell provided funding ($2 million for year 1, $5 million/year for years 2 -5), and the NSB proposed budgets that Shell approved.Funding was administered by the NSB DWM.A 14-member steering committee, including village representatives, Shell and NSB staff, and independent scientists, was created to prioritize research.
Members included two scientists from the NSB and Shell, four independent scientists recommended by Shell and appointed by the NSB mayor, and one representative from each of the six coastal villages (Fig. 1).See Reiss (2012) for a more detailed account of the formation of the BSP.

EVALUATION METHODS
The evaluation of the BSP is based on 36 indicators that are grounded in boundary organizations, frameworks of knowledge co-production, Indigenous knowledge, and contested spaces (Caldwell et al., 2005;Wall et al., 2017).The assessment examines six categories of indicators, common across several evaluation frameworks (e.g., NRC, 2005).These indicators include inputs, processes, outputs, outcomes, impacts, and external factors (Table 1).Input metrics refer to human, social, natural, and financial capitals and capacities, including skill sets, leadership, resource allocation, and involvement of actors across the science-practice boundary.Process metrics are actions taken to achieve program goals, including the level and frequency of engagement, and individual perceptions of legitimacy.Output metrics represent deliverables, such as peer-reviewed publications, workshops or meetings, reports, and other products.Outcome metrics are more conceptual results, including achievement of project goals, continued interest in long-term collaboration, and perceptions of credibility, legitimacy, and relevance.Impact metrics refer to longer-term socio-economic and environmental consequences.External factors are historical and contextual variables outside the control of participants.These six categories of indicators were qualitatively mapped against the primary objectives of the BSP outlined in the NSB-Shell research agreement (NSB, 2010).
The research assessed indicators both qualitatively and quantitatively, in an approach commonly used in evaluating knowledge co-production (Wall et al., 2017).The primary data set was 16 semi-structured interviews that were conducted from September to November 2016 and from May to July 2018 (73% response rate).Interviewees included 12 BSP steering committee members, three external science managers who support research on the North Slope, and staff from NOAA/BOEM Arctic Open Water Meetings.Steering committee members included five village representatives (from four villages; one representative was employed by the NSB), two Shell representatives, one NSB representative (a DWM scientist), and four independent scientists.A second steering committee member from the DWM, who was not available for an interview, provided substantial information on the history of the program.The steering committee members who were interviewed included those who were part of the BSP since its inception (n = 6) and members who later served as replacements (n = 6).Researchers interviewed the majority of the participants over the telephone, and all conversations were recorded and transcribed.A review of BSP steering committee meeting minutes (n = 14 meetings) supplemented the interview data.Researchers coded interview transcripts and steering committee minutes using NVIVO content analysis software for themes related to the six categories of indicators.A web-based search (April 2018) used combinations of the following keywords to identify outputs emerging from the BSP: NSB-Shell Baseline Studies Program, BSP, NSB/Shell, and Collaborative Alaskan Arctic Studies Program.

RESULTS
Analysis of interview transcripts and steering committee minutes provided insight into several knowledge co-production indicators for the BSP.The following subsections outline the external factor, input, process, output, outcome, and impact knowledge co-production indicators for the BSP.These indicators refer to Table 1.

External Factors
External factors both supported and hindered knowledge co-production within the BSP.Although the NSB and Shell both desired and used applied science for decisions related to oil and gas development, the catalyzing event for the BSP was Mayor Itta's need to take a position on pending litigation (EF:3,4).The crisis situation and compelling circumstances surrounding a shared desire to address pending litigation motivated a commitment of resources from both Shell and the NSB to negotiate a research agreement that supported a locally funded baseline studies research program in the Arctic.The NSB also assured Shell that it would not attempt to block Shell's offshore Arctic drilling program (Keating, 2001;Affolderbach et al., 2012).Both the NSB and Shell have the capacity to use and manage information generated from the BSP (EF:2).The existing research infrastructure in Utqiaġvik and other applied Arctic science programs contributed to significant local research capacities to support the BSP (EF:5).The significant turnover in village representatives at the formation of the program reduced opportunities for crosscultural learning and trust building (EF:1).Three of the initial six village representatives were replaced within the first six months of the program, and a fourth was replaced within another year.No additional changes in village membership occurred following these initial changes; there was no turnover in BSP membership for independent scientists nor NSB representatives, and only one change for Shell.Shifting financial priorities and decrease in political

Inputs
Substantial human and financial capacities supported the BSP.Steering committee members held significant experience and expertise in traditional knowledge and Western science that provided capacity to support a research program designed to increase baseline knowledge of Arctic ecosystems (I:1).For decades, scientists had conducted research across the North Slope, which was viewed among local communities and Western scientists as inclusive of traditional knowledge and scientifically credible (e.g., Suydam et al., 2001;George et al., 2004).Village representatives understood local priorities and concerns, and had decades of experience observing change (e.g., Johnson et al., 2014).Independent and Shellcontracted scientists held substantial research experience in Arctic sea ice, ocean currents, marine mammals, environmental science, and acoustics (e.g., Southall et al., 2003;Eicken et al., 2004;Weingartner et al., 2005;Moore et al., 2010).Several independent scientists and DWM staff were experienced in collaborative research approaches with North Slope communities, which contributed to trust and familiarity at the onset of the BSP (I:9 and I:11).Few pre-existing trusted networks with Shell-contracted scientists existed among the village representatives, NSB staff, and independent scientists.A Shell representative had experience developing science-based solutions to mediate conflict between industry and local communities, though this experience was not in Alaska (I:7).Limited social science expertise reduced opportunities to study connections within socio-ecological systems.
Motivations for participating in and partnering with the BSP steering committee varied (I:8).Shell was motivated to regain trust, improve communication, and avoid litigation by supporting a research program that translated community-level priorities and concerns into funded research.DWM scientists desired influence on BSP-funded research, especially as the program was going to proceed regardless of their participation.Independent scientists wanted to address community concerns and increase the relevance of their research.Reputations of DWM and independent scientists were key in motivating participation among several village representatives.A North Pacific Research Board (NPRB) science manager partnered with the BSP to support their emerging Arctic Program.
The research agreement between the NSB and Shell formalized governance procedures, established group norms, outlined the steering committee's duties, and created a mechanism to support communication and collaboration (I:5).The research agreement did not outline the specific roles of the independent scientists in helping to establish the research agenda (e.g., comment on the state of Arctic research, serve as a tie-breaking vote between industry and local concerns, or help translate community concerns into research priorities).The steering committee nominated DWM senior biologist Dr. Robert Suydam as chair (I:10).
Shell, the NSB, village representatives, and independent scientists supported co-production via their contribution of services, funds, and staff or personnel time (I:2).Shell contributed ~$22 million to fund the BSP.Approximately 10% of Shell's funding supported steering committee engagement activities, including travel to attend quarterly in-person meetings and recruit village steering committee representatives (I:6).There were some challenges associated with the limited capacity of the DWM to manage the BSP, a concern articulated by the DWM as the research agreement was being developed.This was partly because of the tremendous workload added to existing research programs of DWM staff, such as drafting requests for proposals and evaluating research proposals.Indeed, the DWM initially declined to participate because of concerns about limited capacity and the desire to remain a trusted local science entity not connected to industry.Additional staff hires provided some capacity in managing the program and writing the science plan.

Processes
The steering committee met quarterly via in-person meetings and teleconferences (P:2).Initial meetings focused on building trust and formalizing procedures, and later meetings were dedicated to identifying research priorities and recommending research for funding (P:1).The sharing of community concerns, Shell's activities, and research updates by independent scientists facilitated equitable opportunities for most individuals to participate at meetings (P:3).Reasons for the significant turnover in village representatives at the formation of the program were not confirmed, though turnover was probably related to personal circumstances (e.g., competing priorities that hindered their participation), programmatic problems (e.g., dissatisfaction with the level of engagement), or both (P:3 -5).The timing of the evaluation, which occurred near the end of the five-year research agreement, limited opportunities for the evaluator to develop trusted relationships, observe participants, and develop a process to track community engagement and BSP-funded outputs more comprehensively (P:1); this is a common occurrence in evaluation (Wond, 2017).Some challenges were resolved in mutually agreeable ways, such as self-recusal in voting among steering committee members who advocated for science priorities that could benefit personal research programs (P:5).Other challenges were not resolved immediately.For example, Shell wanted all funds allocated to the BSP to be spent each year, as unused money could be invested in other projects, while the NSB viewed unused funds as a future resource for a local science program.This issue arose from the significant up-front capacity building required to establish protocols and trust during the early years of the project, which delayed spending funds allocated for research.The design of the research agreement to be renegotiated after a five-year period provided a basis for an adaptive approach to address potentially unresolved tensions.

Outputs
The BSP supported a broad range of research relating to the Arctic marine and terrestrial environment, including satellite tracking of seal movements, stress monitoring and bone loss of bowhead whales, exposure of juvenile salmon to dispersed oil, genetic studies of polar bears and beluga whales, movement of ice floes and surface currents, and knowledge gaps about ice conditions and ocean circulation relevant to emergency response.This research includes 22 studies identified in a DWM BSP project list created in June 2014.These projects have contributed to 13 peer-reviewed publications and reports, as well as several presentations (Table 2; OP:1, 2).In some cases, the BSP supplemented ongoing research efforts supported by enduring funding sources, such as the National Science Foundation, Department of the Interior, North Pacific Research Board, and Alaska Department of Fish and Game (e.g., Weingartner et al., 2017b).Several outputs will probably also emerge within the next few years from ongoing BSP-funded research, including outputs supported through a $1 million contribution to NPRB's multi-partnership $18.6 million Arctic Integrated Ecosystem Research Program (NPRB, 2015).Outreach products included a project website and summary updates in the DWM quarterly newsletter (OP:4).The extra time required to establish trust and formalize proposal review procedures contributed toward outputs being delivered later than Shell had anticipated (OP:3).There were also delays in communicating research findings to the communities, especially after the BSP research agreement ended.

Outcomes
The multiple outputs emerging from BSP suggest that the program advanced its core objective to expand baseline knowledge of Arctic ecosystems, especially related to bowhead whales, physical oceanography, seals, and sea ice (OC:1).However, Shell's non-renewal of the research agreement reduced opportunities to implement longerterm baseline assessments in the Arctic.Further, Shell's exit from the Arctic prior to establishing any sustained drilling platforms prevented opportunities to assess longer-term impacts of offshore drilling infrastructure on marine mammals and ecosystems and evaluate mitigation strategies to protect subsistence resources.
The relevance of BSP-funded research varied across and within groups (OC:3).In a few instances, BSP-funded research was relevant for Shell, the NSB, and several villages.For example, the analysis of the movement of near-shore surface currents in the Chukchi Sea provided supplemental information to BOEM-funded research on the connections of surface water flows between Wainwright and Point Lay and the rapid movement of currents from the Burger drilling site to Utqiaġvik, which could be used to enhance understanding of the potential movement of oil from a spill (Weingartner et al., 2017a, b).Similarly, a ringed seal study assessed the annual and seasonal migration movements of adult and juvenile seals (Von Duyke et al., 2017a).At the same time, two village representatives perceived low relevance because too much science centered on Utqiaġvik compared to the other villages.Outside the immediate orbit of the BSP, some outputs were less relevant to decision factors related to oil and gas permits, such as knowledge gaps needed by National Environmental Policy Act assessment analysts.However, contributing $1 million to NPRB's Arctic Integrated Ecosystem Research Program enabled the BSP to provide input into a nimble science funding organization that was seeking guidance on research priorities (NPRB, 2015).
Procedurally, the process of producing science was respectful of stakeholder values and fair in the treatment of opposing interests (OC:4).Steering committee members and external science managers commended the BSP for creating a process to ground research priorities in community-level concerns.However, lack of participation by two of the six villages during the early years of the BSP limited community input into the science priorities, and high rates of turnover among the village representatives suggested local residents may have held feelings of exclusion from the science planning process.
Steering committee members expressed few concerns about the scientific adequacy or trustworthiness of the research funded through the BSP (OC:2), though a marine advocate believed that the program was too close to industry (Kintisch, 2015).Nearly every active member of the steering committee expressed a desire to work on the BSP again, should such opportunities arise (OC:5).

Impacts
No long-term impacts associated with the use of information in risk governance for oil and gas development, protecting the ecological integrity of Arctic systems, or maintenance of subsistence resources in the Beaufort and Chukchi Seas were identified (IM:1).This finding may be related to the significant up-front investments of time and resources of the BSP committee to develop a science program and trusted process for selecting research priorities which led to delays in funding research.Additional factors related to limited long-term impacts may include time lags between initial funding and published findings, Shell's exit from the Arctic following the first five years of the research agreement, challenges in attributing research to impacts, and information availability and accessibility (Penfield et al., 2014).However, BSP-funded research on ocean currents was used to help estimate the location of a small marine vessel for a search and rescue event in July 2017 (Abdel-Fattah, 2018) (IM:3).The BSP also contributed to building science capacity in translating community concerns into relevant research priorities (IM:2).In describing learning based on deliberative dialogue during each steering committee meeting, which always began with listening to village concerns, an independent scientist stated: Initially, these comments would not make a lot of sense to me…how do I take these comments and suggest a research program?...[Later] I would say you told me you go seal hunting earlier this year than previous years.Why?So it's a matter of just slowly asking questions that might lead to me understanding maybe there was physical reason for this change.It became very clear to me that they were very much concerned with how the currents worked…, but it came about because the concern was we need to get our food resources and our food resources may be impacted by the currents.They're not going to start out by saying we need to know the currents because they affect my food resources.

DISCUSSION
Although the challenges of knowledge co-production within individual projects are broadly discussed in the literature, there remains limited work on program-level evaluations of boundary organizations in rural communities and contested spaces.The evaluation of the BSP identified several legacies that may affect knowledge co-production efforts among other science programs and boundary organizations, challenges to co-producing research, and difficulties in evaluating co-production on the North Slope.

Legacies of the NSB-Shell Baseline Studies Program
The closure of the BSP leaves several legacies in the context of designing and implementing a program to co-produce knowledge within a contested space.Procedurally, incorporating local concerns into the identification of Arctic research priorities and using traditional knowledge in research explicitly acknowledge the value of input from traditional knowledge holders.These practices also support aspects of former NSB Mayor Itta's desire for "responsible development" of oil and gas activities in the Arctic (NSB, 2008).Substantively, the peer-reviewed publications, workshop reports, and other outputs leave behind several boundary objects related to geophysical process and ecology that advanced Arctic science, some of which may be used to support risk governance and planning decisions (Table 2).In addition, the steering committee approved annual project lists that ranked the research priorities of the BSP, though the evaluator was not able to acquire these documents.Further, the networking and capacity-building efforts developed

Summary finding
Estimating bowhead whale age using tympanic growth layer groups is reliable to ~20 yrs, somewhat reliable from ages 20-30, and not reliable for older whales.Bowhead whales grow rapidly during the nursing period and sustain baleen and head growth between weaning and ~5 yrs old (limited growth in the rest of their body).Scarring data on bowhead whales indicate that line entanglement rates from fishing gear are relatively higher than other scarring from ship strikes and whale predation.Ship strikes and predation from killer whales are relatively higher on larger bowhead whales.Tested an approach to analyze bowhead whale baleen using commercially available enzyme, which potentially could be used in assessing reproduction and stress.Assessed sensory hairs of the bowhead whale.Conducted baseline study of the cochlea in beluga whale.Signs of acoustic trauma were not observed.
Hair sampling stations set near Barrow and Point Lay provided polar bear counts, locations, and timing of movement.Caribou densities near Wainwright, Alaska, are lower than those of the western segment of the Central Arctic herd and are not part of the core calving range.Fox dens are located in rugged terrains, often along rivers with greater depths of permafrost.Assessment of ringed seal spatial use, dives, and haul-out behavior in the Beaufort, Chukchi, and Bering Seas.
The spatial structure of water masses varies interannually and there are significant differences in chemical properties, phytoplankton communities, and thermohaline properties within the northern Bering and Chukchi Seas' continental shelves.Four different approaches are used to assess the annual circulation and water properties in the landfast ice zone of the Alaska Beaufort Sea.There are interannual variations in hydrographic properties in the northeastern Chukchi shelf relating to bottom water salinities and surface meltwaters.A northwestward baroclinic flow was found that opposed a model-predicted motion, suggesting a zonal flow convergence on the northeast side of Hanna Shoal. In

Challenges in Co-producing Knowledge on the North Slope
The evaluation of the BSP revealed several challenges to knowledge co-production in rural contested spaces.Some challenges, such as balancing trade-offs in relevance, credibility, and legitimacy, as well as negotiating differences in decision contexts, are common across multiple regions, sectors, and context (Cash et al., 2003).For example, the administration of the BSP by the DWM and inclusion of independent scientists on the steering committee legitimized efforts for the village representatives and enhanced credibility of the science for the NSB and Shell, respectively.However, leveraging the DWM and independent scientists' expertise led to potential conflicts of interest among steering committee members who evaluated the relative merits of and competed for science projects.Although an independent board of directors or science proposal review team (e.g., NPRB) could help avoid conflicts of interest, greater institutional complexity may reduce the availability of funding for and relevance of research that directly addresses local community concerns.
Other challenges may be especially difficult to address on the North Slope, such as participant availability, prioritizing research needs, and logistics.Although some individuals on the North Slope held extensive local knowledge, applied research capacity, or both, these persons were often engaged in other efforts, which limited their time to engage in the BSP.Distrust of Western scientists and oil and gas industries further limited the availability of village representatives, though efforts such as NOAA's Open Water Meetings have provided opportunities to facilitate dialogue between oil and gas industries and Alaska Native communities (NOAA NMFS, 2013).Conflicts of interest also reduced the availability of individuals to serve on the steering committee.For example, a federal science manager declined to participate on the BSP steering committee as an independent scientist because the scientist worked for an agency that regulated Shell and didn't want to be perceived as benefiting from Shell's exploration program.
Prioritizing applied research needs remains a key challenge for boundary organizations across the North Slope (e.g., BSP and NSSI) given the diverse priorities, values, goals, and needs of numerous stakeholder groups across multiple levels of governance.Prioritizing needs is further complicated by the significant up-front time that is needed to develop trust and build capacity among non-Indigenous partners to interpret stories accurately and learn how to translate community concerns into research priorities (Watson and Huntington, 2014).
There were also logistical challenges to support knowledge co-production.Travel between villages is often by air, and limited visibility can lead to delayed or cancelled flights.Shell representatives were prevented from going to some of the villages on commercial flights that did not conform to Shell's safety standards.Limited internet connectivity and telecommunication infrastructure also impeded remote engagement and preparation for upcoming meetings, such as the dissemination of science proposals for review prior to the steering committee meetings.Additionally, shortages in housing, offices, and people complicated capacity-building efforts to support the addition of new major research programs in Utqiaġvik (NSB, 2015).

Challenges in Evaluating Knowledge Co-production on the North Slope
The assessment of the BSP revealed several challenges in evaluating knowledge co-production on the North Slope.Obtaining community-level feedback is not always straightforward, given the extensive history of previous applied science research programs and interview fatigue (Streever et al., 2011).Confidentiality concerns, which may be especially acute in small communities and for contested issues, may also have affected response rates and response candidness (LaFrance, 2004).Also, interpreting stories from Elders accurately requires experience in communicating with rural Indigenous populations (Watson and Huntington, 2014).Disentangling differences in perceived legitimacy, relevance, and salience is especially complex when individuals often hold multiple affiliations with different missions and priorities (e.g., village corporation and tribe); this complexity suggests that the categories and metrics used to assess knowledge co-production need to be less coarse to encompass the diversity of stakeholders (Parente, 2017;Wall et al., 2017).

CONCLUSIONS
This paper evaluated the BSP on the North Slope of Alaska to assess factors contributing to knowledge co-production within an emergent boundary organization in the context of a contested political environment.Findings cohere with existing research on factors supporting knowledge co-production, including compelling circumstances to provide incentives for cooperation and commitment of resources, governance procedures to establish group norms and expectations, and leveraging existing human capacities and resources (Keating, 2001;Affolderbach et al., 2012;Leith et al., 2016).The evaluation also identified specific challenges that must be considered in the design and maintenance of boundary organizations within rural and Indigenous communities for contentious issues; these challenges include limited availability of personnel, prioritizing concerns and research needs, limited telecommunication infrastructure, additional time required to build trust, and logistics.
There are insights for emergent boundary organizations seeking to support knowledge co-production within rural and Indigenous contested spaces (which was a desired outcome for the BSP steering committee).Supporting knowledge co-production in rural Indigenous communities may require special budget considerations.First, a higher proportion of the total budget than in urban areas may be needed to support engagement activities, considering the high costs of travel, limited internet connectivity, and time required to develop trust.Leveraging existing local science capacities and trusted networks can extend limited resources and enhance credibility and legitimacy, though trade-offs must be considered given differences in decision contexts and goals, as well as potential conflicts of interest (Blades et al., 2016;Leith et al., 2016).Second, aligning project timelines and budgets in manners that support trust and relationship building at the onset of the program, rather than immediately prioritizing and funding research, provides opportunities to foster group norms, ground the legitimacy of the boundary organization, build capacity among scientists to translate community concerns into research questions, and mitigate the challenges associated with disparate expectations and decision contexts.
Providing adaptive approaches to revisit governance agreements increases opportunities to address emergent challenges, though tensions between reorganization and stability must be addressed (Garmestani and Benson, 2013).Feedback from external program evaluations can provide specific recommendations when governance agreements are renegotiated, especially when evaluations are developed at the onset of a program (Chouinard and Cousins, 2009).Interpreting responses and stories from Elders may require training and additional capacity building for the evaluator, especially for individuals with limited experience working with Alaska Native communities (Watson and Huntington, 2014;Black et al., 2015).Investigating the co-production of evaluation frameworks that account for Indigenous ways of knowing and Western science may enhance opportunities to understand how to support knowledge co-production in ways that increase community resilience.

FIG. 1 .
FIG. 1. Communities participating in the North Slope Borough-Shell Baseline Studies Program.

TABLE 2 .
Identified reports and peer-reviewed publications from the BSP (April 2018).
a workshop, local experts and university scientists identified knowledge critical to support emergency preparedness and response in Arctic offshore waters.Recommendations for future research are provided.