The approach taken to assess societal benefit in Phase II of the Expert Panels work is clear

The approach taken to assess societal benefit in Phase II of the Expert Panels work reflects the ROADS Guiding Principals

The link, if one was provided, led me to an accessible and useful overview of the societal benefit framework used.

Please comment on any improvements you feel are needed in this linked description.

If an approach other than the IAOAF was used, the Expert Panel has adequately demonstrated where the chose framework ties or augment's the IAOAF

The main findings from the societal benefit assessment are clear and well supported by the information provided

The candidate Shared Arctic Variables are clearly tied to the results of the societal benefit assessment

Further comments, questions or suggestions related to the relationships to other efforts

The opportunities presented by developing these SAVs are clear and well-justified

Further comments, questions or suggestions related to the opportunities for the proposed SAVs

The funding support outlined for the next steps is realistic for the needs of Phase III under this effort

The communications and engagement plans outlined for the next steps are well-developed and clear.

This Expert Panel demonstrates an overall maturity and readiness for Phase III work in accordance with the Guiding Principles.

Further comments on readiness related to the timeline, funding, communications and engagement

2025/08/19 4:19:03 pm EEST

2025/08/19 5:13:50 pm EEST

2025/08/19 5:44:33 pm EEST

The SBAs were relevant, but too abstract. More specific examples relevant to the EP members would be beneficial. It is a quite a jump in logic to go from the identified SBAs from the IAOAF to the candidate SAVs.

IAOAF was not linked, but unnecessary as it is the main one associated with ROADS.

It is not well-supported. There was discussion, but not well argued with either explicit examples from EP members or citations.

It is quite a jump in logic from the SBAs discussion to the candidate SAVs, especially for permafrost temperature and activate layer thickness. How specifically will it help in infrastructure/operations and disaster preparedness?

The listed other efforts is good, but how do they relate to the SBAs? An analysis of that would be beneficial to the proposal.

Permafrost-related well-being is compelling and new opportunity. But permafrost temperature and active layer thickness is already measured, how will this EP improve how those are measured. It could be argued that the EP would explore more societal benefits in Arctic communities to make them more relevant.

Besides the meeting, there is not other plans described. Who was lost? How often are you meeting? What will the facilitation of the meeting in Reyjakvik be?

More details are needed in the SBA analysis and connection to the identified SAVs. A plan for approaching phase III is needed. Funding for Phase IV is needed. I worry that the Indigenous members were the ones that dropped off, only "diverse" group was described.

2025/08/19 5:47:00 pm EEST

The group completed their assessment using a rating system from 1-100 that reflected the importance of the benefit sub-areas to the topic of permafrost. It was not clear how many participants engaged in the assessment (5? 10?), but they did give a sense of the geographic distribution of the perspectives and the different "sectors". Seeing more clearly presented analytical results (e.g. link to a diagram or table) would be useful for both of these points. In terms of GPs, the engagement in the EP, especially Indigenous engagement (GP1), seems sporadic and tentative. The ties to the global orgs and the ECVs is strongly apparent (GP3). The multi-stakeholder approach to the benefit assessment, as described, did aim for widely shared benefits (GP2).

The text descriptions are clear, but fairly high-level. I think there could be a stronger treatment of the ties between the exisiting ECVs and the societal benefit areas.

The inclusion of the well-being SAV is encouraging and points to a useful tie-back to the societal benefit assessment. The EP author states that the ECVs are not very explicitly tied to the benefit assessment but were adopted for other reasons.

The global and European organizations and projects noted are highly relevant for the ECVs. There is certainly work to do - as they note - on the Wellbeing SAV. This feels like it should be the emphasis for their on-going work as it is not entirely clear how much impact they will be able to have on the ECVs with the limited time left in the project. Presenting a vision for pulling this together would be extremely compelling for future work.

Again, I think the workshop time in Reykjavik would be best spent exploring the well-being SAV as this is where there is the highest potential for addressing the GPS (1-3) and where new and existing work and partnerships can strongly come together in support of new data exchanges to emphasize the "Shared" in SAVs. It would be useful, also, to detail which aspects of current data collection for the ECVs are not well-supported to provide "Shared" benefit. Is it that the resolution is poor, the coverage is poor, the data is not accessible or collected in formats that are useful to addressing local concerns. There was an excellent IARPC Collaborations meeting on Permafrost issues in Alaska on 18-AUG, for which the recording will be available. I highly suggest that the project leads watch the recording of the meeting (< 1hr) prior to the workshop as there was a lot of discussion of community resilience, well-being and data collection. Overall, this is a very important topic for progress. It would be encouraging to see strong engagement and alliances result from this work. This feels like the biggest issue with this EP is engagement. With limited time left in the project, there is probably not much that can be done to address that, but it is a useful thing to track in future work.

Given that funding is in place for the Reykjavik workshop, I think the group should be viewed as 'ready' to take that step (hence my rating of 3), but that they should scale their ambitions toward provided a vision to the AP about a well-being SAV for permafrost and make recommendations for how such an SAV could be taken up by a future group with renewed funding.

2025/08/25 7:21:11 am EEST

They are planning to revise their communication plans during a meeting this fall.

It sounds like some prior work may be needed to onboard new EP members and secure funding.

THIS IS A MOCK EVALUATION INTENDED FOR INTERNAL PURPOSES ONLY.

2025/09/08 5:37:41 pm EEST

Perhaps with more expanded methods sections, this may be more clear, but it's not apparent how the Principles were included in discussions, emphasized, considered individually or as an EP, etc. It seems that each person individually decided on the importance of the IAOAF components and then came together in a discussion (unclear how much time was spent deliberating in a group) on the societal benefits and the SAVs. There are methods approaches that can provide clear and justifiable process behind this type of convening, but this is typically a multistep approach (such as IDEA process for expert solicitations) and requires more convening and discussions than once individually and once collectively.

There was no link, we assume the link was provided to all participants and they accessed it from the same place.

Does the EP see ways to expand or reach outside of the "regional or community-dependent importance of a lot of the IAOAF areas"? This is mentioned but it is not clear if the authors felt that additional frameworks would better support their analysis (no other frameworks used so I assume not), or if they agree with the statement that is made.
It is not clear what is meant by, " ”Resilient communities” was viewed as important, but having this as an aspect of the other IAOAF areas could better serve the purpose of this analysis as the resilience of communities is impacted by changes that fit under changes considered under the other SBA areas."

I think that a big jump was made, either in the decision making of the SAVs from the SBA, or the composers of the text could expand this section because it is not super clear how the decisions were made from being informed and influenced by the SBA. Rather it seems that the EP decided that the GCOS scientific variables were of high importance and clearly defined with a lot of information readily available to draw from. This seems that permafrost temperature and active layer thickness is already well monitored and measured, and begs the question of how will the EP will contribute new and improved observing into the system. The third, which seems much more expansive, is less defined and will be much harder to pinpoint something measurable and useful from. This is a much more complex and abstract undertaking and it's not clear how the EP is going to handle this, nor how deep into this SAV they are going to go with the current (or expanded I hope) composition of the EP.

For the first two variables I think this is more accurate of a statement than the third variable.

I would like to see this section built out with further thought, discussion and reflection within the EP prior to Phase III: "The EP members have several connections to these initiatives and while the interaction modes have not yet been clearly defined, several possibilities for collaboration and information exchange have been discussed by the panel. These will be further defined as part of the refinement of the information needs from different stakeholders planned for the Phase III work." It should also be noted that the EP has used the term "stakeholders" throughout the document, but never Indigenous rights holders, rights holders, or other such term to clearly define the differences of Indigenous Peoples and organizations representing the rights of Indigenous Peoples, from "stakeholders". Indigenous Peoples are not "stakeholders" and should not be defined as such.

Given the loss of some EP members (which ones? and how many?) it is unclear who is going to be doing this work moving forward. ACA would be a good time to utilize what the EP has come up with from the SBA and take it out to a more diverse audience and workshop it/ ground truth it with the participants of ACA. I would seek out opportunities to gather feedback and input from as many people as are willing to engage on this portion of the work.

Completing the next phase given the contents of this Phase II document seems like quite a contracted and shortened timeframe to clearly address and move forward with expanded and informed SBA and SAV components, especially given the shrinking of the EP in size.

There is really no information given here, yet the EP mentions that "The key opportunities identified in developing this SAV is the opportunity for better information exchange between the different communities and stakeholders. It has become clear that the local actors rarely know of all the different monitoring efforts being carried out on the permafrost theme by authorities and scientists. Better awareness is expected to help drive collaboration and inclusion.The SAV development will also hopefully be able to provide direct guidelines and tools for local communities wanting advice and inspiration on how to address permafrost changes." - these things will require significant coordination and communication and yet are undefined in the provided document. How are communities going to be made aware of other efforts? How are guidelines going to be shaped and developed and delivered in appropriate ways? What is the plan for widespread communication and awareness in small remote communities? How will networks be strengthened? How will providing information as to what is happening in other locations help another community without other support? etc.
It is also unclear what is meant by "considering strengthening the panel during the on site meeting", but this seems particularly important and time sensitive.

See previous answers and I hope that we can have a collective discussion in an upcoming meeting on these points as an AP.