| A | B | C | D | E | F | G | H | I | J | K | L | M | N | O | P | Q | R | S | T | U | V | W | X | Y | Z | AA | |
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
1 | ATRT3 Final Report Annex E: Public Comment Analysis | ||||||||||||||||||||||||||
2 | Commenter | Report Section | Comment (available here: https://mm.icann.org/pipermail/comments-atrt3-draft-report-16dec19/) | Response | Change | Where? | |||||||||||||||||||||
3 | ALAC | 10.5 - Option 2 - Organizational Reviews | One of the most important issues tackled by ATRT is that of how future reviews, and by extension, ICANN’s foremost accountability function would be designed. From an ALAC perspective, the most important consideration is how appropriate oversight and representation of billions of internet end users can be ensured going forward. Option 2 is the preferred option of the ALAC, However It appears that option 2 requires more fleshing out to inform the community about what it would specifically entail. While the status quo is not working according to a variety of stakeholders, more description and consideration should go into option 2 going forward, or indeed any other solution that is supposed to solve the current concerns. For example, what options are available to optimize and streamline review processes, and are there any elements that can be removed, or reviews combined, without causing problems in the current ICANN structure? While option 2 addresses some of these concerns, the description in the current report is too curt to determine its impact. The proposals should be more detailed and consider the pros and cons, e.g. when it comes to the specific concern of option 2. It is obvious that the review team cannot provide a complete solution for a variety of reasons, including procedural and due to the amount of required work. Yet, it would be very useful to provide more details on how this approach could look like, and what would have to be considered when putting the proposal into practice. Tackling how these changes would improve on current levels of accountability and transparency is important to maintain ICANN’s standing and potentially its existence going forward. From an end user perspective, more, not less, accountability and transparency is required from the ICANN community and org. ICANN should welcome input from outside and utilize additional studies carried out by reputable researchers across the globe, and such research should not necessarily be commissioned by ICANN. | ATRT3 has significantly evolved Option 2 into its recommendation on Reviews. | With respect to Organizational Reviews ATRT3 proposes: ATRT3 is proposing to move to a three tier system in its recommendation to evolve Organizational Reviews and to implement a continuous improvement program for SO/ACs: 1 An annual survey of members/participants in each SO/AC. The results of these would be public and used as input to the continuous improvement programs in each SO/AC as well as the Holistic review. 2 Evolving the current Organizational Reviews into reviews of the continuous improvement programs in each SO/AC (SO/AC Continuous Improvement Programs -SO/AC CIPs) to consider the results of the surveys of members/participants, assess the progress of the continuous improvement efforts (at least every three years) and producing a report which will feed into the Holistic reviews. 3 A Holistic review every 7 to 8 years to: Review continuous improvement efforts of SO/AC/NC based on good practices. Review the effectiveness of the various inter SO/AC/NC collaboration mechanisms. Review the accountability of SO/ACs, or constituent parts, to their members/constituencies (this will include an in depth analysis of the survey results). Review each SO/AC/NC as a whole to determine if they continue to have a purpose in the ICANN structure as they are currently constituted or if any changes in structures and operations are desirable to improve the overall effectiveness of ICANN as well as ensure optimal representation of community views (but taking into consideration any impacts on the Board or the Empowered Community). ATRT3 believes its recommendation ensures a better system for the improvement of SO/ACs and does so in a more regular and controlled fashion. | Section 8 of the ATRT3 Final Report. | |||||||||||||||||||||
4 | ALAC | 10.5 - Option 2 - Specific Reviews | Concluding SSR (as well as or any other) reviews in short, 5-7 day workshops seems difficult to achieve, at least under current conditions. Such short time frames will complicate asking questions to staff or constituencies. Therefore, the preparations for such workshops need to be extremely well thought-through and comprehensive. In terms of accountability, it would be necessary for ICANN to undergo continuous assessment as favored by ATRT (e.g. regular security audits) with reports being made available so that the SSR review can essentially review those reports instead of conducting their own lengthy research. The review team should consider how delays (e.g. slow staff responses) could be dealt with in this case. | Based on comments ATRT3 chose to limit its changes to Specific Reviews. ATRT3 is proposing a moratorium on RDS and SSR Reviews going forward until the next ATRT can evaluate these properly. ATRT3 believes there should be one additional CCT review which would be limited in time and that ATRT reviews should remain quite similar to what they are now. Finally ATRT3 adds a Holistic Review to the set of Specific Reviews. | Please read section 8.4 of the ATRT3 for complete details. | Section 8 of the ATRT3 Final Report. | |||||||||||||||||||||
5 | ALAC | 11.4.2 Accountability Indicators | We support the suggested changes to the ICANN Public Comment, public input as well as the accountability indicators. We believe those changes would make the Public Comments more effective and show better transparency. We strongly believe that a wide, open and inclusive process should be maintained in policy development process especially with input representing the multistakeholder environment. Policy development must be transparent, efficient and should not be biased or skewed towards a group. | ATRT3 significantly evolved its thinking on Accountability Indicators into a recommendation on "Accountability and Transparency Relating to Strategic and Operational Plans including Accountability Indicators" which should continue to meet the intent of the comments made by the ALAC. | Significant, please see section 9.4 of the ATRT3 Final Report | Section 9 of the ATRT3 Final Report. | |||||||||||||||||||||
6 | ALAC | 12.4.1 - Developing a Prioritization Process | Prioritization is a complex process that requires a lot of information and support from ICANN org, and any teams must include individuals with insight into the ICANN structures and processes. At the same time, this process requires a level of transparency (wherever possible) so that the community can follow and understand the choices made, and must also involve individuals with an “outsider” perspective to ensure a balanced and holistic assessment. Process and outcomes need to be extremely well thought through, resourced, and transparent. | ATRT3 has converted this suggestion into a recommendation in its Final Report while refining it. | Some refinements in Section 10 of the ATRT3 final report. | Section 10 of the ATRT3 final report. | |||||||||||||||||||||
7 | ALAC | 5.4.2 Public Comments | We support the suggested changes to the ICANN Public Comment, public input as well as the accountability indicators. We believe those changes would make the Public Comments more effective and show better transparency. We strongly believe that a wide, open and inclusive process should be maintained in policy development process especially with input representing the multistakeholder environment. Policy development must be transparent, efficient and should not be biased or skewed towards a group. | This has been made into a formal recommendation by ATRT3. A number of refinements were made in converting this into a recommendation. | Yes but in keeping in the spirit of the original suggestion. | Section 3 of the ATRT3 Final Report. | |||||||||||||||||||||
8 | ALS CAPDA | 10.5 - Option 2 - Organizational Reviews | With regard to the reviews and the two options, we support the latter, in particular because the notion of continuous improvement is absolutely relevant. The holistic review will verify after the two cycles that the concept of continuous improvement really provides the right solutions for the evolution of SO/AC and ICANN in general. But the next review will need to be holistic to provide a global picture and study ICANN's "building blocks" and the relationships between them, and also to propose improvements that will in particular reduce the complexity of the organization. This is an important element in terms of ICANN's accountability and transparency. | ATRT3 has opted to make a recommendation based on Option 2 which includes a continuous improvement component which should meet what is being proposed in the comment. | Please read section 8.4 of the ATRT3 for complete details. | Section 8 of the ATRT3 Final Report. | |||||||||||||||||||||
9 | ALS CAPDA | 10.5 - Option 2 - Specific Reviews | With regard to specific reviews we agree with the proposals made in the report but we believe that one or more new journals could be created, for example on the issue of DNS abuses. These will be conducted in the form of workshops of 3 to 5 days. We suggest merging the organization holistic review projects and ATRTx to allow for a completely comprehensive vision of ICANN once every 7 years and not the 5-year project as currently planned. | Based on the comments ATRT3 chose to limit its changes to Specific Reviews | Please read section 8.4 of the ATRT3 for complete details. | Section 8 of the ATRT3 Final Report. | |||||||||||||||||||||
10 | ALS CAPDA | 11.4.2 Accountability Indicators | Finally, with regard to the accountability indicators currently published on the ICANN website, we suggest that future indicators be useful, up-to-date and for this reason they should be defined on the basis of THE community'sproposalby ICANN org. | ATRT3 significantly evolved its thinking on Accountability Indicators into a recommendation on "Accountability and Transparency Relating to Strategic and Operational Plans including Accountability Indicators" which should continue to meet the intent of the comments made. | Significant, please see section 9.4 of the ATRT3 Final Report | Section 9 of the ATRT3 Final Report. | |||||||||||||||||||||
11 | ALS CAPDA | 12.4 - Developing a Prioritization Process | Proposals for prioritization should be recommendations, not just suggestions. The use of the new operational standards for specific reviews that are proposed to serve as the basis for the operation of the annual priority-setting process must be tested to ensure that they meet all prioritization of all recommendations. The proposal that prioritisation should be carried out in an open and transparent manner and that every decision must be justified and documented seems useful, but with currently more than 200 recommendations to prioritize is not toomuchdemand? Won't that lengthen the time frame too much? It would also be useful to add to the criteria to use that of the needs/priorities of the various ICANN players. | ATRT3 has converted this suggestion into a recommendation in its Final Report while refining it. | Some refinements in Section 10 of the ATRT3 final report. | Section 10 of the ATRT3 final report. | |||||||||||||||||||||
12 | ALS CAPDA | 5.4.2 Public Comments | As part of the public comment procedure, icann's translations of a summary and key issues into ICANN's official languages are very important improvements. It should also open the possibility of organizing exchanges/debates based on previous comments (such as a collective chat). | This has been made into a formal recommendation by ATRT3. A number of refinements were made in converting this into a recommendation. The idea of a chat has not been incorporated into the ATRT3 recommendation. | Yes but in keeping in the spirit of the original suggestion. | Section 3 of the ATRT3 Final Report. | |||||||||||||||||||||
13 | ALS CAPDA | 9.4.1 - ATRT2 recs | Proposals for prioritization should be recommendations, not just suggestions. The use of the new operational standards for specific reviews that are proposed to serve as the basis for the operation of the annual priority-setting process must be tested to ensure that they meet all prioritization of all recommendations. | This was originally only a suggestion which has been upgraded into a recommendation. ATRT3 implementation shepherds will support the implementation of the remaining ATRT2 recommendations | Yes but in keeping in the spirit of the original suggestion. | Section 7.4 of the ATRT3 Final Report. | |||||||||||||||||||||
14 | ALS CAPDA | Other - Election of Board members | I think it is relevant, useful and desirable for a proposal around the transparency of election processes to be developed. You are unaware thatin theyears the terms of reference regarding the evaluation and improvement of the governance of the Board of Directors, there are subjects that do not have to be addressed in the report under discussion and which are important. In the efficiency and effectiveness of the ICANN Board of Directors, its composition and in particular its diversity is not addressed. These shortcomings seem to me to be relevant now. | Selection of Board members is not in the remit of the ATRT3 although this was indirectly considered in the ATRT3 survey question dealing with diversity on the Board which resulted in a suggestion. | N/A | N/A | |||||||||||||||||||||
15 | ALS CAPDA | Recommendations and suggestions | Following the webinar, recommendations and suggestions, the distinction does not seem very clear to everyone, however, even if the recommendations require significant work to meet the requirements of the new standards, there should be more. Moreover, if all proposals are of the same importance and if the Executive Board does not distinguish why are they treated and named differently? | ATRT3 has taken on these comments in only making 5 recommendations and leaving all suggestions as optional. | In its introduction ATRT3 states - "In a context where there are 325 review recommendations awaiting approval or implementation ATRT3 has chosen to be pragmatic and effective in making recommendations. Although ATRT3 makes both recommendations and suggestions it only requires the implementation of its 5 recommendations. Suggestions are meant to be exactly that “suggestions” and it is left to those concerned by these individual suggestions, which can be found in Annexes A and B of this report, to decide if they should, or not, be implemented.". In keeping with this ATRT3 has moved all suggestions related to its Survey results out of the main report into Annex B. | Introduction | |||||||||||||||||||||
16 | BC | 10.5 - Option 1 | The BC does not support Option 1. We have little confidence that the “new oversight mechanism” in Option 1 would address dissatisfaction expressed with the current review processes. | Option 1 was not retained | Option 1 was removed from Section 8 of the ATRT3 Final Report. | Section 8 of the ATRT3 Final Report. | |||||||||||||||||||||
17 | BC | 10.5. - Option 2 - Organizational Reviews | The BC supports Option 2 for the Organizational Reviews (not for Specific Reviews), with this explanation and further suggestions: (Continued...see "Additional Details" tab) | ATRT3 has significantly evolved Option 2 into its recommendation on Reviews. | Please read section 8.4 of the ATRT3 for complete details. | Section 8 of the ATRT3 Final Report. | |||||||||||||||||||||
18 | BC | 10.5. - Option 2 - Specific Reviews | The BC does not support Option 2 for the Specific Reviews, with this explanation and further suggestions (Continued...see "Additional Details" tab) | Based on comments ATRT3 chose to limit its changes to Specific Reviews. ATRT3 is proposing a moratorium on RDS and SSR Reviews going forward until the next ATRT can evaluate these properly. ATRT3 believes there should be one additional CCT review which would be limited in time and that ATRT reviews should remain quite similar to what they are now. Finally ATRT3 adds a Holistic Review to the set of Specific Reviews. | ATRT3 recommends that the Board and ICANN Org.: -Suspend any further RDS and SSR Reviews until the next ATRT. -Allow one additional CCT review following the next round of new gTLDs. -Continue with ATRT reviews with a modified schedule and scope -Evolve the content of the Organizational Reviews into continuous improvement programs in each SO/AC/NC. -Add a Holistic Review, as a special Specific Review, which will look at all SO/AC/NC and their relations. -Implement a new system for the timing and cadence of the reviews. | Section 8 of the ATRT3 Final Report. | |||||||||||||||||||||
19 | BC | 12.4.1 - Developing a Prioritization Process | That group would spend a year to develop a process for annual prioritization of reviews, considering Budget, Cost of implementation, Complexity and time to implement, Prerequisites and dependencies with other recommendations, Value and impact of implementation, and relevance to ICANN’s Mission, Commitments, Core Values and Strategic Objectives. The BC does not believe this prioritization mechanism is necessary or helpful. | ATRT3 hopes that its recommendation to integrate this process in the standard budgeting process will avoid these issues. | Some refinements in Section 10 of the ATRT3 final report. | Section 10 of the ATRT3 final report. | |||||||||||||||||||||
20 | BC | 9.4.1; 9.4.2 | The BC supports both these suggestions regarding implementation of prior ATRT recommendations. The BC further suggests that “Implementation Shepherds” be recruited from among community volunteers who actively participated in the identified prior review. | This was originally only a suggestion which has been upgraded into a recommendation. | Yes but in keeping in the spirit of the original suggestion. | Section 7.4 of the ATRT3 Final Report. | |||||||||||||||||||||
21 | Board | 3.4 | In connection with Board diversity, the Board believes there is a need for clarification from the ATRT3 on whether it is recommending the role of the EC be changed to allow the EC to substantively evaluate and/or override the nominations that the Nominating Committee (NomCom), the Supporting Organizations (SOs), and the At-Large Community make to the ICANN Board of Directors. (Section 3.4.7, p.46). (Continued, see "Additional Details" tab). | It was hoped that if there were significant issues with respect to diversity of nominations that the EC could point this out. This being said ATRT3 notes that Implementation of the suggestions associated with the survey results are optional and as such are open to change. | No change | Annex B of the ATRT3 Final Report | |||||||||||||||||||||
22 | Board | 13 | In connection with Section 13 - SO/AC and Sub-structures Accountability (page 107), the Board observes that the ATRT3 intent and direction is not yet clear in the Draft Report. As the ATRT3 refines its work in this area, the Board would like to understand what the ATRT3 envisions in terms of potential recommendations or suggestions and more broadly what the intent of this section is relative to the ATRT3 scope of work. To the extent the eventual recommendations or suggestions would be related to the SO/AC and Sub-structures, how does the ATRT3 envision socializing these with the community and would such recommendations and suggestions be intended to be passed through to the relevant community groups? | ATRT3 accepted this comment and dropped this section from its report. | All SO/AC and Sub-Structures Accountability information was removed from the ATRT3 final report. | N/A | |||||||||||||||||||||
23 | Board | 10.4.1 | The Board would suggest the ATRT3 clarify the objective of this intended public comment in more detail. For example, if the objective is to bring increased community awareness to the progress and the outcomes of implementation work, then there may be other more suitable ways to accomplish such objective. The Board observes that if such a public comment goes ahead, improvements to timing and cadencing of reviews should allow for sufficient time to accommodate such a public comment proceeding. | Implementation of ATRT2 recommendations will have to be reviewed by ICANN Org. and the ATRT3 implementation shepherds and then go through prioritization. | None | Section 7 and Annex A of the ATRT3 Final Report | |||||||||||||||||||||
24 | Board | 10.5 - Option 1 | The Board does not consider Option 1 (as described in the Draft Report) to be viable. Option 1 does not address any of the community’s stated concerns that too many reviews occur simultaneously, the reviews are too long and resource-intensive, and too many recommendations are produced. In addition, Option 1 proposes a new accountability layer on top of the recently implemented new accountability measures, which in the Board’s view is not necessary considering that the community has access to a number of accountability mechanisms and processes. | Option 1 was not retained | Option 1 was removed from Section 8 of the ATRT3 Final Report. | Section 8 of the ATRT3 Final Report. | |||||||||||||||||||||
25 | Board | 10.5 - Option 2 - Organizational Reviews | The Board supports the direction of Option 2, which offers interesting considerations and will benefit from clarifications and further refinement to capture ongoing discussions identified above. The Board believes that there is currently a window of opportunity to substantially improve the effectiveness of reviews and their outcomes. The Board acknowledges the complexity associated with this streamlining work, given the range of discussions and dependencies, including the ongoing Bylaws-mandated review work. Given ATRT3’s limited remaining time, the Board encourages the ATRT3 to define overarching criteria that can guide the future review streamlining work. Such criteria should focus on the intent and requirements of the Bylaws, the needs of the ICANN community, as well as ICANN’s Strategic Plan. The Board envisions the future streamlining work to further evolve based on ATRT3 criteria and be informed by public comments received by ATRT3 as well as input gathered over the last few years as the community has been confronting the need to re-imagine reviews. | Please see Annex E of the ATRT3 Final Report for a detailed response. | N/A | N/A | |||||||||||||||||||||
26 | Board | 10.5 - Option 2 - Organizational Reviews | Based on the overarching criteria noted above, with regards to organizational reviews, the ATRT3 might want to consider how to bring consistency and standardization to those individual SO/AC reviews. It might be useful to consider modelling ICANN review processes on industrystandard methodologies/frameworks for assessing organizations and achieving organizational excellence (for example, EFQM or Baldrige excellence frameworks). The Board’s view is that such an approach would support the effectiveness of the holistic review, as proposed by the ATRT3, by providing consistent and comparable data points. In relation to improvements, one area benefiting from further input would be how does the ATRT3 foresee the role of independent, external consultants in Option 2, considering the ATRT3 survey results of 79% of individual responses and 90% of Structure responses agree that Organizational Reviews should continue to be undertaken by external consultants? | Please see Annex E of the ATRT3 Final Report for a detailed response. | N/A | N/A | |||||||||||||||||||||
27 | Board | 10.5 - Option 2 - Specific Reviews | In terms of specific reviews, consideration of overarching criteria could guide the simplification of the review processes and result in more impactful outcomes. The ATRT3 could propose and clarify several areas including, for example: guidance on how to support appropriately skilled and impartial review teams; propose strategies to help future review teams set their scope in a way that allows them to focus on issues most relevant and important to the ICANN community; and encourage review teams to improve the quality of their recommendations, including how to achieve effective and resource-conscious solutions. Additionally, the ATRT3 could also consider focusing on the overarching criteria and recommend that the Board, community, and ICANN org develop streamlined review processes based on those criteria. | ATRT3 believes that the new Operating Standards for Specific Reviews, which its recommendations follow, combined with ATRT3's recommendations on reviews and prioritization address many of the issues in this comment. | Please read section 8.4 of the ATRT3 for complete details. | Section 8 of the ATRT3 Final Report. | |||||||||||||||||||||
28 | Board | 11.4.1, 11.4.2; 5.4.1 - 5.4.3 | Finally, the Board notes that ICANN org has undertaken and/or plans to undertake improvements to both public comment procedures and accountability indicators. The Board encourages the ATRT3 to engage with ICANN org to ensure that any recommendations issued in the Final Report are complementary to the recently completed and ongoing improvements. The Board also notes that ICANN org is completing its analysis of the ATRT3 Draft Report and intends to submit a separate comment on operational matters. | ATRT3 met with Susanna Bennett of ICANN to review these points and focused on Accountability Indicators. This led ATRT3 to significantly expand this recommendation to deal with the Accountability and Transparency Relating to Strategic and Operational Plans including Accountability Indicators. | Significant, please see section 9.4 of the ATRT3 Final Report | Section 9 of the ATRT3 Final Report. | |||||||||||||||||||||
29 | Board | 12.4 - Developing a Prioritization Process | The ATRT3’s thoughts on prioritization work are very detailed, and the Board supports the proposal of a “holistic suggestion with respect to prioritization.” This is the right strategy. The Board believes strongly that prioritization of review recommendations cannot take place in isolation and that the prioritization process must fit into ICANN’s existing budget and planning mechanisms. Furthermore, all parts of ICANN need to take part in the prioritization – ICANN community, ICANN Board and ICANN org. Prioritization of community-issued recommendations needs to take place within the broader context of all ICANN work and must consider implications on community and ICANN org resources and bandwidth, as well as the availability of resources (including funds) whether required up-front only, or on an ongoing basis. (Continued, see "Additional Details" tab). | ATRT3 has converted this suggestion into a recommendation in its Final Report while refining it. | Some refinements in Section 10 of the ATRT3 final report. | Section 10 of the ATRT3 final report. | |||||||||||||||||||||
30 | Board | Recommendations and suggestions | The Board notes that the ATRT3 formulated 35 recommendations, suggestions, and strong suggestions, stating that “The ATRT3 does not consider suggestions to be less important than recommendations. The determination if an item is a suggestion or a recommendation will be finalized in ATRT3’s final report.” The Board understands that, under the Bylaws, the Board is obligated to act only upon recommendations issued by Specific Review teams. The Bylaws, at Section 4.6(a)(vii)(A), also impose an obligation on the ATRT3 to “attempt to prioritize each of its recommendations and provide a rationale for such prioritization.” While the ATRT3’s use of the terms “recommendations,” “strong suggestions,” “suggestions,” and “observations” could be a manner of expressing different levels of priority, the Board requests clarity on which of the ATRT3‘s outputs are formal recommendations. To the extent that the ATRT3 also includes categories of items other than “recommendations” in its final report, this could create confusion as this is not aligned with the Bylaws or Operating Standards. There would not be clarity on the expectation of how or when the Board is to act on suggestions, whether it would be appropriate to hold ICANN org accountable to implementation of those suggestions, or the other attributes of “recommendations.” If the ATRT3 believes the distinction between recommendations and suggestions might be relevant for future review teams, the Board encourages the ATRT3 to include a recommendation on how to use “suggestions” versus “recommendations” in future reviews, allowing for community input, and thereby ensure broad community agreement on such an approach as well as a consistent and transparent output of all of ICANN’s future Specific Reviews. | ATRT 3 has taken on these comments in only making 5 recommendations and leaving all suggestions as optional. | In its introduction ATRT3 states - "In a context where there are 325 review recommendations awaiting approval or implementation ATRT3 has chosen to be pragmatic and effective in making recommendations. Although ATRT3 makes both recommendations and suggestions it only requires the implementation of its 5 recommendations. Suggestions are meant to be exactly that “suggestions” and it is left to those concerned by these individual suggestions, which can be found in Annexes A and B of this report, to decide if they should, or not, be implemented.". In keeping with this ATRT3 has moved all suggestions related to its Survey results out of the main report into Annex B. | Introduction | |||||||||||||||||||||
31 | Board | Assessment of Implementation of Prior Review Recommendations | In relation to implementation, the Board observes that ATRT2 recommendations did not always include guidance on outcome and measurement of success, as detailed in ICANN org’s note to ATRT3 on 11 December 2019. The Board agrees with the ATRT3 that there is room for improvement in ICANN org’s clarification on how implementation has been addressed, as well as delivering clearer and more understandable reporting of implementation progress. As the ATRT3 forms its recommendations, the Board notes that the Operating Standards (Section 4.1) provide guidance for the drafting of recommendations and encourage the ATRT3 to adhere to these as closely as possible. Overall, the Board is committed to an effective outcome-based implementation of clear and focused community recommendations, and believes that concrete fact-based problem statements and clear definition of what the desired outcome will look like, including how implementation should be evaluated by the community and the next review team, will go a long way toward more impactful outcomes. The Board strongly encourages the ATRT3 to consider this when compiling its final report. For illustration purposes, we included several examples: (Continued, see "Additional Details" tab). | ATRT3's recommendation on the implementation of ATRT2 recommendations has taken this into account given the Implementation of ATRT2 recommendations will have to be reviewed by ICANN Org. and the ATRT3 implementation shepherds and then go through prioritization. | ATRT3 recommendation - ICANN Org. shall review the implementation of ATRT2 recommendations in light of ATRT3’s assessment of these and complete their implementation subject to prioritization (see recommendation on the creation of a prioritization process). | Section 7 of the ATRT3 Final Report. | |||||||||||||||||||||
32 | Chokri Ben Romdhane | 5.4.3 | Issue:Public Input The public comment process may include a mechanism that give the opportunity to commentaires to give their opinions about the staff report on the Public Comment proceeding and to claim any omission. | This was attempted as part of the ATRT2 recommendations and discontinued because it was not used. | Yes but in keeping in the spirit of the original suggestion. | Section 3 of the ATRT3 Final Report. | |||||||||||||||||||||
33 | Chokri Ben Romdhane | 7.4.1 | Issue Policy Development Process : The PDP working groups should develop their policies independently of any technical issues, and any technical or jurisdiction study should only support and not influence policies and transform them in technical solutions rather then a policies applying for differents technical issues. All the parties including the board should avoid such technical or juridiction study that will limit the scope of those working groups. | Noted - Implementation of ATRT2 recommendations will have to be reviewed by ICANN Org. and the ATRT3 implementation shepherds and then go through prioritization. | None | Section 5 of the ATRT3 Final Report. | |||||||||||||||||||||
34 | GAC | 4.4.1.1 | The GAC welcomes this proposed suggestion. The liaison (or “point-of-contact” as it is typically called within the GAC) function is an important role not only for GAC members to take to other communities, but for other communities to utilize in outreach to the GAC. Beginning at ICANN63 in Barcelona, the GAC established a new working group to focus on developing improvements to help update and amend its present operating principles. One of the areas identified in chartering that working work was the role and functions of GAC liaison. The GAC will utilize the new working group to develop criteria that will improve the clarity and specificity of the liaison/point of contact roles. The work of the individuals presently assigned by other communities to serve as liaisons “to” the GAC has offered very useful insights and opportunities for learning for the GAC leadership. The GAC can use the examples of those experiences to help fashion guidelines for future liaison and point-of-contact appointments. | Implementation of ATRT2 recommendations will have to be reviewed by ICANN Org. and the ATRT3 implementation shepherds and then go through prioritization. | None | Section 7 and Annex A of the ATRT3 Final Report | |||||||||||||||||||||
35 | GAC | 4.4.1.2 | The GAC welcomes this proposed suggestion. The liaison function is an important role not only for GAC members to take to other communities, but for other communities to utilize in outreach to the GAC. The GAC has recent experience in developing onboarding resources for first-time GAC meeting participants and will consider utilizing that expertise to provide orientation to new liaisons. | Implementation of ATRT2 recommendations will have to be reviewed by ICANN Org. and the ATRT3 implementation shepherds and then go through prioritization. | None | Section 7 and Annex A of the ATRT3 Final Report | |||||||||||||||||||||
36 | GAC | 4.4.2 | In its Recommendation 6.1.H, the ATRT2 recommended that when deliberating on matters affecting particular entities, to the extent reasonable and practical, the GAC give those entities the opportunity to present to the GAC as a whole prior to its deliberations. The GAC is pleased to see that the ATRT3 has determined that the overall implementation and effectiveness of this recommendation are currently deemed to be satisfactory (see ATRT3 Draft Report at page 52). The GAC welcomes the proposed suggestion that it continue to commit to its efforts in this area. The committee will continue its collaborative outreach with other ICANN communities impacted by its advice both during ICANN public meetings and intersessionally, when appropriate. Suggestions from other communities how this might be done or improved upon will be most welcomed. | Implementation of ATRT2 recommendations will have to be reviewed by ICANN Org. and the ATRT3 implementation shepherds and then go through prioritization. | None | Section 7 and Annex A of the ATRT3 Final Report | |||||||||||||||||||||
37 | GAC | 4.4.3 | The ATRT2 recommended (see ATRT2 Recommendation 6.6) that the Board work jointly with the GAC, through the BGRI working group, to identify and implement initiatives that can remove barriers for participation, including language barriers, and improve understanding of the ICANN model and access to relevant ICANN information for GAC members. The recommendation stated that the BGRI working group should consider how the GAC can improve its procedures to ensure a more efficient, transparent and inclusive decision-making and that the BGRI working group should develop GAC engagement best practices for its members that could include issues such as: conflict of interest; transparency and accountability; adequate domestic resource commitments; routine consultation with local Domain Name System (DNS) stakeholder and interest groups; and an expectation that positions taken within the GAC reflect the fully coordinated domestic government position and are consistent with existing relevant national and international laws. The GAC is pleased that the ATRT3 recognized that “significant improvements that have been made by the GAC since the ATRT2 recommendations were made.” (see Draft Final Report at page 52). The GAC agrees with the ATRT3 that “this type of recommendation implies more of a continuous improvement process rather than a single outcome”, and despite being difficult to fully implement looks forward to continuing its active work and regular meetings with Board members in the context of the BGRI (now renamed as the Board/GAC Interaction Group – “BGIG”) to publicize existing improvements and identify new ones in the months and years to come. | Implementation of ATRT2 recommendations will have to be reviewed by ICANN Org. and the ATRT3 implementation shepherds and then go through prioritization. | None | Section 7 and Annex A of the ATRT3 Final Report | |||||||||||||||||||||
38 | GAC | 4.4.4 | The GAC welcomes this suggestion from the ATRT3 and notes that more details on this suggestion in the Final Report would help clarify the expectations in this area. The GAC partnered with Board members late last year (in the context of the BGIG) to discuss improvements to the order and timing of Board responses to formal GAC advice (the Board “Scorecard”) in the hope of providing a more regularized process for the GAC to evaluate the Board responses to that advice. That improved process already features a back-and-forth clarification procedure that enables the Board to confirm its understanding of GAC Communique advice to help it in evaluating the specifics of Communique text. This ATRT3 suggestion offers insights for limiting or potentially eliminating the need for clarification questions entirely. Over the last several ICANN public meetings, the GAC has evolved its Communiqué drafting process to improve the effectiveness and efficiency of GAC member advice deliberations during each public meeting. These improvements have had the effect of putting more focus on discussions surrounding Communique drafting efforts and have led to adding a “rationale” section to accompany certain GAC advice to help improve understanding by providing background and context on particular advice topics (see, e.g., the GAC Montreal Communique at pages 8 and 9.. The ATRT3 suggestion regarding further improvements to the clarity of the Communique content is something that the GAC Leadership will explore closely with the hope that the Final Report will provide more clarity on this recommendation. | Implementation of the suggestions associated with the survey results are optional and as such are open to change. | ATRT3 Survey results and all suggestions associated with these have been removed from the main report and moved to Annex B. | Annex B of the ATRT3 Final Report | |||||||||||||||||||||
39 | GAC | 4.4.5 | The GAC acknowledges this suggestion, and requests that it be clarified. The joint face to face meetings of the Board and GAC and gatherings of the Board-GAC Interaction Group are all open, public, recorded and transcribed. The GAC Communique production effort, clarification process with the Board and ultimate scorecard response from the Board are all public. GAC advice status is regularly tracked and reported on both the GAC and ICANN.org web sites. Also, this ATRT review process in itself provides substantial notice to the community about improvements over the last couple of years with respect to GAC-Board interactions. In using the term “messaging”, does this ATRT3 suggestion contemplate expectations about certain reporting mechanisms or resources that need to be developed? More details on this suggestion in the Final Report would help clarify the expectations in this area. | Implementation of the suggestions associated with the survey results are optional and as such are open to change. | ATRT3 Survey results and all suggestions associated with these have been removed from the main report and moved to Annex B. | Annex B of the ATRT3 Final Report | |||||||||||||||||||||
40 | GAC | 4.4.6 | The GAC welcomes the ATRT3 assessment that the current GAC and Board mechanisms for interactions are deemed a “success”. Noting that they take place in a somewhat different context, the GAC will explore how those Board interaction mechanisms could inform and facilitate further interactions with the GNSO – and even potentially other communities. The leaders of the GAC and GNSO currently work to have at least one intersessional call between the two groups and community members currently interact on topics of similar interest in a number of crosscommunity environments. There may be mechanisms that can improve those interactions. | Implementation of the suggestions associated with the survey results are optional and as such are open to change. | ATRT3 Survey results and all suggestions associated with these have been removed from the main report and moved to Annex B. | Annex B of the ATRT3 Final Report | |||||||||||||||||||||
41 | GAC | Overarching | The GAC appreciates the acknowledgement of the ATRT3 that “the recommendations ICANN makes for the GAC via such processes as the ATRT reviews may have limited applicability or may have to be adapted to fit into the GAC context.” (see ATRT3 Draft Report at page 48). Whatever the substance or form of the final ATRT3 recommendation, the GAC will be mindful of that cautious overall acknowledgement, and asks that the ATRT3, if possible, clarify the meaning of the remark in the Final Report. | ATRT3 believes the GAC is principally referring to its Suggestions based on the results of the ATRT3 Survey and notes that Implementation of the suggestions associated with the survey results are optional and as such are open to change. | ATRT3 Survey results and all suggestions associated with these have been removed from the main report and moved to Annex B. | Section 2 and An Annex B of the ATRT3 Final Report. | |||||||||||||||||||||
42 | Heather Flanagan | 10.5 - Reviews | Rather than require a review team to try and do all its work in a 5 day workshop, which seems unreasonable and almost designed to result in a very shallow review, I personally would recommend that ICANN treat this as a more typical audit process and hire a firm that does audits for a living to evaluate if ICANN is following its own processes and procedures, and to make recommendations where those processes/procedures are insufficient or otherwise not being met. The final report from the audit company can be made available for public comment, thus keeping that public accountability open. It may be that an audit isn’t a correct model in all cases, but it certainly will fill some (e.g., SSR and WHOIS/RDS) but the idea has the potential to cut down on the number of review teams running concurrently, and given the contractual arrangement with an audit firm, you can set a more reasonable time limit (e.g., 6 months per review). | Given the many negative comments with respect to having Specific Reviews as workshops this notion was not retained. | Please read section 8.4 of the ATRT3 for complete details. | Section 8 of the ATRT3 Final Report. | |||||||||||||||||||||
43 | IPC | 10.5 - Reviews - Option 1 | The proposal for a newly created Independent Accountability Office would appear to create an unnecessary level of additional bureaucracy. Further, it would appear to pass responsibility for these accountability mechanisms away from the community (and each of the component parts of the community, being the SO/AC/SG/Cs) and to a single person/small team (which could be subject to lobbying or even capture). Such a change is unnecessary given that the structures and processes of the ICANN Board, staff and community are already in place to manage implementation of reviews. This proposal, as with the prioritization standing panel referred to above, seems to indicate a concerning desire on the part of the review team to move away from bottom-up structures in favour of a top-down approach. | Option 1 was not retained | Option 1 was removed from Section 8 of the ATRT3 Final Report. | Section 8 of the ATRT3 Final Report. | Section 10 of the ATRT3 final report. | ||||||||||||||||||||
44 | IPC | 10.5 - Reviews - Option 2 - Organizational Reviews | Organizational Reviews: The IPC would favour some streamlining, for example by adopting a maximum duration for such reviews. However, we strongly oppose the proposal that these organizational reviews be conducted as 3-5-day workshops focused on self-inspection: (Continued... see "Additional Details" tab) | ATRT3 has significantly evolved Option 2 into its recommendation on Reviews. | With respect to Organizational Reviews ATRT3 proposes: ATRT3 is proposing to move to a three tier system in its recommendation to evolve Organizational Reviews and to implement a continuous improvement program for SO/ACs: 1 An annual survey of members/participants in each SO/AC. The results of these would be public and used as input to the continuous improvement programs in each SO/AC as well as the Holistic review. 2 Evolving the current Organizational Reviews into reviews of the continuous improvement programs in each SO/AC (SO/AC Continuous Improvement Programs -SO/AC CIPs) to consider the results of the surveys of members/participants, assess the progress of the continuous improvement efforts (at least every three years) and producing a report which will feed into the Holistic reviews. 3 A Holistic review every 7 to 8 years to: Review continuous improvement efforts of SO/AC/NC based on good practices. Review the effectiveness of the various inter SO/AC/NC collaboration mechanisms. Review the accountability of SO/ACs, or constituent parts, to their members/constituencies (this will include an in depth analysis of the survey results). Review each SO/AC/NC as a whole to determine if they continue to have a purpose in the ICANN structure as they are currently constituted or if any changes in structures and operations are desirable to improve the overall effectiveness of ICANN as well as ensure optimal representation of community views (but taking into consideration any impacts on the Board or the Empowered Community). ATRT3 believes its recommendation ensures a better system for the improvement of SO/ACs and does so in a more regular and controlled fashion. | Section 8 of the ATRT3 Final Report. | Section 10 of the ATRT3 final report. | ||||||||||||||||||||
45 | IPC | 10.5 (Reviews - Option 2 - Specific Reviews) | The SRs serve specific functions, (should) have specific remits, and likely involve community experts and their educated input. These are among the most important tools for ensuring accountability and transparency across the multistakeholder community, including the Board and Staff. Combining the AT, CCT and RDS reviews (each of which, on its own, can be very deep and intricate) into a single review would create more problems than it solves – the remit of this review would be so expansive as to make it unworkable, exacerbating concerns about workload and the length of time such reviews take. The IPC does not favour the further complication of reviews in this manner. Instead, the SR’s should be maintained and improved upon. ICANN Org has repeatedly contended that it has fully implemented all past community review recommendations. We know, however, from recent assessments by the Review Teams of WHOIS/RDS, ATRT3 and SSR2, that prior reviews were not implemented fully. We would certainly favour clarity of scope and remit at the outset of such individual reviews to ensure that they do not duplicate work already under way or recently completed. Additional efficiencies could be achieved by developing a clear structure and timeline for all such reviews, and a pro-forma template report to be completed. | Given the many negative comments with respect to having Specific Reviews as workshops this notion was not retained. ATRT3 is not proposing to fundamentally change the format of Specific Reviews in its recommendation. However ATRT3 is proposing a moratorium on RDS and SSR Reviews going forward until the next ATRT can evaluate these properly. ATRT3 believes there should be one additional CCT review which would be limited in time and that ATRT reviews should remain quite similar to what they are now. Finally ATRT3 adds a Holistic Review to the set of Specific Reviews. | Please read section 8.4 of the ATRT3 for complete details. | Section 8 of the ATRT3 Final Report. | |||||||||||||||||||||
46 | IPC | 10.5 (Reviews) | In section 10.5, when considering recommendations to address concerns with the current review structure, the Draft Report specifically refers to the suggestions made by the SSAC in SSAC2018- 19 SSAC comments on Long-Term Options to Adjust the Timeline of Reviews. These suggestions appear to the IPC to offer sensible and proportionate changes that would maintain these important accountability checks while addressing concerns about workload and prioritization. These suggestions include: • Staggering reviews so that no more than one Specific Review (SR) and two Organizational Reviews (OR) run concurrently; • Ensuring that the next round of an SR or OR does not commence until prior recommendations have been implemented and operational for 12 months. We would caveat this with some maximum time period limitation to ensure that future reviews are not blocked by a failure to act on a prior review; • Adding a maximum duration for other SRs, similar to the 12-month time period for the AT Review. In applying such a maximum duration, we would suggest that real consideration be given to building-in a preparatory phase during which the precise terms of reference can be developed and approved, existing data is gathered by staff and any new data gathering can take place before the review team commences its review. This would remove time pressures such as those we have seen in ATRT3, where by the time the review team had finalised its ToRs and conducted its surveys, a significant portion of the 12-month term had already passed, putting pressure on both the review team and members of the community who would wish to comment on an extremely lengthy Draft Report. • Focusing SRs on topics of highest priority; • SRs should have a pre-established budget; • Allowing scheduling flexibility for SRs, with appropriate checks and balances. This would ensure that where some other workstream is underway (for example in response to a legislative change) then a SR on the same subject matter might be postponed if the community agrees that this is appropriate. Although the Draft Report refers to these SSAC suggestions, it appears not to propose to adopt them, without any explanation as to why these have been rejected, and instead proposes two completely different, and quite extreme, alternatives – neither of which the IPC supports. Our comments on these two alternatives are as follows: | ATRT3 has significantly evolved Option 2 into its recommendation on Reviews which incorporates many of these elements. | Please read section 8.4 of the ATRT3 for complete details. | Section 8 of the ATRT3 Final Report. | |||||||||||||||||||||
47 | IPC | 12.4.1 - Requirements for a Prioritization Process | The IPC agrees that a number of the proposed listed requirements for a prioritization process seem sensible and should help the community to prioritize its workload and the implementation of the many recommendations of past reviews, in addition to likely future recommendations. In particular, the IPC supports: • The use of the Operating Standards for Specific Reviews as a base; • Work to be conducted in an open and transparent fashion, with each decision being justified and documented; and • That where there are multi-year implementations these should be reviewed annually to ensure that they still meet their implementation objectives and the needs of the community. (Continued... See 'Additional Details' tab) | N/C | Some refinements in Section 10 of the ATRT3 final report. | Section 10 of the ATRT3 final report. | |||||||||||||||||||||
48 | IPC | Overarching | The use of Skype channels. It would appear that the review team has made use of Skype channels for substantive discussion on the work being conducted, as opposed to limiting such means of communication to purely administrative matters such as the fixing of time for calls. This has made it difficult to track the work of ATRT3, as the mailing list has had limited traffic. This is contrary to the Operating Standards for Specific Reviews and would be a concern for any review – but particularly for a review of accountability and transparency.Weight placed on survey responses. The ATRT3 acknowledges that the responses to the Survey for Individuals are not statistically significant but still relies heavily on data from this survey. In addition, a number of the questions asked in the surveys were loaded Yes/No opinion questions that do not give rise to a clear actionable outcome. For example, a question about the level of satisfaction with interaction with the Board does not identify for the review team that those who expressed dissatisfaction are actually dissatisfied in the same way, for the same reason. The IPC does believe that there is a place for such attitudinal surveys but if they are to be used then they should be expertly drafted in order to ensure they deliver meaningful results. Whilst there is clearly an up-front cost to this it would be a worthwhile investment since such a survey could then be used at intervals in order to measure changing attitudes (and hopefully the impact of implementation of recommendations). The statistical significance and surety of survey results would be a true community benefit. | Noted - However ATRT3 would like to note that no significant decisions with respect to its report were made using any type of chat system and that most of the work was done during calls. | None | N/A | |||||||||||||||||||||
49 | ISOC Cameroon | 10.5 - Option 2 - Organizational Reviews | With regard to the reviews and the two options, we support the latter, in particular because the notion of improvementis absolutely relevant. | ATRT3 has opted to make a recommendation based on Option 2 which includes a continuous improvement component which should meet what is being proposed in the comment. | Please read section 8.4 of the ATRT3 for complete details. | Section 8 of the ATRT3 Final Report. | |||||||||||||||||||||
50 | ISOC Cameroon | 10.5 - Option 2 - Specific Reviews | With regard to specific reviews we agree with the proposals made in the report but we believe that one or more new journals could be created, for example on the issue of DNS abuses. These will be conducted under a groupof workshops of 3 to 5 days. We suggest merging the organization holistic review projects with ATRTx to allow for a completely comprehensive vision ofICANN once every 7 years. | Based on the comments ATRT3 chose to limit its changes to Specific Reviews | Please read section 8.4 of the ATRT3 for complete details. | Section 8 of the ATRT3 Final Report. | |||||||||||||||||||||
51 | ISOC Cameroon | 11.4.2 Accountability Indicators | Finally, with regard to the liability indicators currently published on the ICANN website, we suggest that future indicators be useful, up-to-date and for this reason they should be defined on the basis ofICANN org's proposal. community. | ATRT3 significantly evovled its thinking on Accountability Indicators into a recommendation on "Accountability and Transparency Relating to Strategic and Operational Plans including Accountability Indicators" which should continue to meet the intent of the comments made. | Significant, please see section 9.4 of the ATRT3 Final Report | Section 9 of the ATRT3 Final Report. | |||||||||||||||||||||
52 | ISOC Cameroon | 12.4 - Developing a Prioritization Process | The proposal that prioritisation must be carried out in an open and transparent manner and every decision must be justified and documented seems useful, but with currently more than 200 recommendations to prioritize is not too much asked and will it not lengthen the time frame too much? It would also be useful to add to the criteria to use that of the needs/priorities of the various ICANN players. | ATRT3 has converted this suggestion into a recommendation in its Final Report while refining it. | Some refinements in Section 10 of the ATRT3 final report. | Section 10 of the ATRT3 final report. | |||||||||||||||||||||
53 | ISOC Cameroon | 5.4.2 Public Comments | As part of the public comment procedure, icann's translations of a summary and key issues into ICANN's official languages are very important improvements. It should also open the possibility of organizing exchanges/debates based on previouscomments (such as a collective chat). | This has been made into a formal recommendation by ATRT3. A number of refinements were made in converting this into a recommendation. The idea of a chat has not been incorporated into the ATRT3 recommendation. | Yes but in keeping in the spirit of the original suggestion. | Section 3 of the ATRT3 Final Report. | |||||||||||||||||||||
54 | ISOC Cameroon | 9.4.1 - ATRT2 recs | We support ATRT3's proposals following its analysis of the implementation of the ATRT2 recommendations. In particular their retroactive uses for CCT1, RDS2 and WS2. It is important that for each recommendation, the person designated to liaise with the review team is regularly consulted by ICANN org. implementation. | This was originally only a suggestion which has been upgraded into a recommendation. | Yes but in keeping in the spirit of the original suggestion. | Section 7.4 of the ATRT3 Final Report. | |||||||||||||||||||||
55 | ISOC Cameroon | Other - Election of Board members | Secondly, foryears the terms of reference concerning the evaluation and improvement of the governance of the Board of Directors, unless we are mistaken or omission, it seems to us that certain matters of high importance have not been addressed. Thus, under the effectiveness and effectiveness sectionof ICANN's Board of Directors, there are no details on the composition of the Board of Directors,which would, among other things, test whether diversity issues (especially gender- have been taken into account. Also, clear and transparent rules must be enacted for the process of electing to the Board of Directors. | Selection of Board members is not in the remit of the ATRT3 although this was indirectly considered in the ATRT3 survey question dealing with diversity on the Board which resulted in a suggestion. As to clear and transparent rules for the selection of Board members these are properly spelled out in the ICANN Bylaws. | N/A | N/A | |||||||||||||||||||||
56 | ISOC Cameroon | Recommendations and suggestions | To get to the heart of the matter, first of all, we have a problem with the nomenclature, the terms recommendations and suggestions are not well specified as to their definition and especially as to their content. Based on the intricacies of language, recommendations always have a greater force of constraint than suggestions, and recommendations are generally required to take into account, while suggestions are not very restrictive. It should be clarified at this level; if the two words refer to the same content for you, then it is better to choose only one, depending on the content and the strength you want to give (more or less binding). | ATRT 3 has taken on these comments in only making 5 recommendations and leaving all suggestions as optional. | In its introduction ATRT3 states - "In a context where there are 325 review recommendations awaiting approval or implementation ATRT3 has chosen to be pragmatic and effective in making recommendations. Although ATRT3 makes both recommendations and suggestions it only requires the implementation of its 5 recommendations. Suggestions are meant to be exactly that “suggestions” and it is left to those concerned by these individual suggestions, which can be found in Annexes A and B of this report, to decide if they should, or not, be implemented.". In keeping with this ATRT3 has moved all suggestions related to its Survey results out of the main report into Annex B. | Introduction | |||||||||||||||||||||
57 | Laurin Weissinger | 10.5 - Option 2 - Specific Reviews | The aim (option 2) to reduce the SSR review to short workshops is concerning. While the OCTO team is doing a lot of excellent work (e.g. outreach, engagement, stakeholder support, etc.) and SSAC and RSSAC contribute important advice, there are systemic security issues within the DNS and the ICANN ecosystem that need to be addressed. For example, ICANN contracts do not properly address systemic abuse (https://www.icann.org/en/system/files/correspondence/hedlund-to-vayra-04apr18-en.pdf, p.2). Issues with ICANN’s approach to security and anti-abuse are also documented by multiple review teams. It is obvious that improving on security is paramount and requires proper oversight and/or audit mechanisms. Many approaches to addressing security vulnerabilities and lacking anti-abuse provisions do exist. Likewise, many options exist for overseeing the process — but the key to all of them is to have actual oversight, transparency, and consequences / actions to address SSR concerns. Continuous assessment and audits by independent third parties, reviewed by the community in intervals or partly “staffed” by community volunteers, could be an approach as well as shorter but more frequent reviews on SSR, e.g. twice per year. In the latter case, having overlapping terms (at least 1 year overlap, better 1.5/2) to ensure knowledge transfer would be useful. Whatever is finally recommended should be fleshed out more and explain how the future process could uphold a level of oversight and transparency appropriate for such an important concern. | Given the many negative comments with respect to having Specific Reviews as workshops this notion was not retained. With respect SSR reviews given the issues associated with SSR2 and the fact that SSR2 will not be completed prior to ATRT3 completing its work (see section 8.3 of the ATRT3 Final Report) ATRT3 is recommending that future SSR reviews be suspended until the next ATRT review can consider the results and impact of SSR2 as well as the needs of such a review going forward. | Please read section 8.4 of the ATRT3 for complete details. | Section 8 of the ATRT3 Final Report. | |||||||||||||||||||||
58 | NCSG | Option 1 | 8. We welcome the ATRT3 efforts in attempting to set criteria for the proposed reviews; however, this Stakeholder Group has unsure feelings towards the Options 1 and 2 scopes presented on the draft by the ATRT3. Both of the options fail in addressing the problem of community burdensome, since the Option 1 opts for keeping the set of Specific and Organizational Reviews as already is in contrast with Option 2 that tries to consolidate distinct Specific Reviews into a single one, which would not be manageable, in practice, to community accountability. 11. Moreover, Option 1 suggests an oversight mechanism that should be responsible for auditorship the implementation of the recommendations of the reviews done by the community. While this would reduce the responsibility of the community, duplication of efforts, and be an efficient way to guarantee that ICANN implemented the recommendations without having a holistic review, for example, this oversight mechanism could also present problems if it's an external reviewer. 12. In that situation, an external reviewer could, in many cases, have an approach that would be highly dissociated from ICANN's reality and the community's perspective. It's unlikely that a consulting/compliance firm, for example, would take into consideration the aims and interests of each Stakeholder Group when conducting this process, and that is why this Independent Accountability Office if implemented, must be composed by ICANN Org staff members. | Option 1 was not retained due to low support. ATRT3 believes its recommendation for reviews does address the issue of burdent on SO/ACs. As to the ensuring the implementation of review recommendations ATRT3 believes the new Operating Standards for Specific reviews, the new website for reviews as well as ATRT3's recommendation on Prioritization will ensure adequate oversight to ensure implementation. For further details on issues relating to the implementation of recommendations ATRT3 would refer to section 8.3 of its final report | Option 1 was removed from Section 8 of the ATRT3 Final Report. | N/A | |||||||||||||||||||||
59 | NCSG | Option 2 | 9. Furthermore, Option 2 also presents problems when addressing the Organizational Reviews. It's unreasonable to believe that the community will be able to conduct a whole review process, which sometimes takes a whole year into a 5 (five) days workshop, even if focused on SO/AC self-inspection in the context of continuous improvement 10. That would significantly reduce the quality and legitimacy of the content and discussions that takes on during the drafting and evaluation period of these reviews. Also, even if it kept the quality and the scope reduced, it would be at the expense of the community tireless effort during these intense five days workshops that could cause burnouts. Regarding the " Recommendation concerning Specific and Organizational Reviews, " the ATRT3 should avoid the Option 2 as a solution to the problem addressed, since it would sharply reduce the quality of the discussions and work produced, and should also focus on improving the Option 1 proposition by ensuring that the Independent Accountability Office if implemented, would not be composed by external reviewers of ICANN's Community and ecosystem. | ATRT3's recommendation with respect to organizational reviews proposes a three tier system where the 3-5 days reviews are only one option for the SO/ACs. As to the ensuring the implementation of review recommendations ATRT3 believes the new Operating Standards for Specific reviews, the new website for reviews as well as ATRT3's recommendation on Prioritization will ensure adequate oversight to ensure implementation. For further details on issues relating to the implementation of recommendations ATRT3 would refer to section 8.3 of its final report | ATRT3 made significant improvements to its proposal from Option 2 to created its recommendation on Reviews. Please see section 8.4 of the ATRT3 Final Report. | Section 8 of the ATRT3 Final Report. | |||||||||||||||||||||
60 | NCSG | Prioritization | 18. Aside from that, we believe that two other points must be addressed: (i) over the years, this community and the ICANN Org have also faced cases of initiated reforms not being adequately finished due to reasons such as resource allocation and (ii) there are ongoing processes, such as ICANN Multistakeholder Model Review, that could be harmful and possibly result in a duplication of efforts if the recommendations developed for WS2 accountability, for example, are not fully implemented or adopted. 19. The raised points above must be taken into consideration if the community decides for creating this community-led entity since there would be a significant fund allocation due to the one-year deadline presented to complete the tasks assigned, and the prioritization should take into account the already existing recommendations to be implemented before focusing on other reviews. b. Regarding the " Suggestion concerning prioritization towards activities, policies, and recommendations ," the ATRT3 should re-think the community-led entity process since it can lead to a greater community burden. We suggest that the prioritization should be made inside the review process since it would save resources allocation and avoid duplication of effort. But if not possible, the ATRT3 should aim for a more friendly, and longer period term that would not collide with other significant reviews. | As pointed out by the Board "Neither the Bylaws nor the Operating Standards, however, provide a clear and consistent methodology for formulating effective review team or cross-community recommendations, nor do they provide a basis for evaluating resource requirements associated with such recommendations, prioritizing recommendations across the universe of review teams and cross -community working groups, or for budgeting for prioritized recommendations.". Additionally ATRT3 in its final report per its recommendation on prioritization that "Additionally, ATRT3 wishes to align its recommendation with the efforts currently underway to develop a prioritization system to avoid conflicting recommendations or duplication of work. As such, ATRT3 has opted to provide some high-level guidance for the proposed prioritization process. ATRT3’s starting point was the following section from the ICANN Board Paper on Resourcing and Prioritization of Community Recommendations: Draft Proposal for Community Discussion: Section 5 B - “The ICANN community and ICANN org will collaboratively develop a methodology for prioritizing recommendations across review teams and for funding implementation of prioritized recommendations as part of the annual budget process. This methodology will be consistent with the existing budget development process, including the solicitation and consideration of community input. See also the discussion in Section 4 on prioritization.” | Refinemants were made and can be found in section 10.4 of the ATRT3 Final Report. | Section 10 of the ATRT3 Final Report | |||||||||||||||||||||
61 | RrSG | 10.5 - Option 2 - Organizational Reviews | In previous comments the RrSG noted that addressing time and budget concerns should be done primarily through changes to the Organisational Review process and that Specific Reviews would struggle with having restricted 12 month timelines, due to the pressure it would place on community volunteer participants. In other previous comments the RrSG also advocated that Organizational Reviews should lead to continuous lightweight self-review and allow the ring-fencing of foundational elements. Overall, the RrSG therefore believes Option 2 to be the best way forward for Organizational Reviews, for its emphasis on self-review and continuous improvement. However, this is dependent on the self-review workshops and overarching ‘holistic’ review done after seven years being overseen by an independent professional and its recommendations being limited, but appropriately implemented by the organization. Any recommendations should be valid and implementable, so it should be known how long it will take, what it will impact, what the budget will be and when it should be finished. To ensure a reasonable use of ICANN’s budget, the self-review workshops should also be less than three days if the SO/AC deemed it necessary to hold them more than every three years and participation should be limited to those necessary to make the decisions/recommendations. | ATRT3 believes its recommendation with respect to reviews continues to meet these suggestions overall when considering that the Holistic Review will be a Specific Review which will insure independence of the reviewers by having them nominated by all SO/Acs and their recommendations will be bound by the new Operating Procedures of Specific Reviews and subject to the ATRT3 recommended Prioritization Process | Please read section 8.4 of the ATRT3 for complete details. | Section 8 of the ATRT3 Final Report. | |||||||||||||||||||||
62 | RrSG | 10.5 - Option 2 - Specific Reviews | With regards to Specific Reviews, the RrSG would also support Option 2, but notes its concerns about potentially limiting the timeline to as little as 12 months. Although limiting a timeline can be productive by enforcing efficiency, there may need to be some flexibility depending on the nature of the Specific Review If the AT review is intended to encompass relevant portions of CCT & RDS as well, and given that after 7 years it would likely involve different community members and therefore lose some historical knowledge, the burden on volunteers to quickly become educated in all relevant matters would be significant. The RrSG would also question what the process is intended to be for any new Specific Review that was required in response to a future issue and going forward how it would be decided whether a Specific Review should be done every 7 years like the AT review, or through a workshop or traditional review process, as with SSR reviews. | ATRT3 is not proposing to fundamentally change the format of Specific Reviews in its recommendation. However ATRT3 is proposing a moratorium on RDS and SSR Reviews going forward until the next ATRT can evaluate these properly. ATRT3 believes there should be one additional CCT review which would be limited in time and that ATRT reviews should remain quite similar to what they are now. Finally ATRT3 adds a Holistic Review to the set of Specific Reviews. | Please read section 8.4 of the ATRT3 for complete details. | Section 8 of the ATRT3 Final Report. | |||||||||||||||||||||
63 | RrSG | 12.4.1 - Requirements for a Prioritization Process | The RrSG is supportive of the ATRT3’s suggested Prioritization Process, as greater coordination and oversight should certainly have a positive impact on volunteer workload, timelines and efficiency. The RrSG believes a professional and neutral facilitator should be used by default, to ensure excessive time (and therefore resources) are not being wasted on handling the differences that inevitably come out of multistakeholder discussions. There should also be a liaison or representative member from GNSO Council in the Prioritization Process, to ensure coordination with the organisation and prioritisation of PDP work. | ATRT3's proposal offers the possibility of all SO/ACs participating in a process that is part of the standard budgeting process which should ensure efficiency. | Some refinements in Section 10 of the ATRT3 final report. | Section 10 of the ATRT3 final report. | |||||||||||||||||||||
64 | RrSG | 3.4.6 | The RrSG notes the survey results indicating an overall dissatisfaction with the ICANN Board’s performance, communication and interactions, but is pleased to see the ATRT3 suggestion that when meeting with community groups, the ‘interactions be less formal and allow sufficient time for a true dialogue on questions of interest to those community members.’ as this is in line with the Contracted Parties House’s (CPH) current efforts to change the format of our meetings with the ICANN Board, which we are looking to trial at ICANN67 in Cancun. The RrSG believes ICANN Board members having broader communication with contracted parties and a greater understanding of how the domain name industry operates will greatly assist their decision making. | Implementation of the suggestions associated with the survey results are optional and as such are open to change. | ATRT3 Survey results and all suggestions associated with these have been removed from the main report and moved to Annex B. | Annex B of the ATRT3 Final Report | |||||||||||||||||||||
65 | RrSG | 3.4.9 | The RrSG has previously commented on multiple occasions about the need for greater transparency and context with regards to financial budgeting. Therefore, the RrSG also supports the ATRT3’s suggestion to ‘tailor budget information for SO/ACs so that they can easily understand budgeting relative to SO/ACs.’ and that ‘A clear rationale in simple language explaining key decisions should be included in these materials’. | ATRT3's proposal for prioritization should indirectly assist in addressing these points. Additionally the Implementation of the suggestions associated with the survey results are optional and as such are open to change. | No changes to the suggestion | Section 10 and Annex B of the ATRT3 Final Report. | |||||||||||||||||||||
66 | RrSG | Overarching | In conclusion, the RrSG is grateful for the ATRT3’s efforts and fully supportive of their suggestions introduce measures to reduce the frequency of Reviews, introduce Organisational self-review and set up a Prioritization Process, provided that independent and professional facilitation is utilised to ensure the greatest adherence to the process and efficiency. | ATRT3 believes its recommendation with respect to reviews continues to meet these suggestions overall. | Yes but in keeping in the spirit of the Option 2. | Section 8.4 of the ATRT3 Final Report. | |||||||||||||||||||||
67 | Russ Housley | 10.5 - Option 2 - Specific Reviews | In my view, a Specific Review could be conducted in less than a year using a schedule roughly like this: (see 'Additional Details' tab) | ATRT3 is not proposing to fundamentally change the format of Specific Reviews in its recommendation but did incorporate the concept of preparation made in this comment. However ATRT3 is proposing a moratorium on RDS and SSR Reviews going forward until the next ATRT can evaluate these properly. ATRT3 believes there should be one additional CCT review which would be limited in time and that ATRT reviews should remain quite similar to what they are now. Finally ATRT3 adds a Holistic Review to the set of Specific Reviews. | Please read section 8.4 of the ATRT3 for complete details. | Section 8 of the ATRT3 Final Report. | |||||||||||||||||||||
68 | RySG | 10.4.1 - Reviews | The RySG is unclear why ATRT3 is suggesting an Implementation Shepherd for their work given this is set out in the Operating Procedures applicable to their review. The RySG is also unclear why ATRT3 is making a new suggestion instead of suggesting the full implementation of the ATRT2 recommendation. | ATRT3's new recommendation on incomplete ATRT2 recommendations meets this suggestion. Implementation of ATRT2 recommendations will have to be reviewed by ICANN Org. and the ATRT3 implementation shepherds and then go through prioritization. | See implementation details in Section 7.4 of the ATRT3 Final Report. | Section 7 and Annex A of the ATRT3 Final Report | |||||||||||||||||||||
69 | RySG | 10.4.2 - Reviews | While the RySG supports this common-sense project management suggestion, we are unclear what issue this is intended to address, since the ATRT2’s recommendation has been implemented. At a minimum we think this should be a new recommendation, accompanied by its own rationale so future ATRTs can determine completion status. | Implementation of ATRT2 recommendations will have to be reviewed by ICANN Org. and the ATRT3 implementation shepherds and then go through prioritization. | None | Section 7 and Annex A of the ATRT3 Final Report | |||||||||||||||||||||
70 | RySG | 10.4.3 - Reviews | The RySG supports this suggestion. However, the RySG would encourage ATRT3 to be more specific about what is meant by “time frame”. Is this a general “as soon as possible vs low priority”, or is it “within 12 months” timeframe? | The ATRT3 proposal for the implementation of incomplete ATRT2 recommendations is: “ICANN Org. shall review the implementation of ATRT2 recommendations in light of ATRT3’s assessment of these and complete their implementation subject to prioritization (see recommendation on the creation of a prioritization process).” Given ATRT2 recommendation 11.7 will be subject to the Prioritization process and that it overlaps and conflicts with the ATRT3 Prioritization process it is expected that this new process would consider that it has overtaken recommendation 11.7 and that 11.7 can be retired given this is one of the options available to the Prioritization process. | No change | Sections 7, Section 10 and Annex A of the ATRT3 Final Report | |||||||||||||||||||||
71 | RySG | 10.5 - Option 1 | The RySG does not support Option 1 as it does not represent a significant departure from the status quo. In fact, the two options presented here seem like somewhat extreme alternatives, and the RySG believes that there may be other options that offer a middle path, where the system of Organizational and Specific Reviews could be improved without a drastic overhaul. The SSAC comments cited on p. 91 of the Draft Report offer a number of improvements to the current Reviews system that the RySG supports. For example, the RySG is in favor of limiting - or at the very least, providing guidelines on - the duration of all reviews and believes that instituting certain work methods can help enable Review Teams to meet those timeframes. In previous comments on the “Long-Term Options to Adjust the Timeline of Reviews,” the RySG expressed strong support for the concept of limiting the duration of Reviews to 12 months. We note, however, that the ability to meet such a deadline will likely require adjustments to the way Review Teams have historically undertaken their work, and we believe the ATRT3 should offer suggestions in this regard. Russ Housely offers some useful recommendations in his comment on the Draft Report, such as having ICANN Staff provide Review Teams with detailed documentation about the implementation status of prior review recommendations at the outset, and using face-to-face meetings coincidental with ICANN meetings to tackle early work such as establishing Terms of Reference. While conducting Organizational Reviews in a workshop format may be suitable (and offer improvements) for Organizational Reviews, we are skeptical that such a format would be appropriate for the SSR Review, which generally requires research and analysis. | Option 1 was not retained | Option 1 was removed from Section 8 of the ATRT3 Final Report. | Section 8 of the ATRT3 Final Report. | |||||||||||||||||||||
72 | RySG | 10.5 - Option 2 - Organizational Reviews | The RySG does not support Option 1 as it does not represent a significant departure from the status quo. In fact, the two options presented here seem like somewhat extreme alternatives, and the RySG believes that there may be other options that offer a middle path, where the system of Organizational and Specific Reviews could be improved without a drastic overhaul. The SSAC comments cited on p. 91 of the Draft Report offer a number of improvements to the current Reviews system that the RySG supports. For example, the RySG is in favor of limiting - or at the very least, providing guidelines on - the duration of all reviews and believes that instituting certain work methods can help enable Review Teams to meet those timeframes. In previous comments on the “Long-Term Options to Adjust the Timeline of Reviews,” the RySG expressed strong support for the concept of limiting the duration of Reviews to 12 months. We note, however, that the ability to meet such a deadline will likely require adjustments to the way Review Teams have historically undertaken their work, and we believe the ATRT3 should offer suggestions in this regard. Russ Housely offers some useful recommendations in his comment on the Draft Report, such as having ICANN Staff provide Review Teams with detailed documentation about the implementation status of prior review recommendations at the outset, and using face-to-face meetings coincidental with ICANN meetings to tackle early work such as establishing Terms of Reference. While conducting Organizational Reviews in a workshop format may be suitable (and offer improvements) for Organizational Reviews, we are skeptical that such a format would be appropriate for the SSR Review, which generally requires research and analysis. | ATRT3 Believes its recommendation on Reviews strikes the right balance when considering all comments and hopes the RySG can support this approach. | Section 8.4 of the ATRT3 Final Report. | Section 8 of the ATRT3 Final Report. | |||||||||||||||||||||
73 | RySG | 11.4.1 Accountability Indicators | The RySG supports this suggestion. | ATRT3 significantly expanded this recommendation to deal with the Accountability and Transparency Relating to Strategic and Operational Plans including Accountability Indicators | Significant, please see section 9.4 of the ATRT3 Final Report | Section 9 of the ATRT3 Final Report. | |||||||||||||||||||||
74 | RySG | 11.4.2 Accountability Indicators | The RySG generally supports this suggestion but is concerned that it lacks specificity. We suspect ICANN would mark at least items 2-4 as complete. We suggest a “SMART” recommendation that ties the recommendations to the problem to be solved so that staff will know when they’ve hit the target. We additionally observe that any review of the Accountability Indicators be dependent on the outcome of a communications exercise with the community on what the Accountability Indicators are and what purpose they serve. If concerns are raised during the communications exercise across ICANN Structures, then ICANN may wish to consider reviewing the Indicators. Generally, the RySG supports that the data set out in the Accountability Indicators is meaningful and notes that we currently have a group focused on improving the usefulness and quality of the DAAR data. | ATRT3 found a number of significant issues with the accountability Indicators - see Annex C. ATRT3 significantly expanded this recommendation to deal with the Accountability and Transparency Relating to Strategic and Operational Plans including Accountability Indicators | Significant, please see section 9.4 of the ATRT3 Final Report | Section 9 of the ATRT3 Final Report. | |||||||||||||||||||||
75 | RySG | 12.4.1 - Developing a Prioritization Process | The RySG deeply appreciates the work the ATRT3 has done to try to tackle this complex topic. We agree that prioritization is critical to control costs, complete work in a timely manner, avoid volunteer burnout, and ensure projects don’t bleed into one another. We think to the extent work is prioritized today it’s prioritized by the budget and the various Councils (and, to some extent, the Board). We unfortunately see the formation of yet another standing- type committee of ICANN insiders as another layer of separation from the bottom-up multistakeholder process and are particularly concerned that the committee might be co-opted by long-time industry insiders without the benefit of fresh new perspectives. Additionally, the committee will further slowdown work by requiring a year for prioritization. Instead the RySG, in line with our suggestions to the Evolving MSM report, suggests stronger controls at the SO/AC level, better recommendations and project scoping (including budgeting), and smaller (and better-managed) projects. We particularly think this ATRT3 has made good suggestions, including having a single community-wide website for projects. The GNSO’s PDP 3.0, this ATRT3’s recommendations for streamlined reviews, and the RySG’s MSM comment to design “ongoing work as a spiral, with small concrete projects that people can participate in as time allows but that overlap so that we aren’t making decisions in a vacuum,” all will feed into a more natural and streamlined prioritization process. We hope that these various, community-wide, incremental improvements mean we won’t need a committee of insiders to tell the community what to work on. | ATRT3 notes that the new Operating Standards for Specific Reviews should address many of the concerns with respect to these reviews going forward. ATRT3 also notes that its recommendation to evolve Organizational reviews into a three tier set of reviews which also implement an SO/AC based continuous assessment program should also address a number of the concerns. ATRT3's recommendationon for a Prioritization Process is simply a last step, once the backlog has been cleared, to allow the community to decide on its priorities. | See sections 8 and 10 of ATRT3's final report. | Sections 8 and 10 of the ATRT3 final report. | |||||||||||||||||||||
76 | RySG | 12.4.2 | The RySG supports the use of plain language in ICANN documents. However, the RySG notes that engaging in the budget consultation does require an interest in engaging and basic financial literacy. Further, the use of plain language should not undermine the nuance and often specialized content of documents. | Implementation of ATRT2 recommendations will have to be reviewed by ICANN Org. and the ATRT3 implementation shepherds and then go through prioritization. | None | Section 7 and Annex A of the ATRT3 Final Report | |||||||||||||||||||||
77 | RySG | 12.4.3 | The RySG supports this suggestion and appreciates the specific instructions so that Staff can follow through. | Implementation of ATRT2 recommendations will have to be reviewed by ICANN Org. and the ATRT3 implementation shepherds and then go through prioritization. | None | Section 7 and Annex A of the ATRT3 Final Report | |||||||||||||||||||||
78 | RySG | 3.4.1 | The RySG supports these recommendations, but for the reasons noted below, do not believe these recommendations belong linked to ATRT2 Recommendation 2. We believe ATRT2 Recommendation 2 is completed. When we follow the issue back, we note that the recommendations do not address the issues flagged by any of the ATRTs. They do address several survey concerns and definitely apply below, but do not address any of the issues surrounding metrics related to Board training materials. We note that the way the ATRT3 structured its report, the RySG had to go back and read and compare the analysis in the ATRT2 report to get a complete picture of the issues presented. We strongly suggest the ATRT3 map recommendations flowing from ATRT1 through ATRT2 to ATRT3 so the community can track the work. We also strongly encourage this and future RTs to close issues where the recommendations are implemented but could be improved with minor adjustments. The minor adjustments could form the basis for new “improvement” recommendations. | Implementation of ATRT2 recommendations will have to be reviewed by ICANN Org. and the ATRT3 implementation shepherds and then go through prioritization. | None | Section 7 and Annex A of the ATRT3 Final Report | |||||||||||||||||||||
79 | RySG | 3.4.10 | The RySG supports this suggestion. The RySG would suggest that any review or evaluation of the ITI in the future use the Board resolution which established the ITI to assess its effectiveness https://www.icann.org/resources/board-material/resolutions-2017-09-23-en#2.c. | Implementation of the suggestions associated with the survey results are optional and as such are open to change. | ATRT3 Survey results and all suggestions associated with these have been removed from the main report and moved to Annex B. | Annex B of the ATRT3 Final Report | |||||||||||||||||||||
80 | RySG | 3.4.2 | The RySG cautiously supports the first recommendation as we think it will further assist in prioritization. We do believe the scope of this project will be significant and may be costly. We are not sure how the second recommendation would be implemented as we aren’t sure what would constitute a “request” from an SO and who would decide what actions are required. | Implementation of ATRT2 recommendations will have to be reviewed by ICANN Org. and the ATRT3 implementation shepherds and then go through prioritization. | None | Section 7 and Annex A of the ATRT3 Final Report | |||||||||||||||||||||
81 | RySG | 3.4.3 | The RySG is unclear on how this recommendation has not been implemented. A redaction register, along with a uniform redaction policy is available here: https://www.icann.org/resources/pages/publication-practices-2016-06-30-en. | There is no uniformity in the redaction standards of various types of documents which is what this is focusing on. | None | Section 7 and Annex A of the ATRT3 Final Report | |||||||||||||||||||||
82 | RySG | 3.4.4 | The RySG supports this suggestion, but would note that since the ATRT2 recommendation was for the Board to respond in a timely manner then, unless there is serious concern that this has not been happening, being prescriptive as to the actual time period may not really be necessary. | Implementation of ATRT2 recommendations will have to be reviewed by ICANN Org. and the ATRT3 implementation shepherds and then go through prioritization. | None | Section 7 and Annex A of the ATRT3 Final Report | |||||||||||||||||||||
83 | RySG | 3.4.5 | The RySG does not support this suggestion as it is vague and not measurable. For example, it is unclear what the ATRT3 means by “enhance”. Further, the RySG notes that the Fellowship Program has recently been reviewed and the NextGen Program is currently being reviewed. We support the specific recommendation to improve captioning. | Noted - Implementation of ATRT2 recommendations will have to be reviewed by ICANN Org. and the ATRT3 implementation shepherds and then go through prioritization. | None | Section 7 and Annex A of the ATRT3 Final Report | |||||||||||||||||||||
84 | RySG | 3.4.6 | The RySG supports this recommendation, but finds that it lacks logical context between the problem and the suggestion. We suggest one way to make that connection below. Problem statement: There are mixed views from SO/ACs about their current interactions with the Board, where some view the interactions as useful whilst others do not. Whilst the survey question indicates some dissatisfaction, it does not really assist in identifying what that dissatisfaction is – are meetings too formal, not formal enough, is there insufficient access to Board between ICANN meetings, or are there concerns that in the past, when ICANN was smaller and less formal, everyone knew each other and so Board members were more visible? It could be any or all of these things, and likely is not the same for each group depending on their actual form of interaction. RySG Comment - Third Accountability and Transparency Review Team (ATRT3) Draft Report (January 2020) 10/31 This presents a need to retain the current approach for the structures that find their interaction with the Board useful and develop greater flexibility in how to engage with the Board for the structures do not find the interaction useful. For the purpose of ATRT3, our recommendation is focused on Board and SO/AC interaction at ICANN meetings. If SO/ACs wish to interact with the Board between ICANN meetings, we encourage them to reach out to the Board to explore options for doing so. Recommendation: ● Individual structures and sub-structures provide the Board with a request for how they wish to engage at the upcoming ICANN meeting along with the topic or topic(s) they wish to discuss. Examples of “how” include: small group discussions, formal Q&A with prepared responses, a presentation to the Board to educate them on a new topic, a request for the Board to present to the structure on a new project. ● This request could be made on the same timeline as the current request for questions from the Board in advance of ICANN meetings. Alternatively, the Board may wish to seek input sooner to allow them additional time to prepare. ● This would be the same process if structures wish to meet with a specific group of Board members (e.g., SO-appointed Board members or a Board committee). ● This is not intended to prevent or deter the Board or Board members from suggesting topics or formats for interactions at ICANN meetings. ● This recommendation is an alteration to current practice and should require minimal resources to implement. ● This should be implemented at the next ICANN meeting following the adoption of the recommendation by the Board. ● To assess whether this change alters the effectiveness of Board and SO/AC interactions, the Board and SO/ACs should make an effort to share with the other, formally or informally, whether they found their engagement useful after an ICANN meeting has concluded. | Implementation of the suggestions associated with the survey results are optional and as such are open to change. | ATRT3 Survey results and all suggestions associated with these have been removed from the main report and moved to Annex B. | Annex B of the ATRT3 Final Report | |||||||||||||||||||||
85 | RySG | 3.4.7 | The RySG notes that ATRT3 seems to have limited its focus on diversity to gender diversity, even though survey responses pointed to geographical, gender, stakeholder, and experiential diversity. We would like to see the suggestion broadened to include the types of diversity the community should consider. | Implementation of the suggestions associated with the survey results are optional and as such are open to change. | ATRT3 Survey results and all suggestions associated with these have been removed from the main report and moved to Annex B. | Annex B of the ATRT3 Final Report | |||||||||||||||||||||
86 | RySG | 3.4.8 | The RySG supports this recommendation, but again notes no obvious connection between the survey question and the recommendation. We suggest that the Final Report would likely benefit from the inclusion of a problem statement which this recommendation is intended to address, and provide the following example language: “Although Structures seem largely aware of the Board training program, the same does not appear to hold true for individuals.” | Implementation of the suggestions associated with the survey results are optional and as such are open to change. | ATRT3 Survey results and all suggestions associated with these have been removed from the main report and moved to Annex B. | Annex B of the ATRT3 Final Report | |||||||||||||||||||||
87 | RySG | 3.4.9 | The RySG notes that the Survey feedback provided by Structures is that they would like to see more information on how financial decisions are made at ICANN. Given this, the RySG supports the suggestions made by the ATRT3. | ATRT3's proposal for prioritization should indirectly assist in addressing these points. Additionally the Implementation of the suggestions associated with the survey results are optional and as such are open to change. | No changes to the suggestion | Section 10 and Annex B of the ATRT3 Final Report. | |||||||||||||||||||||
88 | RySG | 4.4.1.1 | The RySG is unclear on how ATRT3 reached the conclusion that the effectiveness of implementation hinges on the quality of liaisons to the GAC. Have concerns about liaison quality been raised? What is the problem or issue being addressed by this suggestion? | Implementation of ATRT2 recommendations will have to be reviewed by ICANN Org. and the ATRT3 implementation shepherds and then go through prioritization. | None | Section 7 and Annex A of the ATRT3 Final Report | |||||||||||||||||||||
89 | RySG | 4.4.1.2 | The RySG is unclear on how ATRT3 reached the conclusion that the effectiveness of implementation hinges on the quality of liaisons to the GAC. Have concerns about liaison quality been raised? What is the problem or issue being addressed by this suggestion? | Implementation of ATRT2 recommendations will have to be reviewed by ICANN Org. and the ATRT3 implementation shepherds and then go through prioritization. | None | Section 7 and Annex A of the ATRT3 Final Report | |||||||||||||||||||||
90 | RySG | 4.4.2 | As the RySG has noted in our overarching comments, we have concerns about ATRT3 making recommendations or suggestions of a “continuous improvement” nature. We recommend that the ATRT3 revisit this suggestion in the Final Report and attempt to make it more specific, meaningful, achievable, realistic, and time bound (SMART). | Implementation of ATRT2 recommendations will have to be reviewed by ICANN Org. and the ATRT3 implementation shepherds and then go through prioritization. | None | Section 7 and Annex A of the ATRT3 Final Report | |||||||||||||||||||||
91 | RySG | 4.4.3 | As the RySG has noted in our overarching comments, we have concerns about recommendations or suggestions of a “continuous improvement” nature. We recommend that the ATRT3 revisit this suggestion in the Final Report and attempt to make it more specific, meaningful, achievable, realistic, and time bound (SMART). Specifically for this suggestion, the RySG questions whether it is really needed for continuous improvement on Board/GAC interaction and accountability of GAC members? | ATRT 3 in its recommendation on Reviews is proposing that the GAC be included in the evolved Organizational Reviews which provide some level of detail as to how continuous improvement programs should be implemented. | None | Section 8 of the ATRT3 Final Report. | |||||||||||||||||||||
92 | RySG | 4.4.4 | The RySG questions whether this is an appropriate suggestion, as it appears to largely reflect an ignorance of the nuances involved in drafting the communique in a manner that is acceptable for all GAC members. The drafting of the communique is open for anyone to attend and this should be encouraged. The Board and the GAC also have a follow up dialogue to ensure a common understanding of the intent of the language. The RySG notes that individual GAC members are accountable to their own governments, and ICANN does not have jurisdiction to manage individual government processes. The RySG is unclear how this suggestion is measurable. If ATRT3 does make this suggestion, the RySG requests that the RT avoid overly vague language that will be difficult to implement and impossible for the community to measure whether the suggestion is fulfilled over time. As such, we recommend that if the ATRT3 includes this suggestion in the Final Report they attempt to make it more specific, meaningful, achievable, realistic, and time bound (SMART). | Implementation of the suggestions associated with the survey results are optional and as such are open to change. | ATRT3 Survey results and all suggestions associated with these have been removed from the main report and moved to Annex B. | Annex B of the ATRT3 Final Report | |||||||||||||||||||||
93 | RySG | 4.4.5 | The RySG does not object to the Board and GAC developing joint messaging on their interactions and mechanisms which support their interactions. However, the RySG is unclear on why this suggestion is being made and where the problem was raised. The Board and GAC have open public meetings, correspondence between the Board and the GAC is published, and minutes are published of their interactions regarding the GAC communique. It’s unclear what the joint messaging would achieve. | Implementation of the suggestions associated with the survey results are optional and as such are open to change. | ATRT3 Survey results and all suggestions associated with these have been removed from the main report and moved to Annex B. | Annex B of the ATRT3 Final Report | |||||||||||||||||||||
94 | RySG | 4.4.6 | The RySG identified in our survey responses that we would appreciate more dialogue with the GAC. The RySG would support the ATRT3 suggestion that the GAC work with other ICANN Structures that would like more dialogue with the GAC, considering the mechanisms in place for GAC/Board interaction, to establish regular interactions. | Implementation of the suggestions associated with the survey results are optional and as such are open to change. | ATRT3 Survey results and all suggestions associated with these have been removed from the main report and moved to Annex B. | Annex B of the ATRT3 Final Report | |||||||||||||||||||||
95 | RySG | 5.4.1 Translation | The RySG believes the implementation of the ATRT2 recommendation is complete. However, it appears that the ATRT 3 has identified a new issue: missing metrics. We have no way to know how translation services are working. ICANN may not be auditing vendors or doing quality surveys. The RySG suggests that ATRT3 draft a new and distinct recommendation related to metrics and quality control over translation services (in all its various forms in the community). While the provision of documents in multiple languages will be helpful to non-native English speakers, another challenge to meaningful participation is not being able to submit comments in languages other than English. We suggest that this concept be explored further to understand whether offering such a capability will enhance participation in public comment proceedings without overly complicating the process of summarizing and presenting comment responses, and without significantly increasing costs. | Implementation of ATRT2 recommendations will have to be reviewed by ICANN Org. and the ATRT3 implementation shepherds and then go through prioritization. | None | Annex B of the ATRT3 Final Report | |||||||||||||||||||||
96 | RySG | 5.4.2 Public Comments | The RySG supports this suggestion, but notes the following items for ATRT3 to consider further as part of this suggestion: ● While specific and precise questions are useful for guiding the type of input sought, it is difficult for groups to respond to these types of questions ● Even when specific and precise questions are appropriate, there should be the flexibility for responses to include “none of the above” answers ● There is a risk that summaries may inadvertently leave out important information (this will depend on the perspective of a given reader) ● We are also concerned that some readers may rely too heavily on summaries and not engage with the nuance of the issues presented ● There is a risk that meaning and context will be lost when non-experts translate “back” the comments | This has been made into a formal recommendation by ATRT3. A number of refinements were made in converting this into a recommendation. | Yes but in keeping in the spirit of the original suggestion. | Section 3 of the ATRT3 Final Report. | |||||||||||||||||||||
97 | RySG | 5.4.3 Other processes vs Public comment | The RySG appreciates ICANN Org establishing Public Comment Guidelines but would request that these Guidelines are made public so that the Community can fully appreciate how it is determined when public comment should apply. The RySG also notes that the Community Leader Updates sent biweekly are also useful summaries of activities going on and opportunities for input. However, the RySG is particularly supportive of ATRT3’s suggestion to “develop a system similar to, and integrated with, the Public Comment tracking system which would show all uses of alternate mechanisms to gather input including results and analysis of these”. The RySG has experienced difficulty finding all input opportunities in the past and would support a mechanism which streamlined communication expectations for non-public comment input. | This has been made into a formal recommendation by ATRT3. A number of refinements were made in converting this into a recommendation. | Yes but in keeping in the spirit of the original suggestion and these comments. | Section 3 of the ATRT3 Final Report. | |||||||||||||||||||||
98 | RySG | 7.4.1 PDP | Support. | Noted - Implementation of ATRT2 recommendations will have to be reviewed by ICANN Org. and the ATRT3 implementation shepherds and then go through prioritization. | None | Sections 5 and 7.4 of the ATRT3 Final Report. | |||||||||||||||||||||
99 | RySG | 9.4.1 - ATRT2 recs | The RySG supports this recommendation. | This was originally only a suggestion which has been upgraded into a recommendation. | Yes but in keeping in the spirit of the original suggestion. | Section 7.4 of the ATRT3 Final Report. | |||||||||||||||||||||
100 | RySG | 9.4.2 - ATRT2 recs | The RySG supports this recommendation, noting the specific issue raised by ATRT3 in report section 9.3 related to the transfer of certain ATRT2 recommendations to CCWG-Accountability WS2 highlights this problem. We agree that the Board may not always be in the best position to implement Specific Review recommendations. Where the Board transfers recommendations to a different process or body, the RySG supports documenting this transfer and tracking the implementation by the new body responsible. | This was originally only a suggestion which has been upgraded into a recommendation. The new tracking of the implementation of review recommendations at https://www.icann.org/resources/reviews combined with the ATRT3 Prioritization process should handle the transferred implementation issues. | Yes but in keeping in the spirit of the original suggestion. | Section 7.4 of the ATRT3 Final Report. | |||||||||||||||||||||