QA process�Refresh session

PPU, PCU and D&D

30 January 2024

​

​

Agenda

1. QA Process Recap
2. Suggestions to perform the QA; reminder of technical requirements
3. Resources & Final Reminders
4. Questions

www.cgiar.org

1. QA Process Recap

www.cgiar.org

Submitted results currently being QAed

Approximately +70% compared to 2022 (exc. KP)

www.cgiar.org

2023 QA Batch 2 timeline  

Anonymous assessorsx

Agree/disagree and make adjustments

Two assessments are made

Initiatives (IDTs)

Lead assessors

Final decision is taken on unresolved disagreements

Highlight:

 - unresolved disagreement 

 - unimplemented changes

3^rd party 

PPU/PCU

ROUND 1: for all fields

ROUND 2: for priority/core fields

Implementation of changes in the PRMS

Data submission

Initiatives (IDTs)

• The window for initiatives to review comments from assessors has been extended to 10 days, i.e. from Feb 2 to Feb 12, and cannot be extended further.
• If, by the end of the Initiatives' window (Feb 12 COB), there are still outstanding comments that have not been addressed, PPU/PCU will consider that the initiative in question is in agreement with the comments provided by the assessors. 
• Consequently, PPU/PCU will proceed with the implementation of the specified changes or removal of the result accordingly.

Extended submission of KP ONLY in PRMS by Feb 12*

We are

here

Feb 12 deadline is intended EoBD time

*KPs submitted on or before 12 February will be QA'ed in Batch 2. �KPs that are submitted between 12  - 16 February  - the KP extended deadline, and the deadline for some Platforms and SGP swill be QA'ed  at the end of  February.

www.cgiar.org

2. Suggestions to �perform the QA

www.cgiar.org

Suggestions to perform the QA

​

1. Consider planning a QA task force for batch 2, given the time constraints.
2. Initiate the QA exercise with a focus on results to be changed to Knowledge Products (KP) (mainly Other Output), to receive a handle from CGSpace.
3. Accept assessors' comments without leaving another comment, when there is agreement – this action is not requested; however, it is essential to take the required actions.

​

www.cgiar.org

Accessible evidence

During the QA process, ensure that evidence links are accessible by all CG addresses to ensure the assessment.

​

In this batch, the introduction of a new feature allowing the uploading of files in the PRMS as a source of evidence, not publicly available, has significantly reduced instances of missing evidence.

​

Prioritize comments on “access to evidence” to complete the QA process.�If not done, it will go to 3rd Party.

Results lacking the necessary evidence will be unsubmitted.

www.cgiar.org

Instructions to flag – Other Output

In the "Other Output" category, assessors are emphasizing the following:

​

1.  Adhering to the updated supplementary instructions for reporting events, blogs, and multilanguage results. Refer to the Tech Reporting Guidance, page 29.

​

• Refraining from reporting results that are not connected to the Theory of Change (ToC).

​

www.cgiar.org

Assessors' comments on Gender tag 1

​

​

​

While a substantial number of assessors' comments were approved by Initiatives in batch 1 (especially on IA tags), this approval was not reflected in the PRMS.

Implementing the approved changes in the PRMS is crucial.

When, based on the result title/description, it seems that gender equality was not significant in the design of the result but provides only gender-disaggregated data, a tag 0 is recommended.

​

"Tag 1 should be used when the research findings reveal insights into gender-related implications or impacts and provide valuable information for understanding gender dynamics in agriculture or have potential implications for gender-inclusive policies and practices."

www.cgiar.org

Reminder of technical requirements

• Results currently undergoing QA in the PRMS are not automatically unsubmitted. 
• Initiatives are required to individually unsubmit each result for which a change in the PRMS is necessary. 
• After the change is executed, the Initiative must resubmit the result.
• If an Initiative disagrees with a comment during the QA process, the only action required is to provide a justification for the disagreement. No unsubmision/resubmission is needed.

​

During the QA Initiatives' window (Feb 2 to 12), in addition to implementing the agreed-upon changes, you have the flexibility to modify all fields in the PRMS that were not subject to QA (e.g., contributors), but not those that were validated during the QA.

www.cgiar.org

Reminder of technical requirements: KP

For Knowledge Products:

• If a change to the result is necessary, the Initiative should request CGSpace management to implement the change. �
• Additionally, the Initiative must unsubmit the result and click on the re-sync button -  when available -  once the change is implemented in CGSpace. If the re-sync button is not available, it will be done by the PRMS team at the end of the QA process.�
• If changes pertain directly to data included in the PRMS, such as AI tags, the Initiative needs to unsubmit the result to implement these modifications and then resubmit it.

​

www.cgiar.org

QA batch timeline for AVISA, Impact Platforms* and 2023 KP

Anonymous assessors

Agree/disagree and make adjustments

Two assessments are made

Avisa/IP/Initiative

Lead assessors

Final decision is taken on unresolved disagreements

Highlight:

 - unresolved disagreement 

 - unimplemented changes

3^rd party 

PPU/PCU

ROUND 1: for all fields

ROUND 2: for priority/core fields

Implementation of changes in the PRMS

Data submission

AVISA/IP+KPs

Submission of results by 16 Feb

Data on dashboard on Feb 29

This timeline refers to the QA process planned for:

• AVISA and 3 Impact Platforms that will submit their results in the PRMS later than Initiatives
• KPs published in December 2023, for which Initiatives didn’t get the CGSpace handle on time for submission by 23 January 2024

​

* The 3 Impact Platforms concerned by this timeline: Climate Adaptation and Mitigation; Environmental Health and Biodiversity; Nutrition, Health and Food Security

www.cgiar.org

3. Resources and Final Reminders

www.cgiar.org

Result Updates during Feb 2 – 12 COB

Contributors

• Many Initiatives did not “Map and accept” results as a “contributor”  to a result submitted by another Initiative.
• If you are pending “acceptance” by other Initiatives:
• Unsubmit your result
• Contact the other initiative to Accept/Reject Result
• Direct them to the page at the right
• Have them Accept/Reject
• Submit your result
• Conversely, if you want to contribute, but the result is closed, contact the initiative and ask them to Unsubmit.
• Use the CGIAR Research Initiative Database to find the correct contacts

​

Partners

• You may also use this window to add missing partners that �           were pending a CLARISA code�

​

​

www.cgiar.org

Change Result Type

If QA Feedback advises to change the result type, you may now this without deleting and creating a new result.

​

​

​

RELEASE NOTES

       �        Change “Any” to

       “Knowledge Product”

​

        Change “Any” to “Any”    

​

www.cgiar.org

Resources

Webinars

​

30 Oct 2023 - Information Session on Quality Assurance of 2023 submitted results

• Meeting recording
• QA Information Session (IDTs)  (slides)

​

2 Nov 2023 - Information Session on Quality Assurance of 2023 submitted results

• Meeting recording
• QA Information Session (IDTs)  (slides)

​

20 Nov 23 – QA Platform Virtual tour

• Meeting recording
• QA Platform Virtual Tour (slides)

​

Support Materials

• CGIAR Quality Assurance Process for Type 1 Technical Reporting*
• Type 1 Technical Reporting Guidance 
• Quality Assurance Assessor Guidance
• QA Platform Users Manual 2023 

​

​

​

​

For Technical Questions: prmstechsupport@cgiar.org  

For Content Questions: performanceandresults@cgiar.org 

www.cgiar.org

4. Questions

If you would like to provide feedback on the 2023 reporting and on the QA batches, please share your suggestions with your SG' Senior Program Manager and/or send it to performanceandresults@cgiar.org at any time. This will feed into the Learning & Optimization process planned for May-June.

www.cgiar.org

Thank You!