BDEFGHIJKLMNOPQRSTUVWXYZ
1
2
3
4
5
SectionType of Comment (E-Editorial, T-Technical)CommentsProposed SolutionFinal Decision from Task Group
6
2EThe paragraph which starts…. "This guideline applies to the validation for VCM….." I am not sure what is meant by this? Do you mean that you are not supporting those labs that would like to have a VCM acquisition unit of one manufacturer and then use a viewer from a different manufacturer? If so, why would this not be supported, since all acquisitions should be of similar format.I am not sure the solution, since I am confused by the statement. - maybe remove this stipilation. Since this is process that the laboratory may want to make.No Change: VCM is a recognized universal term that is not vendor specific in either hardware or software. 3D toolmark topographical instruments is also universal to any instrument capable of generating 3D data.
7
2EAt the end of this section is a bullet point list of necessary elements. The last on states " Rention of supporting records" What records is this reference to?Clarify the supporting records that this bullet point is referring to.Change: "Retention of supporting validation records."
8
3EFor FNR and FPR, the formulas are listed at the end of the definitions. But for sensitivity and specificity they are not listed. Since they are references in section 9. May reference section 9 or add/delete the formulas to the definitions for consistency.Change: Added See section 9 where all four equations are listed.
9
3EFor the definition of VCM - the word "manipulation" is usedsuggest a different word to describe what the user is doing, like adjust or optimizeChange: replaced with assessment.
10
6EThroughout this section, clear roles for different personnel are listed. I have concerns that small laboratories will not be able to have different people to fill this roles as outline.Suggest softening langauge to indicate that "when possible" or "if staffing allows"Change: removed shall and replaced with should. It's important that the VM is not part of the test since they will be the one assembling the tests. VM can be from another section or a contractor.
11
8.1.1EThe first sentence of this section is confusing to the reader. It reads as though the samples should be the equivalent of one year's worth of casework. I believe the intent was to have the sample chosen to represent what is seen in casework by the laboratory.Suggest removing the last part of the sentence, so that it reads…. Samples selected for the DV shall be representation of those firearms encountered in casework.Change: Refined the first sentence to "A minimum of a year’s worth of casework should be evaluated for DV sample selection." The goal is to review one year's worth of casework to determine commonly seen class of samples.
12
8.2.1EFor the last sentence in this section… is this similar to what is stated in 8.3.1?If so, suggest that this sentence is written like 8.3.1 or referes to it.No Change: 8.2.1 is the final error rate goal a laboratory should seek to achieve. 8.3.1 is the minimum starting point for a lab with only a single examiner.
13
8.2.3T This section refers to an appendix. I did not see this in the documentAdd appendixChange: Added Appendix A
14
8.3.1/8..3.2TSample numbers are not listed like they are in 8.2.1Suggest adding needed sample number, for user reference.Change: Added sample numbers.
15
9THas listed formulas with abbreviations that are not defined in document.Define abbreviations No Change: all four formulas are defined in section 3.
16
3TWhat is a 'developmentally validated' technology?Clarify or define this phrase/label.Refer to the normative reference ASB 063 Implementation of 3D Technologies in Forensic Firearm and Toolmark Comparison Laboratories
17
8.2.1TThe error rates (e.g., 1%) and confidence ranges (e.g., 95%) are not justified.Provide some reasoning, provide empirical support for the chosen true false positive/negative rates, or acknowledge it's arbitrary.1 % FP error rate comes from existing black box study averages for firearm toolmark analysis. 95 % confidence interval is recognized as the industry standard across multiple scientific disciplines. It's also what NIST uses for all its certified calibrations.
18
8.3.1TSpecified rates here again seem arbitrary and are not justified.Provide some reasoning, provide empirical support for the chosen true false positive/negative rates, or acknowledge it's arbitrary.These error rates of 5 % are set by concensus within the TWG as reasonable baselines for a laboratory with only a single examiner.
19
8.3.2TSpecified rates here again seem arbitrary and are not justified.Provide some reasoning, provide empirical support for the chosen true false positive/negative rates, or acknowledge it's arbitrary.These error rates of 2.5 % FP and 3 % FN are set by concensus within the TWG as reasonable baselines for a laboratory with more than one examiner.
20
AllT
There are existing documents that might be helpful to draw on when revising this draft standard.
It might be helpful to include a reference to, or look at, this document: https://www.anzpaa.org.au/products/products/guideline-for-the-validation-of-forensic-science-methods
Thank you for the reference.
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100