MSA

MEASUREMENT SYSTEM ANALYSIS

(THIRD EDITION)

DEFINITIONS

WHAT IS MEASUREMENT

MEASUREMENT IS ASSIGNMENT OF NUMBERS (OR VALUES) TO MATERIAL THINGS TO REPRESENT THE RELTIONS AMONG THEM WITH RESPECT TO PARTICULAR PROPERTIES.

BY EISENHART (1963)

WHAT IS MEASUREMENT SYSTEM

MEASUREMENT SYSTEM IS THE COLLECTION OF INSTRUMENTS,STANDARDS,OPERATIONS,METHODS,FIXTURES,SOFTWARE,PERSONNEL,ENVIRONMENT & ASSUMPTIONS

Process

.

​

• INPUTS

​

​

​

PROCESS

​

​

​

• OUTPUTS

Measurement process

.

• Standard
• Work piece (part)
• Instrument
• Person
• Procedure
• Environment

​

​

• Measurement

​

​

​

​

​

• Measurement result

​

​

​

​

• Analysis

​

​

​

​

​

• Decision (action)

​

​

Measurement system analysis

Study of combined effect of all measurement contributors.

Assessing their suitability against measurement objective.

​

Objectives of Measurement Process

• Product control-
• Is the part in a specific category i.e. meets specification requirements ?
• process control-
• Is the process variation stable & acceptable

STATISTICAL PROPERTIES TO DEFINE A “GOOD” MEASUREMENT SYSTEM (MS)

​

• 1. ADEQUATE DISCRIMINATION & SENSITIVITY:
• Should be small relative to PROCESS VARIATION or SPECIFICATION LIMIT.
• Rule of 10’s i.e. 1/10th .
• i.e. least count/ resolution of equipment should be 1/10th of process Variation (10 data categories).

​

• 2. MS should be in statistical control
• - Common cause variations only
• - No special cause variation
• 3. Variability of MS < mfg. Variability
• -For process control
• 4. Variability < specification limits (sl)
• - for product control

CONTRIBUTION OF MEASUREMENT SYSTEM ERROR IN MEASUREMENT RESULT

​

1. LOCATION ERROR:

​

ACTUAL VARIATION

OBS. VARIATION

DUE TO MS ERROR

CONTRIBUTION OF MEASUREMENT SYSTEM ERROR IN MEASUREMENT RESULT

​

2. WIDTH (SPREAD) ERROR:

​

​

​

​

​

​

​

​

​

​

​

​

ACTUAL VARIATION

OBS. VARIATION

DUE TO MS ERROR

EFFECT OF MEASUREMENT SYSTEM ERROR ON MEASUREMENT DECISION

1. EFFECT ON PRODUCT CONTROL:

1a. CALLING A GOOD PART AS BAD PART (CALLED TYPE -I ERROR-Customer Bias)

​

​

​

​

​

​

1b. CALLING A BAD PART AS GOOD PART (CALLED TYPE -II ERROR-Producer Bias)

​

​

​

​

EFFECT OF MEASUREMENT SYSTEM ERROR ON MEASUREMENT DECISION

​

2. EFFECT ON PROCESS CONTROL:

​

• 2a. CALLING A COMMON CAUSE AS SPECIAL CAUSE (CALLED TYPE -I ERROR)

​

• 2b. CALLING A SPECIAL CAUSE AS COMMON CAUSE (CALLED TYPE -II ERROR)

​

​

​

​

EFFECT OF MEASUREMENT SYSTEM ERROR ON MEASUREMENT DECISION

​

2. EFFECT ON PROCESS CONTROL:

Observed Variance is equal to Actual Variance and Measurement System Variance.

^{2 obs} = ^{2 act} + ^2MSA

APPRAISER

A

B

C

MEASUREMENT SYSTEM ERRORS

-BIAS

• -LINEARITY
• -STABILITY

-REPEATABILITY

-REPRODUCIBILITY

LOCATION ERRORS

(Accuracy)

WIDTH ERRORS

(Precision)

Bias

• Difference between the observed average of measurements and the true value (reference value) on the same characteristics on the same part.
• A measure of the systematic error of the measurement system.
• It is the contribution of the total error comprised of the combined effects of all sources of variation, known or unknown.

​

Bias

​

Observed

Average

Value

​

Bias

Linearity

• The difference in the bias values through the expected operating (measurement) range of the equipment.
• This is change of bias with respect to size.

1

3

2

MEASURMENT

POINTS

Linearity

BIAS

NO LINEARITY ERROR

CONSTANT LINEARITY

NON LINEAR

1

0

-1

REFERENCE VALUE

Stability (Drift)

The total variation in the measurements obtained with a measurement system-

• on the same master or parts,
• when measuring a single characteristic,
• over an extended time period.

i.e. Stability is the change of bias over time

Stability (Drift)

CAUSES OF INSTABILITY-

• Mechanical wear,
• Electrical instability,
• Changes in external environment,
• Degradation in components with time
• Quite Common with electrical based gauges, optical gauges(dependent on light sources, spectrophotometer etc.

Repeatability (Within system variation)

The variation in measurements obtained

• with one measurement instrument
• when used several times
• by one appraiser
• while measuring the identical characteristic
• on the same part.

Repeatability

Note:Repeatability is commonly referred to as equipment variation(EV), although this is misleading. In fact repeatability is within system (SWIPPE) variation

Reproducibility (Between system variation)

The variation in the average of the measurements

• made by different appraisers
• using the same measuring instrument
• when measuring the identical characteristic
• on the same part.

A

B

C

REPRODUCIBILITY

Gage Repeatability & Reproducibility(GRR)

An estimate of the combined variation of repeatability and reproducibility.

GRR is the variance equal to the sum of within system & between system variances.

² = ² + ²

APPRAISER

A

B

C

GRR

repeatability

reproducibility

• DETERMINING BIAS

Bias

​

Observed

Average

Value

​

Bias

• Typical Preparation Required For Study

• What is influence of appraiser on Measurement System
• Determine no. of Appraisers, Sample Parts, Repeat readings based on:
• Criticality of dimension
• Part configuration
• Use regular operators
• Parts Sample should represent entire production process
• Instrument discrimination should be 1/10^th of Process Variation
• Measurement Method is Standard
• Blind method to avoid appraiser bias
• Equipment should be calibrated

• DETERMINING BIAS

• Selection of reference standard:
• Priority order
• Sample piece else
• Production part else
• Similar other component else
• Metrology standard
• ​

• DETERMINING BIAS

• Establish reference value:
• Identify measurement location
• To the extent possible to minimize the effect of within part variation.
• Measure the part for n10 times
• In standard room/ tool room
• With a measurement equipment of better accuracy.
• Using standard measurement method

• DETERMINING BIAS

• Establish reference value:
• Calculate the average.
• Use this average as
• “Reference Value”.

​

• DETERMINING BIAS

• Data collection:
• Measure the reference standard at the marked location
• Under normal measurement condition i.e.
• Routine measurement equipment,
• Routine operator,
• Routine method,
• Routine environmental condition etc.)
• For n> 10 times

• DETERMINING BIAS

Graphical analysis (analysis for normality):

• Plot the data (observations) as a histogram
• Analyse if any special cause present.
• If yes, identify & remove the cause and recollect data & re-analyse.
• If not, proceed for numerical analysis.
• ​

• DETERMINING BIAS

• Numerical computation:
• Compute average of n readings as  y_i
• i = 1
• n
• - Compute bias(b)= y –X = 4.07-4.05 = 0.02

• DETERMINING BIAS

• Decision making- (checking if bias is significant)
• Select max.(x_i) and min.(x_i) from the data
• Compute
• repeatability = ^{max (xi ) - min (xi)} = 6-2 = 1.051
• d₂* 3.805

• DETERMINING BIAS

• Statistical analysis :Compute the followings-
• _{b =} _r /n =1.051/ 20 =0.2351
• Lower bound (L) = b – [d₂ _b / d₂* (t_v,/2)] = -0.02 –0.2351x2.16= -0.528
• Upper bound (U) = b + [d₂ _b / d₂* (t_v,/2)] = -0.02 +0.2351x2.16= 0.488
• d₂, d₂*, v (degree of freedom) can be obtained from appendix- A
• t_v,/2 can be obtained from t table .  (preferably 0.05) is a measure of confidence
• Bias is acceptable at (1-) level if
• L < 0 < U

• IF BIAS IS STATISTICALLY NON ZERO

• Possible causes can be:-
• Error in master or reference value.Check mastering Procedure
• Worn instruments. This can show up in stability analysis and will suggest the maintenance or refurbishment schedule
• Instrument made to wrong dimensions
• Instrument measuring wrong characteristics
• Instrument not calibrated properly
• Improper use by operator. Review instrument instructions

• DETERMINING LINEARITY

Linearity

The difference in the bias values through the expected operating (measurement) range of the equipment.

This is change of bias with respect to size.

1

3

2

• DETERMINING LINEARITY

• Determine operating range of gage
• Collect data
• Select g > 5 parts covering the operating range of the gage
• Determine their reference values
• Measure each part m > 10 times on the subject gage by one of the operators who normally use the gage
• select the parts at random to minimize appraiser “recall” bias in the measurements

• DETERMINING LINEARITY

• Graphical analysis
• Calculate the part bias (b) for each measurement

b _i,j = y_i._j – (reference value)_i

• Calculate bias average (y) for each part

 Bias_i,j

j = 1

m

• Plot the individual biases and the bias averages with respect to the reference values on a linear graph

_

• Calculate total bias average (y )= ( y)/n

​

• DETERMINING LINEARITY

• Calculate and plot the best fit line and the confidence band of the line using the following equations

For the best fit line, use: y_i = ax_i + b

Where

x_i = reference value

y_i = bias average

​

xy – 1  x  y

=

x² – 1 ( x)²

b = y – ax = intercept

• DETERMINING LINEARITY

For a given x₀, the  level confidence bands are

​

Where s = y²_i – b y_i – a x_iy_i

gm – 2

​

t_gm-2,/2

(x₁ – x)²

1

gm

(x₀ – x)²

+

1/2

s

Lower: b + ax₀ –

t_gm-2,/2

(x₁ – x)²

(x₀ – x)²

1/2

s

Upper: b + ax₀ +

1

gm

+

• DETERMINING LINEARITY

7. Plot the “bias = 0” line

​

8. Linearity acceptable if,

“bias = 0” lie entirely within the confidence bands of the fitted line.

​

• DETERMINING LINEARITY

​

NUMERICAL ANALYSIS

9. IF THE GAPHICAL ANALYSIS INDICATES THAT THE MEASUREMENT SYSTEM LINEARITY IS ACCEPTABLE THEN THE FOLLOWING HYPOTHESIS SHOULD BE TRUE:

H₀: a = 0 slope = 0

do not reject if

​

​

• DETERMINING STABILITY

• DETERMINING STABILITY

• Selection of reference standard: Refer bias study.
• Establish reference value :Refer bias study.
• Data collection:
• Decide subgroup size
• Decide subgroup frequency
• Collect data for 20-25 subgroups
• ​

• DETERMINING STABILITY

• Analysis
• Calculate control limits for X-R chart.
• Plot data on chart
• Analyse for any out of control situation.
• Decision
• Measurement system is stable & acceptable if no out of control condition is observed other wise not stable and needs improvement .

Gage Repeatability & Reproducibility

^2R&R = ^{2 REPRODUCABILITY} + ^{2REPEATABILITY}

A

B

C

• R&R-AVERAGE AND RANGE METHOD

AN APPROACH WHICH WILL PROVIDE ESTIMATE OF BOTH REPEATABILITY AND REPRODUCIBILITY FOR A MEASUREMENT SYSTEM.

• R&R-AVERAGE AND RANGE METHOD

• CONDUCTING THE STUDY
• Selection of sample: n > 5 parts (representing process variation).
• Identification: 1 through n (not visible to the appraisers).
• Location Marking (easily visible & identifiable by the appraisers).
• Selection of appraiser: 2-3 routine appraisers
• Selection of Measuring equipment: Calibrated routine equipment
• Deciding number of trials: 2-3
• Data collection:
• Using data collection sheet
• Under normal measurement condition
• in random order
• using blind measurement process

• R&R-AVERAGE AND RANGE METHOD

Data collection

• Enter appraiser A result row 1.
• Enter appraisers B and C results in row 6 and 11, respectively.
• Repeat the cycle (2nd trial) & enter data in rows 2, 7 and 12.
• If three trials are needed, repeat the cycle and enter data in row 3, 8 and 13.

• For appraiser A, calculate Average (X) & Range(R) for each part and enter in rows 4 & 5 respectively.
• Do the same for appraisers B & C and enter results in rows 9, 10 and14, 15 respectively.
• For appraiser A, calculate average (X_a) of all the averages (row 4) and average (R_a) of all the ranges (row 5) and enter in data sheet.
• Calculate X _b, R_b & X_c, R_c for appraisers B & C also and enter the results in data sheet.
• Calculate average of all the observations (rows 4, 9 &14) of each part and enter result in row 16.
• Calculate Part range (R_p)= Difference of Max. and Min. of row 16 and enter in data sheet (right most row 16).
• Calculate X =(X_a + X_b + X_c )/3 and enter in data sheet (right most row 16).

• R&R- GRAPHICAL ANALYSIS

• R&R- GRAPHICAL ANALYSIS

• Calculate R = (R_a+ R_b + R_c )/3 and enter result in row 17.
• Calculate X_diff = Difference of Max and Min of (X_a , X_b & X_c) and enter in row 18
• Calculate UCLR =D₄ R and enter in row 19 (D₄=3.27 for 2 trials & 2.58 for 3)
• Calculate LCL R =D₃ R and enter in row 20 (D₃= 0 for trials<7)
• Calculate UCLX= X+A₂ R (A₂=1.88 for 2 trials & 1.02 for 3 trials).
• Calculate LCL x= X-A₂ R
• ​

• R&R- GRAPHICAL ANALYSIS
• RANGE CHARTS

• Used to determine whether the process is in statistical control.
• The special causes need to be identified and removed
• Plot all the ranges for each part & each appraiser on range chart
• If all ranges are under control, all appraisers are doing the same job.
• If one appraiser is out of control, the method used differs from the others.
• Repeat any readings that produced a range greater than the calculated UCL_R using the same appraiser and part as originally used.
• Or, discard those values and re-average and recompute R and the limiting value UCL_R based upon the revised sample size.
• Correct the special cause that produced the out of control condition.

• R&R- AVERAGE CHART

• Plot the averages of the multiple readings by each appraiser on each part (rows 4, 9 & 14) on X chart.
• The X chart provides an indication of “usability” of the measurement system.
• The area within the control limits represents the measurement sensitivity (“noise”).
• Approximately one half or more the averages should fall outside the control limits.
• If the data show this pattern, then the measurement system should be adequate to detect part-to-part variation and can be used for analyzing and controlling the process.
• If less than half fall outside the control limits then either the measurement system lacks adequate effective resolution or the sample does not represent the expected process variation.

• R&R- ANALYSIS OF RESULTS -- NUMERICAL

Calculate the following and record in report sheet

• Repeatability or equipment variation (EV) = R x K₁ (K₁ =1/ d₂* )
• K₁ = .8862 for 2 trials & .5908 for 3 trials
• ​
• Reproducibility or appraiser variation (AV)=
• (K₂= .7071 for 2 appraisers & .5231 for 3)
• ​
• GRR=

​

• Part to part variation (PV)= R_P x K₃

​

• Total variation (TV) =

(EV)^{2 +} (AV)²

(GRR)^{2 +} (PV)²

(X_DIFF x K₂)²

(EV)²

nr

-

• NUMERICAL ANALYSIS

• Calculate % variation and ndc as follows

%EV = 100 [EV/TV]

• %AV = 100 [AV/TV]

%GRR = 100 [GRR/TV]

%PV = 100 [PV/TV]

• No. of distinct catagories (ndc)= 1.41(PV/GRR)

Note:

• In case measurement system is to be used for product control instead of process control, TV should be replaced with specification tolerance.
• The sum of the percent consumed by each factor will not equal 100%.

• NUMERICAL ANALYSIS

• Decision making:
• For % R & R

​

• Error < 10% - MS is acceptable
• 10% > Error < 30%- May be acceptable with justification
• Error > 30% - MS needs improvement
• ​
• ndc >= 5

• ATTRIBUTE MEASUREMENT
• SYSTEMS STUDY

• Attribute measurement systems are the class of measurement systems where the measurement value is one of a finite number of categories.
• This is contrasted to the variables measurement system which can result in a continuum of values.
• The most common of these is a go/nogo gage which has only two possible results.
• There is a quantifiable risk when using any attribute measurement systems in making decisions.
• The largest risk is at the category boundaries

LSL

USL

MEASUREMENT SYSTEM SHADED AREAS

• ATTRIBUTE MEASUREMENT
• SYSTEMS STUDY
• ​

METHODOLOGY:

​

• Select n>12 parts as follows:
• Approximately one third conforming,
• One third non conforming & one third marginal (marginal conforming & marginal non conforming)
• Note down the correct measurement attribute (true status).
• Decide the no. of appraiser & no. of trials to be conducted.
• Record the measurement result in data sheet

• ATTRIBUTE MEASUREMENT
• SYSTEMS STUDY
• ​

Possible outcome by the appraiser:

• Correct decisions
• Calling good part as good (good-correct)
• Calling bad part as bad (bad-correct)
• Wrong decisions
• Calling good part as bad (false alarm)
• Calling bad part as good (miss)
• For each appraiser count the data as follows
• Number good-correct (GN)
• Number bad -correct (NB)
• Number total correct (CN): total (GN+NB)
• Number false alarm(NF): that is rejecting ok part
• Number miss(NM): that is accepting rejected part
• Number total (TN): total of (GN+NB+NF+NM)

• ATTRIBUTE MEASUREMENT
• SYSTEMS STUDY
• ​

Methodology:

• For each appraiser calculate the following
• Effectiveness (E)= CN/Total opportunity for correct decision
• Probability of false alarm(Pfa)=NF/ Total opportunity for false alarm
• Probability of miss(PM)=NM/ Total opportunity for miss
• Decision

Parameter acceptable marginal unacceptable

E >.90 .80 to .90 <.80

Pfa <.05 .05 to .10 >.10

Pm <.02 .02 to .05 >.05

• ​
• ​

MSA FOR COMPLEX & NON REPEATABLE MS

GRR STUDY

Sampling:

• Select 6 very similar parts (assumed to be identical) consecutively.
• Or if possible split 1 sample in 6 parts
• Consider these 6 duplicate samples as sample 1
• mark them 1-1, 1-2,……….1-6
• In similar manner select 10 samples (i.e 10 x 6= 60 parts as-
• 1-1, 1-2,…………….1-6
• 2-1, 2-2,……………..2-6
• ……………………………
• 10-1, 10-2, …………10-6

MSA FOR COMPLEX & NON REPEATABLE MS

GRR STUDY

SAMPLING EXAMPLE:

​

MSA FOR COMPLEX & NON REPEATABLE MS

GRR STUDY

SAMPLING EXAMPLE:

​

₁

₂

₃

₄

MSA FOR COMPLEX & NON REPEATABLE MS

GRR STUDY

Sampling:

• Take care that all 1 to 6 parts (duplicate parts) of each samples are as much alike as possible
• However, there must be difference between one group of samples and another group of samples.
• That is duplicate parts within a group should be identical, but between group of parts there should be difference.
• The total combination should represent entire process variation
• Collect data in random order & perform GRR study

MSA FOR COMPLEX & NON REPEATABLE MS

GRR STUDY

Data collection (all preconditions apply):

• Select the following:
• Routine appraiser (2-3),
• Routine measuring equipment
• Routine measurement venue
• Routine measurement environment

MSA FOR COMPLEX & NON REPEATABLE MS

GRR STUDY

DATA COLLECTION EXAMPLE

MSA FOR COMPLEX & NON REPEATABLE MS

GRR STUDY

ANALYSIS:

ANALYSE GRR

USING AVERAGE RANGE

MSA FOR COMPLEX & NON REPEATABLE MS

MSA FOR COMPLEX & NON REPEATABLE MS

MSA FOR COMPLEX & NON REPEATABLE MS