Metrology for Engineering Technicians

Principles, Measurement Systems, and Data Analysis

Appendices

Formula reference, key definitions, and notes & clarifications. Use the links below to jump to a section.

Appendix A: Formula Reference (Chapters 1–8)

This appendix consolidates all formulas used throughout Chapters 1–8 of this textbook. It is intended as a quick-reference sheet for students and practitioners. Units must always be verified before applying any formula.

A.1 Chapter 1 — Gage Blocks and Steel Rules

Greatest Possible Error (GPE)

GPE = ± (smallest scale division ÷ 2)

Unit Conversion

1 in = 25.4 mm
mm to in: mm ÷ 25.4
in to mm: in × 25.4

Percent Difference

%D = ((B − A) / A) × 100

A.2 Chapter 2 — Dial Calipers

Final Measurement (Inch or Metric)

Measurement = Blade Reading + Dial Reading

Greatest Possible Error

Inch caliper GPE = ±0.0005 in
Metric caliper GPE = ±0.01 mm

Percent Difference Between Instruments

%D = ((Value₂ − Value₁) / Value₁) × 100

A.3 Chapter 3 — Micrometers

Inch Micrometer Measurement

Measurement = Sleeve (0.100 in) + Sleeve (0.025 in) + Thimble (0.001 in) + Vernier (0.0001 in, if present)

Metric Micrometer Measurement

Measurement = Sleeve (mm) + Half-mm (0.5 mm, if visible) + Thimble (0.01 mm)

Greatest Possible Error

Inch micrometer GPE = ±0.0005 in
Metric micrometer GPE = ±0.005 mm

A.4 Chapter 4 — Bias and Confidence Intervals

Bias

Bias = Measured Value − Actual Value

Degrees of Freedom

df = n − 1

Confidence Interval for the Mean

x̄ ± (t · s / √n)

Where: x̄ = sample mean, t = Student's t-statistic, s = sample standard deviation, n = number of trials.

Upper and Lower Confidence Limits

UCL = x̄ + (t · s / √n)
LCL = x̄ − (t · s / √n)

A.5 Chapter 5 — Dial Gauges and Indicators

Dial Gauge Measurement

Measurement = (full revolutions × range per revolution) + (dial divisions × discrimination)

Sign Convention

Clockwise motion = positive (+)
Counterclockwise motion = negative (−)

A.6 Chapter 6 — Statistical Process Control (SPC)

X-Bar Chart Control Limits

UCLx̄ = x̄̅ + A₂·R̄
LCLx̄ = x̄̅ − A₂·R̄

R Chart Control Limits

UCLR = D₄·R̄
LCLR = D₃·R̄

Where constants A₂, D₃, and D₄ depend on subgroup size.

SPC Constants for X-Bar and R Charts (added in this edition)
Subgroup size (n) A₂ D₃ D₄
21.88003.267
31.02302.574
40.72902.282
50.57702.114
60.48302.004
70.4190.0761.924
80.3730.1361.864
90.3370.1841.816
100.3080.2231.777

A.7 Chapter 7 — Spherometers and Diopter Gauges

Radius of Curvature (Spherometer)

R = d² / (8h) + h / 2

Where: R = radius of curvature, d = spherometer base diameter, h = sagittal height.

Radius–Diopter Relationship

R (mm) = 350 / P
P (diopters) = 350 / R

Meniscus Lens Total Power

Ptotal = P₁ + P₂

A.8 Chapter 8 — Integrated Analysis

Power-Law Relationship

y = a·xb

Logarithmic Form

log(y) = log(a) + b·log(x)

Used for log-log plotting and trend identification in Excel.

Appendix B: Key Definitions (Chapters 1–8)

This appendix provides concise definitions of key terms used throughout the textbook. Definitions are written in clear, non-technical language suitable for quick review.

B.1 Chapter 1 Definitions

Metrology
The science of measurement.
Precision
The smallest increment an instrument can reliably display.
Discrimination
The spacing between scale divisions on a measuring instrument.
Greatest Possible Error (GPE)
The maximum expected reading uncertainty, equal to half the smallest scale division.

B.2 Chapter 2 Definitions

Dial Caliper
A measuring instrument that combines a linear scale with a dial to provide precise dimensional readings.
Outside Measurement
Measurement of external features such as diameter or width.
Inside Measurement
Measurement of internal features such as holes or slots.

B.3 Chapter 3 Definitions

Micrometer
A high-precision measuring instrument used for small external dimensions.
Thimble
The rotating component of a micrometer used to read fine increments.
Ratchet Stop
A mechanism that ensures consistent measuring force.

B.4 Chapter 4 Definitions

Bias
A systematic difference between measured values and a known reference value.
Confidence Interval
A range of values that likely contains the true population mean.
Student's t-Statistic
A value used in confidence interval calculations for small samples.

B.5 Chapter 5 Definitions

Dial Gauge (Indicator)
A comparative instrument used to measure deviation from a reference.
Zeroing
Setting an instrument reference point before measurement.
Cosine Error
Measurement error caused by misalignment between instrument and surface.

B.6 Chapter 6 Definitions

Statistical Process Control (SPC)
A method of monitoring process performance over time using statistics.
Common Cause Variation
Natural variation inherent in a stable process.
Special Cause Variation
Variation caused by identifiable, abnormal factors.

B.7 Chapter 7 Definitions

Spherometer
An instrument used to measure the curvature of spherical surfaces.
Sagittal Height
The vertical distance between a curved surface and the spherometer base plane.
Diopter
A unit of optical power equal to the reciprocal of focal length in meters.

B.8 Chapter 8 Definitions

Measurement System
The combination of instrument, operator, method, environment, and analysis.
Log-Log Plot
A graph where both axes use logarithmic scales.
Engineering Judgment
Decision-making based on data, experience, and technical reasoning.

Notes and Clarifications

This section provides brief clarifications and instructional notes intended to support consistent interpretation of the material throughout this textbook. These notes do not introduce new content and do not change course expectations.

1. Instrument Scale Conventions

Throughout this text, the terms precision, discrimination, and smallest division are closely related. For clarity:

This terminology is used consistently across chapters for instructional clarity.

2. Decimal Inch Steel Rules

Some examples in Chapter 1 use decimal-inch steel rules with increments such as 0.02 in. While fractional rules (1/32, 1/64) are more common in some shop environments, decimal-inch rules are widely used in inspection and quality settings. The examples are intended to reinforce reading logic rather than represent tool availability.

3. Micrometer Vernier Scale Usage

In Chapter 3, micrometers equipped with vernier scales are shown with an effective resolution of 0.0001 in. When the vernier scale is read correctly and consistently, the effective GPE may be taken as ±0.00005 in. In instructional laboratories, however, reported uncertainty may still be conservatively stated as ±0.0005 in depending on course or departmental practice.

4. Dial Gauge Full-Revolution Interpretation

Dial gauges measure deviation, not absolute size. One full revolution of a typical inch dial gauge corresponds to 0.100 in, not 1.000 in. When interpreting dial gauge readings:

The Chapter 5 worked examples and answer key in this edition have been corrected to follow this rule.

5. Control Limits vs. Specification Limits

As emphasized in Chapter 6:

A process may be in statistical control and still fail to meet specifications if it is improperly centered.

6. Optical Power Constant in Chapter 7

The relationship R (mm) = 350 / P used in Chapter 7 is an instructional approximation appropriate for typical laboratory optics and common refractive indices. This constant is used for educational purposes to simplify analysis and comparison and should not be assumed universal for all optical materials or professional optical design.

7. External Resources and Links

External resources, simulations, and video links included in this textbook are provided for optional enrichment. Availability and content of third-party resources may change over time and are not required for course completion.

Closing Note

These clarifications are intended to support accurate interpretation, consistent grading, and alignment with common industry practices. The core instructional content of this textbook remains unchanged.