Uncertainty Analysis and Error Propagation Methodology*

By Andrey Gunawan
*A PDF version of this document can be downloaded here.

The multiple-measurement uncertainty analysis that is implemented to the datasets in this database, and is described herein, is based on a textbook method,Âą which follows the American National Standard Institute/American Society of Mechanical Engineering (ANSI/ASME) Power Test Codes (PTC) 19.1 Test Uncertainty and NIST Technical Note 1297. It is also consistent with the international guidelines established by the International Organization for Standardization (ISO).

Uncertainty analysis at 95% confidence level for reporting thermal diffusivity measurement: In order to meet our established Measurement Acceptance Criteria,² at least 3 samples (M = 3) from 3 different locations in the bulk material are tested. Using the default setting of our NETZSCH LFA 467 HT, 3 measurements (N = 3) of each sample are taken at each set temperature. The NETZSCH software automatically calculates and provides the mean and the standard deviation of each sample’s thermal diffusivity (\alpha):

{ \bar { \alpha } }_{ m }=\frac { 1 }{ N } \sum _{ n=1 }^{ N }{ { \alpha }_{ mn } }
{ s }_{ { \alpha }_{ m } }={ \left[ \frac { 1 }{ N-1 } \sum _{ n=1 }^{ N }{ { \left( { \alpha }_{ mn }-\bar { \alpha } _{ m } \right) }^{ 2 } } \right] }^{ { 1 }/{ 2 } }

At each temperature, the mean value from the 3 different samples are subsequently averaged to yield a mean \alpha of the bulk material, using pooled averaging:

\left< \bar { \alpha } \right> =\frac { 1 }{ M } \sum _{ m=1 }^{ M }{ { \bar { \alpha } }_{ m } }

            Uncertainties or errors in this LFA measurement can be grouped into (1) instrument, (2) spatial variation, and (3) temporal variation errors. First, consider the instrument error. The instrument error is assigned a systematic standard uncertainty based on the manufacturer’s statement, which is naturally assumed to be stated at 95% confidence level. For example, the technical datasheet accompanying the NETZSCH LFA 467 HT states an instrument uncertainty or accuracy to within ± 3% of the reading;3 thus,

{ \left( { b }_{ \bar { \alpha } } \right) }_{ 1 }={ \left( \frac { B_{ \bar { \alpha } } }{ 2 } \right) }_{ 1 }=\left( \frac { 0.03\times \left< \bar { \alpha } \right> }{ 2 } \right)

The subscript keeps track of the error group (e.g., subscript 1 is for the instrument error). No random uncertainty is assigned to the instrument error:

{ \left( { s }_{ \bar { \alpha } } \right) }_{ 1 }=0

Such random uncertainty can be assumed negligible, because conventionally manufacturers are required to test large number of repetitions and replications to confidently publish the accuracy statement in their datasheet. In this case, NETZSCH claimed that the ± 3% uncertainty of the reading was based on 900 tests with high and low \alpha specimens with at least 3 different devices at room temperature.3

            Consider next the spatial variation error contribution to the estimate of the mean \alpha of the bulk material. This error arises from the spatial nonuniformity in the bulk material. An estimate of spatial \alpha distribution within the bulk material can be made by examining the mean thermal diffusivities of the 3 measured samples (M = 3)  from 3 different location in the bulk material. The mean thermal diffusivities between the 3 samples show a standard deviation of:

{ s }_{ { \alpha } }={ \left[ \frac { 1 }{ M-1 } \sum _{ m=1 }^{ M }{ { \left( { \bar { \alpha } }_{ m }-\left< \bar { \alpha } \right> \right) }^{ 2 } } \right] }^{ { 1 }/{ 2 } }

Thus, the random standard uncertainty of the mean thermal diffusivities between the 3 samples is found from:

{ \left( { s }_{ \bar { \alpha } } \right) }_{ 2 }=\frac { { s }_{ \alpha } }{ \sqrt { M } }

with degrees of freedom,

{ \left( { \nu }_{ { s }_{ \alpha } } \right) }_{ 2 }=M-1

In contrast with the instrument error, no systematic uncertainty is assigned to the spatial variation error,

{ \left( { b}_{ \bar { \alpha } } \right) }_{ 2 }=0

One could reasonably argue that the random standard uncertainty (of the mean thermal diffusivities between the 3 samples) represents a systematic uncertainty because it is an effect that would offset the final value of the estimated \alpha of the bulk material.

            For each sample, the temporal variation error in the LFA output during each of the 3 flashes (N = 3) at each temperature cause data scatter, as evidenced by the respective standard deviation values of each sample’s thermal diffusivity. Such temporal variations are caused by random local \alpha variations as measured by the LFA sensor, sensor resolution, and the LFA furnace temperature control variations during fixed operating conditions. Since we have insufficient information to separate these, they are estimated together as a single error. The pooled standard deviation is

\left< { s }_{ { \alpha } } \right> ={ \left[ \frac { 1 }{ M\left( N-1 \right) } \sum _{ m=1 }^{ M }{ \sum _{ n=1 }^{ N }{ { \left( \bar { \alpha } _{ mn }-\left< \bar { \alpha } \right> \right) }^{ 2 } } } \right] }^{ { 1 }/{ 2 } }={ \left[ \frac { 1 }{ M } \sum _{ m=1 }^{ M }{ { s }_{ { \alpha }_{ m } }^{ 2 } } \right] }^{ { 1 }/{ 2 } }

to give a random standard uncertainty of

{ \left( { s }_{ \bar { \alpha } } \right) }_{ 3 }=\frac { \left< { s }_{ \alpha } \right> }{ \sqrt { MN } }

with degrees of freedom,

{ \left( { \nu }_{ { s }_{ \alpha } } \right) }_{ 3 }=M\left( N-1 \right)

We also assign:

{ \left( { b}_{ \bar { \alpha } } \right) }_{ 3 }=0

            The measurement systematic standard uncertainty is calculated by combining the instrument, the spatial variation, and the temporal variation errors using the root sum square (RSS) method,

{ b }_{ \bar { \alpha } }=\sqrt { { \left( { b }_{ \bar { \alpha } } \right) }_{ 1 }^{ 2 }+{ \left( { b }_{ \bar { \alpha } } \right) }_{ 2 }^{ 2 }+{ \left( { b }_{ \bar { \alpha } } \right) }_{ 3 }^{ 2 } }

And, similarly, for the measurement random standard uncertainty,

s_{ \bar { \alpha } }=\sqrt { { \left( { s }_{ \bar { \alpha } } \right) }_{ 1 }^{ 2 }+{ \left( { s }_{ \bar { \alpha } } \right) }_{ 2 }^{ 2 }+{ \left( { s }_{ \bar { \alpha } } \right) }_{ 3 }^{ 2 } }

Consequently, the combined standard uncertainty in the mean \alpha of the bulk material is calculated as:

u_{ \alpha }=\sqrt { { b }_{ \bar { \alpha } }^{ 2 }+{ s }_{ \bar { \alpha } }^{ 2 } }

And the combined degrees of freedom are found using,

{ \nu }_{ \alpha }=\frac { { \left( \sum _{ k=1 }^{ K }{ { \left( { s }_{ \bar { \alpha } } \right) }_{ k }^{ 2 }+{ { \left( { b }_{ \bar { \alpha } } \right) }_{ k }^{ 2 } } } \right) }^{ 2 } }{ \sum _{ k=1 }^{ K }{ \left( { { \left( { s }_{ \bar { \alpha } } \right) }_{ k }^{ 4 } }/{ { \left( { { \nu }_{ { s }_{ \alpha } } } \right) }_{ k } } \right) } +\sum _{ k=1 }^{ K }{ \left( { { \left( { b }_{ \bar { \alpha } } \right) }_{ k }^{ 4 } }/{ { \left( { { \nu }_{ b_{ \alpha } } } \right) }_{ k } } \right) } }

where k refers to (the subscript of) each instrument, spatial variation, or temporal error. Note that when the degrees of freedom in the systematic uncertainty is large (e.g., 900 – 1 = 899, based on the 900 tests that was done by NETSCH), the second term in the denominator is negligible.

            Finally, at 95% confidence level, the combined standard uncertainty is multiplied by the appropriate t value from the Student’s t Distribution table, which consequently yields the best estimate of the thermal diffusivity of the bulk material:

{ \alpha }^{ , }=\left< \bar { \alpha } \right> \pm { t }_{ { \nu }_{ \alpha },95 }{ u }_{ \alpha }

Uncertainty analysis at 95% confidence level for reporting specific heat measurement: In contrast to the LFA, our NETZSCH STA 449 F3 continuously takes measurement and calculates the specific heat ({ c }_{ p }) of a sample across a set temperature range. In order to match the discrete temperature coordinates of the thermal diffusivity (LFA) measurement, the post-measurement { c }_{ p } data are interpolated using a 1-D linear data interpolation function in MATLAB (i.e. interp1) before their uncertainties are analyzed. This is done so that we are able to accurately report both material’s thermal diffusivity and specific heat, and ultimately estimate its thermal conductivity, at the same temperature coordinates.

            In order to meet our established Measurement Acceptance Criteria,2 at least 3 samples (Z = 3) from 3 different locations in the bulk material are tested. After interpolating the post-measurement data, the mean and the standard deviation at each temperature coordinate is calculated as

\bar { { c }_{ p } } =\frac { 1 }{ Z } \sum _{ z=1 }^{ Z }{ { { c }_{ p } }_{ z } }
{ s }_{ c_{ p } }={ \left[ \frac { 1 }{ Z-1 } \sum _{ z=1 }^{ Z }{ { \left( { c }_{ { p }_{ z } }-\bar { { c }_{ p } } \right) }^{ 2 } } \right] }^{ { 1 }/{ 2 } }

            For STA measurement, uncertainties or errors can be grouped into (1) instrument error, and (2) combined temporal and spatial variation error. Similar to the LFA measurement, the instrument error is assigned a systematic standard uncertainty based on the manufacturer’s statement, which is naturally assumed to be stated at 95% confidence. In this example, the technical datasheet accompanying the NETZSCH STA 449 F3 states an instrument uncertainty or accuracy to within ± 3.5% of the reading (from room temperature to 1500 °C).4 Therefore, the standard uncertainties are assigned as

{ \left( { b }_{ \bar { { c }_{ p } } } \right) }_{ 1 }={ \left( \frac { B_{ \bar { { c }_{ p } } } }{ 2 } \right) }_{ 1 }=\left( \frac { 0.035\times \bar { { c }_{ p } } }{ 2 } \right) \quad \quad \quad \quad \quad { \left( { s }_{ \bar { { c }_{ p } } } \right) }_{ 1 }=0

The subscript keeps track of the error group (e.g., subscript 1 is for the instrument error). No random uncertainty is assigned to the instrument error for a similar reason described for the LFA’s random uncertainty.

            We can combine and estimate the spatial and the temporal variation errors as a single error, because each sample is tested at different time. Each sample that is taken from different locations in the bulk material represent the spatial nonuniformity in the bulk material, which eventually contributes to the spatial variation error. The time variation naturally contributes to the temporal variation error. Together, they assign a random uncertainty of

{ \left( { s }_{ \bar { { c }_{ p } } } \right) }_{ 2 }=\frac { { s }_{ { c }_{ p } } }{ \sqrt { Z } }

with degrees of freedom,

{ \left( { \nu }_{ { s }_{ { c }_{ p } } } \right) }_{ 2 }=Z-1

We do not assign a systematic uncertainty to this error, so .

{ \left( { b }_{ \bar { { c }_{ p } } } \right) }_{ 2 }=0

            Similarly, the standard uncertainties in the STA measurement are combined using the RSS method,

b_{ \bar { { c }_{ p } } }=\sqrt { { \left( b_{ \bar { { c }_{ p } } } \right) }_{ 1 }^{ 2 }+{ \left( b_{ \bar { { c }_{ p } } } \right) }_{ 2 }^{ 2 } }
s_{ \bar { { c }_{ p } } }=\sqrt { { \left( s_{ \bar { { c }_{ p } } } \right) }_{ 1 }^{ 2 }+{ \left( s_{ \bar { { c }_{ p } } } \right) }_{ 2 }^{ 2 } }

Consequently, the combined standard uncertainty in the mean { c }_{ p } of the bulk material is calculated as:

u_{ { c }_{ p } }=\sqrt { { b }_{ \bar { { c }_{ p } } }^{ 2 }+{ s }_{ \bar { { c }_{ p } } }^{ 2 } }

And the combined degrees of freedom are found using,

{ \nu }_{ { c }_{ p } }=\frac { { \left( \sum _{ k=1 }^{ K }{ { \left( { s }_{ \bar { { c }_{ p } } } \right) }_{ k }^{ 2 }+{ { \left( { b }_{ \bar { { c }_{ p } } } \right) }_{ k }^{ 2 } } } \right) }^{ 2 } }{ \sum _{ k=1 }^{ K }{ \left( { { \left( { s }_{ \bar { { c }_{ p } } } \right) }_{ k }^{ 4 } }/{ { \left( { { \nu }_{ { s }_{ { c }_{ p } } } } \right) }_{ k } } \right) } +\sum _{ k=1 }^{ K }{ \left( { { \left( { b }_{ \bar { { c }_{ p } } } \right) }_{ k }^{ 4 } }/{ { \left( { { \nu }_{ b_{ { c }_{ p } } } } \right) }_{ k } } \right) } }

correspondingly assuming that the degrees of freedom in the systematic standard uncertainty is large. In a similar manner, at 95% confidence level, the combined standard uncertainty is multiplied by the t value from the Student’s t Distribution table, which consequently yields the best estimate of the specific heat of the bulk material:

{ { c }_{ p } }^{ , }=\bar { { c }_{ p } } \pm { t }_{ { \nu }_{ { c }_{ p } },95 }{ u }_{ { c }_{ p } }

Propagation of error (uncertainty) to a result at 95% confidence level for reporting the best estimation of thermal conductivity: Thermal conductivity (k), thermal diffusivity, specific heat and density (\rho) are related through, k=\alpha { c }_{ p }\rho. Hence, the best estimate of the mean thermal conductivity is calculated based on that relation,

\bar { k } =\left< \bar { \alpha } \right> \times \bar { { c }_{ p } } \times \rho

For most Gen3 CSP materials, \rho value is provided by the material’s provider or taken from the vendor’s advertised input. It can be assumed constant across the temperature range. We can also assume a negligible systematic and random errors for \rho.

            The uncertainty in the mean k of the bulk material is therefore calculated as follows.  The systematic and random standard uncertainties propagate through to the result (k), and are calculated about the operating point as established by the mean values for \alpha and { c }_{ p }. That is,

s_{ \bar { k } }={ \left[ { \left( \frac { \partial k }{ \partial \alpha } { s_{ \bar { \alpha } } } \right) }^{ 2 }+{ \left( \frac { \partial k }{ \partial { c }_{ p } } { s_{ \bar { { c }_{ p } } } } \right) }^{ 2 } \right] }^{ { 1 }/{ 2 } }

and

b_{ \bar { k } }={ \left[ { \left( \frac { \partial k }{ \partial \alpha } { b_{ \bar { \alpha } } } \right) }^{ 2 }+{ \left( \frac { \partial k }{ \partial { c }_{ p } } { b_{ \bar { { c }_{ p } } } } \right) }^{ 2 } \right] }^{ { 1 }/{ 2 } }

            The degrees of freedom in the k calculation is determined by

{ \nu }_{ k }=\frac { { \left[ { \left( \frac { \partial k }{ \partial \alpha } { s_{ \bar { \alpha } } } \right) }^{ 2 }+{ \left( \frac { \partial k }{ \partial { c }_{ p } } { s_{ \bar { { c }_{ p } } } } \right) }^{ 2 }+{ \left( \frac { \partial k }{ \partial \alpha } { b_{ \bar { \alpha } } } \right) }^{ 2 }+{ \left( \frac { \partial k }{ \partial { c }_{ p } } { b_{ \bar { { c }_{ p } } } } \right) }^{ 2 } \right] }^{ 2 } }{ \left[ { { \left( \frac { \partial k }{ \partial \alpha } { s_{ \bar { \alpha } } } \right) }^{ 4 } }/{ { \nu }_{ { s }_{ \alpha } } }+{ { \left( \frac { \partial k }{ \partial { c }_{ p } } { s_{ \bar { { c }_{ p } } } } \right) }^{ 4 } }/{ { \nu }_{ { s }_{ { c }_{ p } } } } \right] +\left[ { { \left( \frac { \partial k }{ \partial \alpha } { b_{ \bar { \alpha } } } \right) }^{ 4 } }/{ { \nu }_{ { b }_{ \alpha } } }+{ { \left( \frac { \partial k }{ \partial { c }_{ p } } { b_{ \bar { { c }_{ p } } } } \right) }^{ 4 } }/{ { \nu }_{ { b }_{ { c }_{ p } } } } \right] }

with degrees of freedom in the random standard uncertainties to be

{ { \nu }_{ { s }_{ \alpha } } }=M\left( N-1 \right)
{ { \nu }_{ { s }_{ { c }_{ p } } } }=Z-1

and, correspondingly, the degrees of freedom in the systematic standard uncertainties are assumed to be large.

            The expanded uncertainty in the mean value of k is

{ u }_{ k }={ t }_{ { \nu }_{ k },95 }\sqrt { { b }_{ \bar { k } }^{ 2 }+{ s }_{ \bar { k } }^{ 2 } }

Hence, at 95% confidence level, the best estimate of the thermal conductivity is reported as

{ k }^{ , }=\bar { k } \pm { u }_{ k }

This measurement of thermal conductivity has a percentage of uncertainty of

\frac { { u }_{ k } }{ \bar { k } } =\frac { { t }_{ { \nu }_{ k },95 }\sqrt { { b }_{ \bar { k } }^{ 2 }+{ s }_{ \bar { k } }^{ 2 } } }{ \bar { k } } \times 100

which should be <15%, in order to fulfill the Measurement Acceptance Criteria.2

References

  1. R. S. Figliola and D. E. Beasley, Theory and Design for Mechanical Measurements, 5th Edition ed., Hoboken, NJ: John Wiley & Sons, Inc., 2011, (Chapter 5).
  2. Measurement Acceptance Criteria. [Online]. Available: http://gen3csp.gatech.edu/criteria/.
  3. Data Sheet for LFA 467 HT HyperFlash®. [Online]. Available: https://www.netzsch-thermal-analysis.com/us/products-solutions/thermal-diffusivity-conductivity/lfa-467-ht-hyperflash/.
  4. Key Technical Data STA 449 F3 Jupiter®. [Personal Communication with NETZSCH]. Available: upon request.

Andrey Gunawan

Research Engineer II

Andrey Gunawan is a Research Faculty at STEEL, and is currently co-leading this Gen3 CSP project with Shannon Yee. He received his B.S. in Aeronautics & Astronautics from Institut Teknologi Bandung, M.S. in Aerospace Engineering from the University of Southern California, and Ph.D. in Mechanical Engineering from Arizona State University. Gunawan has over four years relevant experience working at high temperatures with challenging material constraints on the SunShot CSP: APOLLO project, developing a high-temperature (~850°C) sodium ion expansion heat engine. He is the recipient of several honors and awards including The Electrochemical Society (ECS) Joseph W. Richards Summer Fellowship (2014), Honorable Mention of the Link Foundation Energy Fellowship (2014), and together with his co-authors he received the Best Paper Award(s) in the ASME Power Division Student Paper Competition at the ASME Power & Energy 2015 and 2018 conferences.

About the Solar Energy Technologies Office

The U.S. Department of Energy Solar Energy Technologies Office supports early-stage research and development to improve the affordability, reliability, and performance of solar technologies on the grid. Learn more at energy.gov/solar-office

Acknowledgement

This work was funded in part or whole by the U.S. Department of Energy’s Office of Energy Efficiency and Renewable Energy (EERE) under the Solar Energy Technologies Office Award Number DE-EE0008371.