Purpose of the compliance level (CL)
In order to validate data archived, the Registry team uses a curation process that
includes a method to assess and communicate the quality of the data. A rating metric,
called compliance level, was defined and is applied to the data. The rating is reflective
of the quantity and quality of characterization performed and reported for a nanomaterial.
The Registry team is currently reviewing the Data Readiness Levels Draft, as introduced
by the National Nanotechnology Initiative, and anticipates applying these concepts
to the Registry compliance level once they have matured and been accepted by the
community.
In the Registry site, compliance levels are represented by medals. This document
describes how compliance level is calculated.
Compliance Level and PCCs
The Registry's Minimum Information About Nanomaterials (MIAN) for Physico-Chemical
Characteristics (PCCs) is a partitioning of the MIAN data into twelve categories
which encapsulate the most relevant information about a material (e.g. Particle
Size). For each PCC, a record is assigned a Compliance Level (CLPCC): a calculated
value which represents that record's adherence to community standards in characterizing
and reporting nanomaterial properties. Records are awarded one of four medals based
on CL scores for each PCC (from best characterized to least): Gold, Silver, Bronze,
and Merit.
|
Medal Awarded
|
Icon
|
CLpcc
|
|
Gold
|
|
90 — 100
|
|
Silver
|
|
75 — 89
|
|
Bronze
|
|
60 — 74
|
|
Merit
|
|
< 60
|
Terminology
The following terminology is used in the discussions of CL that follow:
|
PCC
|
Default Stakeholder Weighting Factors
|
|
Group
|
Data associated with a given PCC are broadly categorized into several groups: Measurement
Type, Technique, Instrument, and several meta-data groups (uncertainty, replicates,
etc.)
|
|
Element
|
Each group consists of elements; data which are categorized into a given group are
called that group's elements.
(For the group "Measurement Type", "Mean Hydrodynamic Diameter"
is an example of an element)
|
|
Multiplier
|
Each group has associated with it an integer value, called its multiplier,
which defines the importance of the group to the compliance level score
|
|
Points
|
Each element of each group has associated with it an integer, called its points
value.
|
|
Weight
|
For each group within a PCC, the weight is the product of the group multiplier and
the
maximum points value assigned to any element of that group.
|
|
Measurment
|
A measurement is a single characterization of a particle. It may include data for
any or all of the groups for a given PCC.
|
The compliance level score is a quantitative value calculated by an algorithm that
runs in the backend of the Registry Curation tool. In general, the compliance calculation
awards a higher point value to group elements with greater specificity (ex. "mean
aerodynamic diameter" vs. "size").
The Algorithm
The algorithm used to calculate CL is given by the following expression:
G = group • MG
= group multiplier • PG
= points awarded to Element P of group G • WG
= weight assigned to group
For a given PCC, the calculation proceeds by assigning a score to each measurement;
these scores are summed over all measurements associated with the data record to
give the PCC CL score. This sum is capped at the maximum score that a single measurement
can receive; if the sum of the contributions of each of the measurements associated
with a data record to the PCC CL exceeds this value then the CL for that PCC is
set to this max value.
For each PCC, the scores for each measurement associated with a given data record
are calculated by the algorithm shown above. For each group, G, the associated multiplier,
MG, is multiplied by the ratio of the points value associated with the relevant
element in G to the weight for G; these values are summed over all groups to give
the numerator in this expression. The denominator is simply the sum of the multipliers
over all the groups.
Intuitively, the maximum CL score a data record can achieve for a given PCC is the
sum of all its group multipliers. The points awarded is the sum of those same group
multipliers as in the denominator, but with each term scaled back by the ratio of
the points to the weight (which is always less than or equal to 1).
A backend tool allows the Registry team to quickly and easily update elements that
affect the calculation of the CL so that new developments in the scientific community
can be reflected in the CL as needed.