← Previous: Visually Evaluating Image Quality

Numerically Evaluating Image Quality

Digitization Program Planning

The unpredictable nature of color has compelled most major cultural heritage institutions to include numerical analysis of image quality into their quality control workflows. There are a variety of ways this can be accomplished, and when properly implemented, can significantly contribute to the creation of consistent and correct PDOs. However, it is critical to avoid blind pursuit of numeric perfection. Numbers are oblivious to intention and context. Each image quality metric (e.g. illumination uniformity, sampling efficiency, etc.) can be analyzed by a machine, but such analysis should not be followed without context and understanding.


Test Targets & Spectral Measurements

There are three ways to confirm that the PDO has accurate color and tone: evaluation of a Device Level Target, evaluation of an Object Level Target, or a direct measurement of the object with a spectrometer.

Device Level Target (DLT)

A Device Level Target fills most or all of the captured scene. This allows measurements to be taken across the entire frame and provide extensive visual features for analysis. For example, a device level target may have dozens, or even hundreds, of color patches, a full grayscale ramp, an inch/cm ruler or other dimensional reference, a slanted edge target, gradually converging lines with annotated cycle rates, and several identical patches spread out across the target (to evaluate various kinds of intra-frame consistency).

However, a device level target, by definition, can only be included in substitution for an object; it is not intended to be imaged alongside an object. Therefore, this type of target is most useful for calibration and process QC, by including one at the beginning and end of a production run, for example. Note that a color chart by itself, such as the X-Rite Color Checker, is not a device level target. Such color charts only contain color patches. Wherever possible, a true Device Level Target like the one produced by Image Science Associates is strongly preferred as it provides a variety of test features beyond color patches.



Object Level Target (OLT)

An OLT is small enough, and of an appropriate narrow aspect ratio, to sit alongside an object during the course of a normal digitization production run. Many institutions mandate that all digitized material include an Object Level Target in-frame.

An Object Level Target usually carries features similar to a Device Level Target. The range of these features is not as large as Device Level Targets because of their smaller size. However, high quality targets like those from Image Science Associates still provide for evaluation of color, tone, DPI, and SFR.

Some institutions are still making use of legacy targets such as the Color Control Patches from Kodak. These sorts of targets were developed primarily for printing-press workflows and color-separation films decades before the advent of modern imaging guidelines like FADGI and METAMORFOZE. The manufacturing tolerances of these targets were not very tight; a patch on a specific copy of these legacy targets may vary considerably from the stated “correct” value of that patch. Modern targets such as those from Image Science Associates are strongly preferred over legacy targets because they were developed for digital image creation and are made to very tight tolerances.

Measurement from the Object

A spectrometer such as the X-Rite i1 can be used to make a spectral measurement of a specific point on a object. This can be useful where the object may contain material which suffers strong metameric failure, or for forensic or scientific analyses. However, it has four significant limitations in mass digitization programs: speed, location specificity, and illumination angle, and the requirement for contact.

Speed: Measurements using a spectrometer are tedious time consuming. Each measurement only takes a moment, but can only be taken after the location has been carefully selected and the spectrometer precisely positioned.

Location Specificity: Each measurement is inherently linked to a specific location on the object, so the exact location on the object must be carefully recorded. Moreover, it’s most useful to select an area of homogenous tone since it can be difficult to correlate the location in the PDO with sufficient precision to isolate a particular brush stroke or detail. This eliminates (or at least greatly challenges) use in areas of high frequency detail.

Illumination Angle: Most spectrometers provide illumination which is fixed at 45°. Those with “flexible” angles of illumination are locked to a selection of three or four hard-wired angles (e.g. 30/45/60°). There are systems called goniospectrometers, and allow for a full spectral reading at any arbitrary angle of illumination, but they are incredibly slow to use; they are useful for research, but are not suitable for use in a mass digitization program. In practice, nearly all such measurements made in Cultural Heritage environments are made with a spectrometer that illuminates the object at 45°. This is in stark contrast to the actuality of digitizing real world objects, which often require non 45° illumination to avoid issues like glare and reflections.

Requirement for Contact: Most commercial spectrophotometers require direct contact, or extremely close proximity, with the surface being measured. This limits their applicability for objects where handling protocols may forbid direct contact (or high risk of direct contact). Spectrophotometers capable of taking measurements at a distance are available, but are manufactured for the scientific-research market and carry commensurate price, size, and technical acumen.

Target Analyzers

An Object Level Target or Device Level Target in a PDO must be analyzed to be useful. Several software packages exist which can automate this process, including the GoldenThread software package from Image Science Associates, and the Delt.ae website.

ISA GoldenThread – This software is especially useful for continuing process control, as it provides professional-level traceability and quality auditing for long term digitization projects.The Four-Day Weekend Effect [see The Four-Day Weekend Effect] provides an interesting example of what insights such traceability can provide. The author, Don Williams, sits on the Still Image Digitization Advisory Board that created the FADGI Guidelines, and has taken great care to ensure that the exact algorithms and implementations of GoldenThread comply with the spirit and letter of the FADGI Guidelines.

Delt.ae – This website is provided by the European for-profit digitization company Picturae. It is notable as a no-cost solution for basic target evaluation. Development has been sparse since 2012, but basic support is provided for a variety of targets. Certain specifics of their implementation are not ideal, so the results from GoldenThread should be prefered when results between them differ. Since the tool is hosted online, rather than installed locally, it can be limited by the connection speed and network policies of the client. Purchasing GoldenThread ensures perpetual access, whereas there is no guarantee the Delt.ae website will be available indefinitely.

Have a question or want to know more?
Fields marked with an * are required