Keywords

1 Introduction

In order to supply high-quality products to the market, manufacturing industries have given product inspection as much attention as processing and assembling. There are two types of inspections: functional inspection and appearance inspection. In functional inspection, the effectiveness of a product is inspected. In appearance inspection, small visual defects such as scratches, surface dents and unevenness of the coating color are inspected. The automation of functional inspection has advanced because it is easy to determine whether a product works or not [1]. However, it is not as simple to establish the standards to determine whether the appearance of a product is defective. First, there are many different types of defects. Second, the categorization of a product as non-defective or defective is affected by the size and depth of the defect. Third, some products have recently become smaller and more detailed. Finally, production has shifted to high-mix, low-volume production. It is thus difficult to develop technologies that can discover small defects and to create algorithms that identify multiple types of defects with high precision. Therefore, appearance inspection still depends on visual inspection using human senses [2].

Recently, a visual inspection method utilizing peripheral vision was proposed [3–7], and the effectiveness of the method has been tested in manufacturing factories [8]. Human vision is divided into two ranges. Central vision is the 1–2° range of vision on either side of the center of the retina. The remaining range of vision is called peripheral vision. The spatial resolution of human vision decreases significantly with increasing angle from the center of the retina [9]. The visual inspection method utilizing peripheral vision involves two process: first, a wide range is searched by peripheral vision; then, the defect is decided by the high spatial resolution of the central vision. Thus, low-level processes such as sampling and characteristics clustering are processed by peripheral vision, next high-level processes such as discrimination is processes by central vision to reduce the amount of information to be processed. From this, an efficient visual information processing is realized [10], the visual inspection method utilizing peripheral vision which can be realized high accurate inspection in a short time has been expected.

In order to prevent dust in the air and on clothing from adhering to the product, it is generally recommended that visual inspection for a high-quality product be performed in a clean room. However, in situations where the production process becomes more complex and more outsourcing in recent years, it is often not conducted in a clean room. Also, there are many cases where visual inspection process is not a clean room because of constraints of existing equipment and economic problems. This is the present situation that must be inspected the dirty inspection surface in visual inspection. That is, in a situation where both the dirt that can be removed and the defect that cannot be removed are mixed, inspectors are required to detect only the defect. This causes reduced defect detection accuracy.

In order to consider the relationship between the amount of dirt on the inspection surface and accuracy of defect detection, this study analyzes images of inspection surfaces in an actual factory and creates a model for dirt density based on the pixel values of the images. Then, experiments are conducted using dirt density, defect location and defect characteristics as experimental factors. The effects of these factors on defect detection accuracy are examined.

2 Analysis and Modeling of Dirt

2.1 Analysis of Dirt in Actual Factory

In order to analyze conditions of dirt on an inspection surface, a field survey at Company X, which performs visual inspections, was implemented. The company produces transparent parts for automobiles. Neither the production process nor the visual inspection process is conducted in a clean room. Therefore, the visual inspection is performed in location where the inspection surface is dirty.

In the field survey, images of the inspection surface used for the visual inspection process are taken by a camera; these images are then sorted in order of dirtiness. In order to evaluate the distribution of pixel values, histograms of these images are analyzed as shown in Fig. 1. As the results, it was obtained that both the dispersion of pixel values in the image and the shape of the histograms are changed according to the amount of dirt on the inspection surface.

Fig. 1.
figure 1

Frequency of pixel value for each photographed image

Full size image

2.2 Modeling of Dirt

2.2.1 Quantification of Dirt

The analysis method explained in Sect. 2.1 is applied to the inspection surface in the factory in order to quantify the dirt. A known amount of dirt is intentionally scattered on the inspection surface intentionally, and the relationship between the applied and the distribution of pixel values in the photographed image is considered.

As for an inspection surface of a product, the product with a height of 100 mm and a width of 100 mm in the company is used. As for dirt, the powders with a diameter of 0.18–0.25 mm are used. The powders are scattered during the production processes to prevent the products from coming into contact with each other, and those are one of the typical dirt in the visual inspection process.

The amount of scattered powder varied from 0.0 g to 1.0 g, in 0.1 g increments, and the histogram for each level of powder is examined. The results for the 0.0 g, 0.3 g, 0.6 g and 0.9 g levels are shown in the bar graph in Fig. 2. It was obtained that the dispersion of pixel values in the image is varied according to the amount of scattered powder.

Fig. 2.
figure 2

Frequency and probability of pixel value for each scattered amount of dirt

Full size image

2.2.2 Formulation of Dirt

Using the pixel values derived in Sect. 2.2.1, to formulation of the dirt is considered. Specifically, to determine the dispersion of pixel values for each amount of scattered powder, kernel density estimation is applied. The commonly used Gaussian kernel function k(x) (Reference) is employed in this study. It is calculated using Eq. (1), where x is the frequencies of the pixel value. Bandwidth h is calculated in Eq. (2) using the standard deviation, σ, of the frequencies of the pixel values. Then, the kernel dispersion \( {\hat{\text{f}}}\text{k(x)} \) is calculated in Eq. (3) using the kernel function k(x) and the bandwidth h. The results for the 0.0 g, 0.3 g, 0.6 g and 0.9 g levels are shown in the line graph in Fig. 2. It was found that it is possible to estimate the probability density function of the distribution of pixel values with high accuracy.

$$ K(x) = \frac{1}{{\sqrt {2\pi } }}e^{{ - \frac{1}{2}x^{2} }} $$
(1)
$$ h = \frac{0.9\sigma }{{256^{\frac{1}{5}} }} $$
(2)
$$ \hat{f}k(x) = \frac{1}{256h}\sum\nolimits_{i = 0}^{255} {k(\frac{x - xi}{h})} $$
(3)

2.2.3 Imaging of Dirt

Using the probability density function shown in Sect. 2.2.2, the ability to create images of specific amounts of dirt is considered. First, the pixel values are assigned to create image according to the probability density function. Then the pixels are colored according to the pixel values. The results for the 0.0 g, 0.3 g, 0.6 g and 0.9 g levels are shown in image of Fig. 3. It was found that it is possible to create the images corresponding to desired amounts of scattered powder.

Fig. 3.
figure 3

Created images for each application amount of dirt

Full size image

3 Experimental Design

3.1 Experimental Task

Experimental subjects are tasked with visual inspecting a model that is displayed on a monitor (CG276, EIZO Inc.). A model with a height of 300 mm, and a width of 300 mm, and a black 10 mm diameter circle (used as a fixation point) on the center is used. This model is shown in Fig. 4.

Fig. 4.
figure 4

Inspection model

Full size image

In order to lead inspection utilizing peripheral vision, the subjects are requested to fix only at the fixation point during the experiment. If no defect is detected, the subject presses the SPACE KEY on the key board, and the next inspection model will be displayed. If a defect is detected, the subject presses the ENTER KEY.

The experimental layout is shown in Fig. 5. In order to ensure a uniform viewing distance between each subject and the inspection model, the chin holder is placed at 400 mm from the inspection model to fix the head of a subject.

Fig. 5.
figure 5

Experimental layout

Full size image

3.2 Experimental Factors

3.2.1 Dirt Density of the Inspection Model

The dirt density with each inspection model is realized by using a background image that is created as described in Sect. 2.2.3. Four images (0.0 g, 0.3 g, 0.6 g and 0.9 g) shown in Fig. 3 are employed in the experiment. The image of 0.9 g of dirt was chosen as the upper limit, after factory of Company X confirmed that the inspection surface is never dirtier than the image representing 0.9 g of dirt. Hereafter, the four types of the inspection model are called Non dirty (0.0 g), Slightly dirty (0.3 g), Somewhat dirty (0.6 g) and Very dirty (0.9 g) respectively.

3.2.2 Defect Location

The inspection model is divided into sixteen parts (4 × 4 horizontally and vertically), and the defect is located at the center of either one of these parts. As shown in Fig. 6, the parts are divided into four areas, from area (1) to area (4) according to the distance from the fixation point.

Fig. 6.
figure 6

Defect location

Full size image

3.2.3 Defect Characteristics

The characteristics of the defects are defined by the luminance contrast between the inspection model and the defect, and the size. The shape of all defects is circular. The luminance contrast of each defect is one of three different levels: 0.10, 0.15 and 0.20. There are also two types of defects in the factory: those that are darker than the inspection surface, such as scratches, and those that are brighter than defects than the inspection surface, such as adhered inks. Therefore, the luminance contrast of each defect is also specified as either dark or bright.

The size of the defect is specified by a diameters of 1.0 mm, 1.5 mm and 2.0 mm. These defects are determined by assuming the standard of the appearance inspection.

All defects employed in the experiment are shown in Fig. 7. The experiment is conducted for the four different location areas (sixteen parts) of defect, three different luminance contrast levels, two different types of defect, and three different sizes of defect. This gives a total of 288 (location parts (16) × luminance contrast (3) × dark and bright (2) × size (3)) defective inspection models for each level of dirt on the inspection models.

Fig. 7.
figure 7

Defects employed in experiment

Full size image

3.3 Experimental Procedure

Twelve subjects, between aged of 21 and 26 years, are employed in this experiment. Only subjects with a corrected eyesight score greater than 1.0 are employed. In order to familiarize the subjects with the experiment, an overview and the procedure of the experiment are explained, and the preliminary experiment are tasked. In the experiment, to inspect 578 (288 non-defective and 288 defective) inspection models is tasked for each dirt of the inspection model.

The experimental room temperature is set between 18 and 24 °C, and the humidity is set between 40 and 60 %. Since the luminance of the inspection model and the defect are affected by external and internal light (such as fluorescent lighting), the experiment is conducted in a dark room. The purpose and contents of the experiment are explained to the subjects in writing, and the informed consent of all subjects is obtained.

Using the results of the experiment that are obtained by the above procedures, the defect detection rate is calculated, which is the number of detected defects divided by the number of total defect. It is expressed in Eq. (4) and used as the evaluation index.

$$ Defect \;detection\;rate [\% ]= \frac{Number \;of\;detected\;defects}{Number \;of\;total\;defects} $$
(4)

4 Experimental Results and Discussion

4.1 Individual Characteristics of Subjects

Using the defect detection rate, the effect of the dirt of the inspection model is examined. Owing to the possibility that the individuality of the subject might have affected the result, the uniformity of the results for all subjects is verified.

The defect detection rate of the subjects for each level of dirt on the inspection model is shown in Fig. 8. As the result of the Smirnov-Grubbs test (significance level 1 %) shows, there are no outlier values in the defect detection rates of any of the subjects. Therefore, the data from all twelve subjects are used.

Fig. 8.
figure 8

Defect detection rate for each subjects

Full size image

4.2 Effect of Dirt on the Inspection Model on Defect Detection

The effect of the dirt on the inspection model on the defect detection rate is shown in Fig. 9. The defect detection rate becomes significantly less as the inspection model becomes dirtier. One-way ANOVA (analysis of variance) is executed with the dirt on the inspection model (4) as for a factor. As the result, a significant difference of 1 % is observed for the main effect [F(3, 44) = 45.17, p < 0.01]. In addition, as sub-effect tests of the main effect, an analysis of multiple comparisons is executed. As a result, a significant difference of 1 % is 6 observed between Non dirty and the other levels, and between Slightly dirty and Very dirty, whereas a significant difference of 5 % is observed between Slightly dirty and Somewhat dirty.

Fig. 9.
figure 9

Defect detection rate for each inspection model

Full size image

Based on the above results, in order to realize high defect detection accuracy with visual inspection utilizing peripheral vision, such inspection should be performed only when there is no dirt on the inspection surface. To accomplish this, it is necessary to take measures such as washing the inspection surface prior to the visual inspection process or making a clean room for the visual inspection process.

5 Conclusion

In order to consider the relationship between the dirt on the inspection surface and defect detection in visual inspection utilizing peripheral vision, images of an inspection surface in an actual factory were analyzed, and the dirt was modeled. Then, the dirt density of an inspection model, defect location and defect characteristics were designed as experimental factors, and the experiment was conducted with twelve subjects. As the result, it is clarified that the defect detection rate becomes significantly less as the inspection surface becomes dirtier. Consequently, it is shown that the importance of a clean inspection surface for a highly accurate visual inspection process.

In future studies, we will consider in more detail the relationship between the dirt on the inspection surface and defect detection for each location area and for each characteristics of the defect.