Image quality test in CCTV.
Contents
How to read the test pattern ?
Introduction.
Test station arrangement.
Picture framing procedure.
Image sharpness adjustment and test.
Monitor image geometry test.
Reflection check test.
Camera resolution test.
Camera picture band.
Grey scale.
Colours imaging.
Person identification.
Character recognition.
Observations and conclusions.
We present You the "test pattern", designed by our company, which is the result of many weeks of our work, connected with a few hundreds of tests and trials, to let the CCTV cameras parameters to be measured.
The basic purpose of this test pattern is to check the individual elements of closed circuit television system, as well the whole system in respect of established requirements fulfilment.
Using the characteristic test pattern elements it is possible to evaluate the quality and draw the conclusion regarding the used camera and lens. We may also evaluate the image quality on the industrial monitor and measure some parameters.
In case of archiving systems we can see, how the image quality changes, depending on used method, resolution and recording compression.
The test pattern allows for assessment of system and its elements in respect of technical parameters, and suitability for the following applications: human activity, person and character recognition.
The pattern may be downloaded from our website: http://www.delta.poznan.pl/download/image_quality_test.zip
and printed with the colour ink printer, having the minimum resolution of 600 dpi.
Fig.1.
Prepared test pattern.
The test pattern may be divided into a few functional parts. The central part allows for precise focusing and monitor geometry check.
Lower to the right there are resolution and camera picture bandwidth tests, above them the reflection check in the picture channel test.
In the left part there are person and human activity recognition tests.
In the right part there is the character recognition test with various character sizes.
The greyscale strip in the bottom allows for camera/monitor gamma factor check (the optimal picture brightness and contrast setting).
The colours strip in the top allows to check the colours imaging correctness.
The sharpness test has been placed also in the test pattern corners.
The important elements of the test are white and black triangles on the edge, which allow for proper camera visual field framing.
In the next chapters the test station arrangement and its elements preparation will be described. Then, the individual tests with the procedure reasons will be presented. The test procedures description is enriched with the real measurements examples.
Contents
The test station should meet some requirements. The test pattern screen must be fixed horizontally in the location, which allows for free camera placement in front of it, and proper screen lighting. The screen must be fixed to the rigid base. The screen should be preferably lighted uniformly from both sides with the scattered light. The lighting should be positioned in the way to avoid the reflections on the screen in the image observed by the camera.
The camera should be preferably fixed on the tripod. The photographic tripod may be used. It has the three-dimensional orientation adjustment of the fixed instrument and the height adjustment, which allows for precise exposure conditions setting. The tripod should also be placed on the horizontal surface, so the camera position changes do not require the readjustment in all planes. The helpful element is the circle level, which is the element of many tripods.
The camera on the tripod should be oriented in relation with the screen in the way, that the camera and lens optical axis extension intersect the test screen in the sharpness test centre. Using the variable focal length (zoom) lens it is possible to check the camera positioning by zooming the test pattern centre in the camera visual field - if setting is correct, the test screen centre will not "escape".
he proper lens selection is also very important matter. The cameras are equipped with various sizes of image sensors. We can distinguish sensors of the following sizes 1", 2/3", 1/2",
1/3", 1/4". On the Polish market the CCTV cameras with 1/3"
and 1/4" sensors are most often used. It is worth to mention, that the camera lens is normalized to the particular sensor size. It is allowable to use the lens, normalized to the bigger sensor in the smaller sensor camera, but not inversely (the camera image will have black envelope). Two types of lens fixing are used - C and CS. The C lens may be used in C and CS cameras, and the CS fixing requires the distance ring between the lens and the camera - otherwise image focusing in required distance will not be possible. Another element, which should be mentioned, is the lens focal length. In case of 1/2" sensor cameras the 12 - 25 mm lenses are the good choice. In cameras with sensors 1/3" and 1/4" correspondingly: 8 - 16 mm and 6 - 12 mm. Using shorter focal lengths gives the camera visual field larger than 30°, which causes the picture spherical distortions. Larger focal lengths cause the distance between the camera and the test screen increasing. The good solution is using the variable focal length lens (manual adjustment), which significantly facilitates the visual field framing process (see next chapter).
It is good, if the lens also has the manual iris, which allows for optimal image brightness and contrast setting, depending on the camera sensitivity and screen lighting.
The various cameras tests should be preferably performed with the same lens, which is the condition of comparable results.
Fig.2. Exemplary test station.
Finally, it is worth to pay some attention to the monitors. When we want to test the cameras resolution, the best method is using the monochrome monitor with the highest possible resolution (it is recommended, that the monitor has resolution of the order of 1000 TV lines in the centre).
Colour monitors have lower resolutions. Using the colour monitor is allowable, but it should have resolution comparable with TV set resolution. Using these monitors, we can test the lower resolution cameras - below 500 TV lines (most colour cameras have resolution up to 480 lines).
The camera test image may also be split into the particular CCTV devices, or input to the monitoring system, testing this way the final image quality, which may be great comparing criterion of the individual devices or the whole solution, which is to be used.
In case of archiving systems it is also possible to test the recording quality of the particular device or software, depending on the individual recording parameters (resolution, compression, recording method, etc.).
Contents
The basis of reliable readings is the correct test pattern image framing in the camera visual field.
After the initial activities, consisting of test screen horizontal fixing, choosing the proper lens and fixing the camera on the tripod, we must frame the visual field. Place the camera, so the lens optical axis is perpendicular to the test screen surface, and intersects it in the very centre (sharpness test centre). Practically, the camera should be levelled at the proper height in front of the test screen, so the central lens point is equally distant to every test pattern corner. Having the variable focal length lens, we can easily check it, by zooming the test centre, if the optical axis intersects the centre, during zooming the test pattern will be centered in the monitor.
When the camera is placed perpendicularly at the proper height, we must set the distance from the test screen (move the tripod perpendicularly along the straight line) to frame the image properly. Using the variable focal length lens, we can zoom the image (in the appropriate range), instead of moving the whole tripod. Check the focus after each correction.
Fig.3.
The image, which should be visible in the camera visual field after the proper test pattern framing.
The image must be framed in the way, that the black frame with white triangles should hide outside of visual field, but the black triangles should be still visible. It sometimes happens, that the image may not be precisely framed horizontally and vertically (the monitor proportions may slightly differ from 4:3 proportion) - it is important to exactly adjust the image horizontally. Many monitors cut off the image edges, so the synchronization impulses are not visible.
Now, pay attention to the fact, that the traditional industrial monitor does not display 100% of image on the screen. Some monitors are equipped with the underscan function, allowing for monitor operation mode change, so 100% of displayed image is visible. The framing should be performed with the activated function.
In case we have the standard monitor, which has no underscan function, about 10% of the image (at the edges, especially in the corners) is not displayed. The, to properly frame the test, perform the following activities:
-
Set the camera, so the black triangles at the test edge are visible possibly close to the monitor image edge.
-
Using the monitor vertical synchronization adjustment knob (V-hold) change the setting in the way to get the stable image of the vertical synchronization signal (horizontal black stripe between two image fields). Find the setting, which displays that signal in the screen centre.
-
The horizontal and vertical edges, which were invisible in the screen corners, are now located in the central monitor part, which allows for test pattern framing.
-
The positioning triangles on the vertical edges are now located closer to the monitor central part (smaller distortion) - frame horizontally, locating the black triangles at the image edge. The horizontal edges are visible with the synchronization signal - frame vertically, locating the black triangles, so they touch the black synchronization stripe edge.
Fig.4. Adjusting the standard monitor image to frame the test.
All image processing systems in the tested camera must be deactivated (e.g.: AGC, BLC, Auto-Iris).
Contents
The image sharpness is adjusted, using the circular linear test in the test screen centre. The image sharpness should be adjusted with the largest possible lens iris hole (so the image is still clear), because the depth of focus is the smallest, which allows for more precise setting. The sharpness adjustment consists in observation of grey area in the test centre, when we stop to distinguish the black and white lines. As we improve the sharpness, the grey blurred area decreases. The best sharpness corresponds with the smallest area we can see. For various cameras that area size may differ - it depends on various factors, e.g. the camera resolution.
Fig.5. Sharpness test
After the precise sharpness setting we can reduce the iris hole to the medium values (F/5.6, F/8), which give the best lens optical resolution.
In the test screen corners there are also the sharpness tests, which allow for sharpness comparing between the visual field centre and edges. It may be one of comparing criteria for the lenses.
The sharpness adjustment example is shown below:
Fig.6.
The image sharpness is correctly set in the middle example.
Contents
For the approximate image geometry testing the sharpness test, due to its shape (circle), may be used, as well as the circle drawn at the camera visual field height, and the image linearity test.
Noticing, that the circle and sharpness test are not circular, and the linearity test fields differ from the squares means, that the image is flattened or stretched in the given plane. The linearity test, besides the monitor linearity, allows also for detecting the serious lens or camera sensor faults - if we notice the irregular imaging in the tested area.
Fig.7.
Picture geometry imaging elements.
That test does not allow for precise geometry test, but enables detection of serious shape imaging faults in the visual field centre. It is important, because the image quality should be the best in the visual field centre, and if there are some irregularities found, the further tests are not valid. It may also help in proper selection of lens and monitor for the tests.
Contents
Fig.8.
Reflection check test.
The reflection check may be often caused in the coaxial cable, in case of its damage or camera output and monitor input mismatch (impedance). In case of long cables that phenomenon is more visible than in short cables, where it may be not noticeable. So the cable between camera and monitor should not be too long. Of course, we use the coaxial cable with the impedance 75 Ohms. The industrial monitor has the switch, adjusting the signal input impedance, located on its back. Make sure it is set at 75 Ohms.
The reflection check is visible as the image element displacement to the right. If there are more such displacements, we can talk about the multiple reflection. The reflection is best visible on high contrast, clear shapes and edges. The reflection in our test is also well visible on the linearity test lines.
The example below shows the reflection check phenomenon:
Fig.9.
The reflection on the test image. The rectangles displacements are visible.
The reflection check should be eliminated, as it does not allow for precise reading of resolution tests.
Contents
Using this test we can read the horizontal and vertical image resolution.
The resolution tests are based upon the following statement: if the test screen area is precisely framed in the camera visual field, then we can assume, that the test are width / height equals camera sensor width / height. We know the test area height, and we can divide it into e.g. 400 parts and draw 400 lines, each with width 1/400 of interval, equal the test height. Those lines should be well distinguished, so preferably they should be alternately black and white. When we observe such lines on the camera image and we can clearly distinguish them, it means, that the camera resolution must have at least as many lines as we can see, i.e. 400 in above example. To define the lines resolution we of course don't need 400 lines, only few is enough.
The industrial cameras have various sensors resolutions. The camera resolution is the resultant of image sensor resolution (expressed in pixels) and the electronic systems, used for signal processing. The most frequently used cameras have resolutions from 240 TVL (television lines) to 600 TVL. Therefore we need, using the same method as above, define various lines samples with various thickness (corresponding with the defined lines number on the interval, whose length equals the test height). To increase the test legibility there are lines of variable thickness with the scale, showing which number of lines, i.e. what camera resolution each sample corresponds to. The test scale includes values from 240 to 640 TVL, divided into ranges 20 lines each.
With such linear test the resolution reading is based upon finding the place, where the black and white lines cannot be distinguished. That point is not clear, so the reading allows for defining resolution with the precision of each range, i.e. 20 lines.
Fig.10.
Resolution test.
There is also one more problem, which concerns the lines resolution reading limit. There is the interference between the lines, drawn on the test screen, and the camera sensor structure (fig. 11 below), which significantly hinders defining, if the given range lines are still visible or not, because the interference lines start to overlay the image. This phenomenon is especially visible at higher resolutions (thinner lines). The monitor screen structure itself also hinders the precise reading. Due to above problems, the test lines are not horizontal (vertical), but inclined at some angle, which decreases the interference phenomenon. The inclination angle has been selected after many trials, so the interference lines are least visible.
From the vertical test the horizontal resolution may be read, which equals the camera sensor resolution. On the horizontal test we read the vertical image resolution.
Fig.11.
Interference phenomenon, visible in the upper part as the cross.
Below the exemplary cameras resolution readings are shown:
(a) (b) (c)
Fig.12.
The exemplary cameras resolution:
(a) - CMOS camera -
read resolution: 320 lines,
(b) - CCD camera -
read resolution: 340 lines,
(c) - CCD camera - read resolution: 460 - 480 lines.
The resolution reading requires some practice, which comes after a few tests.
The screen dumps slightly decrease the presented samples quality.
In fact they are more clear.
Contents
Fig.13.
Camera picture bandwidth test.
The camera picture bandwidth (MHz) defines, in most simple way, the camera ability to image the picture details. Higher vertical resolution of camera sensor forces the wider picture bandwidth of the electronics. The higher that parameter is, the higher resolution of camera sensor will be, therefore the image will be less blurred (with better sharpness and detail level).
The test lines sample thickness may be determined upon the basis of displaying time of the whole picture line. The determined black and white line thickness corresponds to the certain impulse duration time, which is the inverse of sample frequency, characterizing the bandwidth.
This parameter value is read from the test in the following way:
-
See, which lines sample is still well distinguished (alternating black and white lines are visible).
-
The searched parameter determines in MHz value of next sample (where the lines become blurred). This parameter reading method is used, because the limit value cannot be observed.
Fig.14a.
Camera picture band measurement.
The last distinguished sample is 3 MHz, the camera picture band is 4 MHz - the lines start to get blurry (left example). In the right example it is correspondingly 4 MHz, so the band is 5 MHz.
Fig.14b.
Colour stripes, resulting from the sensor structure.
In case of colour cameras an interesting phenomenon may be observed, the red and blue stripes. That effect results from the CMOS / CCD matrix structure (colour filters on the matrix pixels). The amplifiers with low parameters may limit the camera picture band. If the amplifier in the monitoring system does not transmit the whole camera band, the lower quality image on the monitor is the result, and the image directly from the camera would be better.
Contents
Fig.15.
Grey scale.
he greyscale test allows for checking the gamma factor of camera and monitor. The gamma factor determines the image characteristics regarding the contrast and brightness. The linear greyscale has been chosen (the logarithmic scale is also used), because most of cameras have linear characteristics.
The greyscale allows also for optimal adjustment of monitor image brightness and contrast. Setting of the best possible monitor image may be performed in the following way:
-
Set the camera video signal at 1Vpp (with the camera set at the framed test screen picture).
-
Set the contrast knob in the middle position.
-
Set the brightness knob in the position, where all grey shades on the scale are visible. Correct the contrast setting, when required.
-
Pay attention for the lighting conditions in the room, which influence the contrast and brightness levels.
-
It is important, that the monitor screen is not too lighted, so the brightness level may be adjusted at lower values. In such case we get better sharpness of monitor electrons beam (less electrons are used). The monitor screen usage is less, and the picture has better sharpness.
Contents
Fig.16.
Colours imaging test.
This test is suitable for testing the colours imaging fidelity in colour cameras. The colours scale is the same as in typical TV test. The colours of two basic colours models, i.e. CMY and RGB, as well as white and black, are presented in the proper sequence.
The test allows also for testing the colour camera in respect of colours vision in low light. We can observe, how the camera reacts for light changes, and if individual colours are still distinguished.
The proper colours imaging by the camera is influenced by the light source colour temperature and camera white balance automatic adjustment (if such adjustment exists). The light source colour temperature should be similar to daylight. In case of colour cameras, after switching on lightning, the camera should be turned on again, so the white balance systems may find the right setting.
Contents
The tests group on the right has been prepared to test the system ability to recognize the human face and motion.
The basic assumption during the test preparing was determining the specific assessment criterion, which allows to say, that in given conditions the human face is recognizable. Tests performing consisted in directing the camera to the face of person, holding the test screen. The test screen initially consisted of black and white squares of various size. The camera was moved away to get various face sizes in relation with the camera visual field.
In the first tests stage we have observed the relations between the recognizing of screen squares and face details.
Due to the above we have chosen the square of such size, which corresponds with dividing the face into 20 parts.
Fig.17.
Group of tests for human recognition.
Many hours of tests, performed in various distances, let to prepare the scale, where individual ranges correspond with the particular human face size. The upper range fields are filled with squares (upon the basis of chosen test screen), which should be identified with human face details. Black and white colours have been replaced with colour samples from various face examples from the adjacent picture, so the brightness and contrast differences between individual squares correspond to the differences, existing in the human face. The lower range field is not filled with squares, its colour changes fluently from dark to light shade. The test scale determines camera visual field percent, which is occupied by the human face. The practical test use is the following: if we can distinguish the upper field squares in the viewed picture, for the given range, then the human face of the particular size, corresponding to this range, will still be recognizable. On the adjacent photo there are exemplary human faces, which complement the test. The biggest face corresponds to 19% range of visual field, the child face to 13%, and the girl face (bigger one) to approx. 6%. The rectangles diagram above the photo shows the human face size for the given range in relation with camera visual field.
Observing the picture with the whole monitoring system we can not only talk about the suitability of the particular camera or lens for the identification purposes, but also check the whole system for that purpose, and in case of recording also analyse the influence of parameters and quality. For example, by increasing the compression level of recorded picture and decreasing the recording resolution we can tell, that the human face recognition abilities are significantly impaired. The face details (upper field squares) are not clearly visible. The picture distortions also appear, which result from the very compression algorithm. It makes recognition, if the squares in the particular range are still visible difficult. Then, reading the test data we must observe, to which moment there are no compression distortions in the lower, control ranges fields. If the compression distortion (lower fields) will be bigger than the squares, corresponding to the face details (upper fields), for such range (face size) we cannot talk about correct recognition.
The relations between the object size on the monitor screen and the system tasks, connected with that object, are defined by standards. The Polish standard, taking the European standard as the basis (PN-EN 50132-7) defines the object size on monitor screen in relation with operator tasks, e.g.: recognition, identification, detection or inspection.
The adopted assumptions say, that the object is the person, and the limit CCTV system resolution exceeds 400 TV lines. The standard recommends the following minimum object size, depending on requirements:
-
identification - the object should occupy minimum 120% of visual field height,
-
recognition - the object should occupy minimum 50% of visual field height,
-
intruder detection - the object should occupy minimum 10% of visual field height,
-
crowd control - the object should occupy minimum 5% of visual field height.
It is worth to mention, that the standard has been defined for the analogue systems.
According to the above standard, the particular system is suitable for human identification (face recognition), if the human body occupies 120% of camera visual field. Because the standard does not define the human height, we assumed 180 cm, and after including the body proportions, the head height (for 120% of the whole body) should be equal to 19% of screen height. It corresponds to our scale 19%, marked with green colour. For the identification very good is the system, which allows for recognizing the human face, when the head height is smaller than 19% of screen height (it corresponds to the blue scale range: 6 % - 16 %). When the size of human face, which is to be recognized, is located in the red scale part (above 22%), then, according to the standard, such system may not be used for the human identification.
The standard defines also, when we talk about recognition - motion control. When in the camera visual field we can see the human, whose height equals 5% of that field, we can say, that the tested system is suitable for activity recognition. The movement control test, showing the human body of discussed size, has been located next to the rectangles diagram.
The human body height (5%) and the given face size (rectangles diagram) may be transferred to the monitor screen (e.g. by drawing them on the film). It allows for comparing the human face / body size on the real picture from the camera, installed in its final location. It may also be the criterion of lens focal length selection, with the assumption of defined distance from the observed people.
Fig.18.
Exemplary views of test image.
On the left example we can distinguish the details (upper field squares) at 16% of scale - the used solution allows for face recognition according to the standard.
(e.g. for 13% no squares are visible, but smooth tonal transition).
The right example shows the high image compression influence - we cannot say about the test squares recognition (upper field) due to strong compression algorithm distortions - the square distortions are visible in the lower test fields (we call them control fields).
In the right example the compression is too strong for face identification.
Contents
Fig.19.
Character recognition test.
The purpose of this test is to assess the system ability for character recognition at the particular size. This ability may be suitable for systems recording image from the camera, directed to the object, whose characteristic features, as: description, symbol, number have the fundamental importance (are the main purpose) for the archiving process.
The characters sequence in the particular test line is not incidental. It contains the series of information, describing the font size features.
The characters sequence in the particular test line is not incidental. It contains the series of information, describing the font size features.
The characters sequence is divided into digits groups, describing some information, which are separated by random letters. The first two digits with comma define the font height in mm. The next group of two digits is the font size, expressed in points. The third group - three digits with comma - define the font line thickness, expressed in mm. The last group - three digits with comma - express the font line thickness, converted into points. The digits on the right are the lines numbers. Their size, as well as the "Characters" word size (in the bottom) is the same, as the biggest line.
This test allows, besides the camera and lens ability check, allows also for the archiving systems to assess the recording parameters (especially compression) influence on the recorded data legibility.
Fig.20.
Image from the character recognition test.
It is possible to unequivocally recognize the characters to line 5 maximum.
Contents
The cameras with CMOS sensors, in comparison with CCD cameras, show the characteristic picture noise (especially the cheapest plate cameras (CHIP type)). It is connected with lower sensitivity of these cameras type and the CMOS sensor characteristics (reading and analysis method of information from the sensor matrix).
The interesting thing about those cameras is that for the declared by the producers approximate resolution of 240 TVL, the observed resolution during the test exceeded 300 TVL. However, in case of CCD cameras, not always the cameras resolutions, declared by the producers, corresponded to the test results. Various cameras models were also characterized by various picture contrast levels. Very good result were achieved for the cameras, equipped with the CCD SONY sensor, their resolution corresponded to the declared value of 480 TVL. It is worth to notice, that the cameras resolution, given by the producers, indicates the horizontal resolution. The tested cameras have been produced in the Far East countries.
The exemplary tests readings in this article have been recorded using the computer and TV card software (picture capturing). It must be noticed, that the image observed on the computer screen has lower sharpness. It may seem, that it is slightly blurred, in comparison with the analogue CCTV monitor. That fact is probably connected with the signal conversion from analogue to digital form in the capturing card converter. Despite the fact, that the computer monitor technology and structure exceeds the TV monitors (e.g. in respect of pixel size), due to above fact the reading is slightly hindered.
In respect of sharpness the monochrome monitor was the best. In that case, however, too long focusing on the picture details more quickly causes the tiredness effect (large screen pixel size, refreshing frequency 50 Hz). The camera resolution was easiest to read from the monochrome monitor.
Comparing many image archiving systems, we often use the following terms: "more details", "better picture articulation", "lower sharpness", etc. As it is difficult to create the subjective scale of picture "articulation" or "sharpness", we were looking for objects, which are the best for such assessment. It must not be objects, which we see for the first time, like face, but which we see everyday, preferably many times. The alphanumerical signs are such objects. Therefore our test contains digits and letters, which are perfect for such comparisons (e.g. in case two different cameras picture dump to the hard disk). The image comparing (usually exported in JPG format) is the best for small, contrasting objects. The letters and digits are great for that purpose. It results from the compression algorithms of smooth tonal transitions and high-contrast small objects, used in image compression.
Contents
We wish You satisfaction from using our test pattern.
Image quality test
