Vision systems for robotic manufacturing applications have significantly evolved over the last decade. While the vision systems of old were unreliable, clunky and expensive, today’s systems are anything but. Proper vision systems can make the difference between an efficient robotic system and one that is not working optimally.
HERE ARE 5 MYTHS AND TRUTHS ABOUT VISION SYSTEMS.
Credits : freepik.com
MYTH #1: VISION SYSTEM ARE COMPLICATED
In actuality, modern vision systems are very simple to install and use. Most of the algorithms and communications are built in, so it can be very easy and quick to make adjustments without the help of a trained engineer. New users are often surprised just how easy it is to use and maintain their vision systems.
MYTH #2: VISION SYSTEMS ARE NOT RELIABLE
If a vision system is properly applied, it will be highly robust, repeatable and reliable. Today’s vision system components are very robust, even in harsh environments. They are built to operate in rugged applications. Unlike a human, a vision system will see accurately every time. It never gets tired, takes a break or goes home for the evening.
MYTH #3: ALL VISION SYSTEMS ARE THE SAME
There is no truly out-of-the-box solution for vision systems. Each application is unique, and many factors of need to be considered. Anyone who tells you there’s a plug-and-play option for your operations is not selling you a solution that’s properly engineered for your needs. Customized vision systems are the only ones that will work efficiently and reliably.
MYTH #4: VISION SYSTEMS ARE ALWAYS THE BEST SOLUTION
While vision systems are helpful in many robotic applications, there are some jobs in which vision may not be the answer. For example, operations that have drastic changes from part to part moving quickly on a single line may not benefit from a vision system because more discriminating inspection may be necessary. In addition, a vision system helps provide tight tolerances, so applications with loose tolerances may be just fine with sensors and not need to be upgraded to a vision system.
MYTH #5: VISION SYSTEMS ARE TOO EXPENSIVE
Just 10 years ago, typical vision systems cost an average of $30,000. Today, that same system may cost only $5,000 to $15,000. The evolution of vision technologies have brought down the cost considerably. In fact, many companies can see an ROI relatively quickly because a vision system requires fewer special fixtures and conveyors, decreases downtime for fixture changeout, and increases operations overall.
An efficient manufacturer must get products in and out of a cell quickly and reliably. Vision systems paired with robotic operations can put an operation at a competitive advantage by providing opportunities to make more and streamline the process for optimum profitablilty.
MACHINE VISION TECHNOLOGY has found its way into applications inside and outside of factory settings, riding a wave of progress in automation technology and growing into a sizable global industry. Quite a bit of future technology will depend on machine vision, and the market will grow accordingly.
In 2017, according to a recent report, the global machine vision market was valued at $7.91 billion1. By 2023, the global market is expected to reach $12.29 billion – a compound annual growth rate (CAGR) of 7.61%. This robust growth is caused by a number of broader economic factors.
WHAT’S DRIVING LONG-TERM GROWTH IN MACHINE VISION?
The main drivers of growth in the machine vision market are the need for quality inspection and automation inside factories, growing demand for AI and IoT integrated systems that depend on machine vision, increasing adoption of Industrial 4.0 technology that uses vision to improve the productivity of robotic automation, and government initiatives to support smart factories across the globe.
Machine vision software will be one of the fastest growing segments between 2017 and 2023. The main reason for this is the expected increase in integration of AI into industrial machine vision software to enable deep learning in robotics technology.
PC-based INDUSTRIAL MACHINE VISION PRODUCTS, the oldest form of industrial machine vision, will retain a large portion of machine vision market share because of their ease of use and processing power.
Credit : freepik.com
WHAT TRENDS ARE WORTH WATCHING NOW?
While there are several main factors in the expected long-term growth of the global machine vision market, there are a few trends to keep an eye on now that are changing the way machine vision technology is deployed.
Industrial Internet of Things (IIoT): while AI and IoT technology are long-term drivers of growth, the IIoT is connecting production technology with information technology in today’s factories to increase productivity. The IIoT depends on heavily on machine vision to collect the information it needs.
Non-Industrial Applications: driverless cars, autonomous farm equipment, drone applications, intelligent traffic systems, guided surgery and other non-industrial uses of machine vision are rapidly growing in popularity, and often call for different functionality in machine vision than industrial applications. These non-industrial uses of machine vision are being deployed today and could be an important part of machine vision growth.
Ease of Use: Machine vision systems can often be complex from the user’s perspective. As mentioned above, PC-based machine vision systems will remain popular, despite their age, because of their ease of use. The desire for ease of use may drive further standardization in machine vision products, which could make them even easier to deploy inside and outside of factory settings.
The machine vision market is poised for long-term growth. The IIoT, growing non-industrial applications and ease of use are all helping buoy today’s machine vision market, but there are several other factors effecting long-term market expansion.
Many applications demand more from a lens than the performance delivered by STANDARD IMAGING LENSES, which introduces the need for ruggedized imaging lenses. A new imaging choice is the Stability Ruggedized imaging lens.
JESSICA GEHLHAR, VISION SOLUTIONS ENGINEER, AND CORY BOONE, OPTICAL ENGINEER, EDMUND OPTICS
Historically, Industrial and Ingress Protection Ruggedized imaging lenses have addressed many environmental and application challenges. But as imaging systems increase the number of moving elements and as products move through inspection systems faster for higher throughput, these movements require greater calibration and image performance. As applications like factory automation, measurement, robotics and autonomous vehicles continue to expand and develop, the need for Stability Ruggedized imaging lenses will increase along with the progression of the industry.
Each of these applications presents environmental challenges–such as shock, vibration, and contamination–to imaging systems. Unlike lab and observatory setups, which tend to have relatively controlled environments, manufacturing facilities can be rife with environmental operating difficulties.
To address these challenges, ruggedized imaging lenses have a number of features and benefits. But to determine the best ruggedized lens for an application, first let’s clearly define the various ruggedization techniques.
A STANDARD IMAGING LENS can be insufficient in some applications and environments due to the large number of moving parts within the lens assembly, such as the doubled threaded focus adjustment, the multi-leaf iris diaphragms, and their corresponding thumb screws. For example, the thin overlapped iris leaves are especially susceptible to high shock and vibration, which can cause them to easily spring out of place and be damaged. By replacing the iris leaves with a fixed aperture, the survivability of the lens can be greatly improved.
Another component of the lens that may come loose during shock and vibration are the thumbscrews. Although they may not completely fall off, they can be loosened enough that the focus changes, potentially degrading the image quality. On a machine vision inspection system, faulty image quality can increase the potential to reject passing units or pass failing units. Debris and contaminants in the area can compound these effects.
Historically, there have been two primary ruggedization techniques to address these environmental difficulties – Industrial Ruggedization and Ingress Protection Ruggedization.
IN AN INDUSTRIAL RUGGEDIZED IMAGING LENS, MANY OF THE MOVING PARTS OF A STANDARD IMAGING LENS ARE ELIMINATED:
the multi-leaf iris is replaced with a fixed aperture stop
the focus adjustment is replaced with a simple single thread
the thumb screws are replaced with set screws.
Industrial Ruggedization prevents many of the unintentional movements and focus shifts described above, and therefore maintains ideal image quality. Industrial Ruggedized imaging lenses can also prevent a user from accidentally changing the focus and iris settings.
In an Ingress Protection Ruggedized imaging lens, the lens assembly is either fully enclosed or sealed with O-rings (or RTV silicone) to withstand environmental contaminants. IP66 or IP67 environmental ratings are the most well known standards for particulate and water resistance.
Enclosures and seals can be especially critical for lenses used to inspect food quality. The lenses must withstand direct exposure to liquids and humidity in such wash down applications.
Many manufacturing, processing, and packaging applications are in unfavorable environments where dust, debris, dirt, adhesives or fluids are commonplace. Ingress Protection Ruggedized imaging lenses are designed to withstand these harsh environments.
Some of today’s more demanding applications in factory automation, measurement, robotics, and autonomous vehicles levy additional requirements on imaging systems beyond those of Industrial Ruggedization and Ingress Protection Ruggedization. In such situations, there’s another type of ruggedization: Stability Ruggedization. The EDMUND OPTICS TECHSPEC COMPACT RUGGEDIZED (CR) SERIES FIXED FOCAL LENGTH LENSES are an example of Stability Ruggedized imaging lenses.
While Ingress Protection Ruggedization prevents contamination and Industrial Ruggedization eliminates moving parts, Stability Ruggedization maintains (or stabilizes) optical pointing and positioning even after heavy shock, vibration and temperature change. In a Stability Ruggedized imaging lens, the individual lens elements are glued into place to prevent them from moving within the housing.
In an optical system, lens elements sit within the inner bore of the barrel. The space between the outer diameter of the lens and inner diameter of the barrel is typically less than 50 microns; decenters of even tens of microns are enough to significantly affect the pointing of the lens.
When using a Stability Ruggedized lens, if an object point falls on the exact center pixel, it will always fall there even if the lens has been heavily vibrated; therefore pixel shift is reduced and the image is stabilized.
Even in clean inspection environments with well-controlled robotic movements, there are challenges. Conveyer lines and robotic systems move at higher speeds and handle heavier products than ever before. Vibration comes from operation speeds and weight of the objects, or by the lines and systems located next to equipment. These high intensity environments create tough shock and vibration requirements for imaging systems, making the lens performance more critical.
Additionally, users have higher resolution and image quality expectations. As camera pixel sizes become smaller, even slight misalignments in imaging systems become apparent. Pointing and alignment changes that were once unnoticeable with a camera having 5.6 micron pixels may be very obvious with a camera having 1.4 micron pixels either over time or after a strong shock.
Stability Ruggedization is important in applications where the field of view must be calibrated, such as measurement equipment, 3D stereo vision, lenses used for robotic sensing, and lenses used for tracking object locations. These applications often require the pointing to be stabilized to values much smaller than a single pixel.
In 3D stereo vision, two imaging lenses are used to image a pattern that’s been projected onto a 3D object. The two images are then compared to extract the 3D information about the object, but to do this the angle of the two lenses and field of view must be well calibrated.
Once calibrated, any shift in the pixel mapping will offset the information in the 3D model, affecting the measurements. Many times these systems experience heavy shock and temperature shifts during shipping or relocation. If the system were to require recalibration for each relocation, it would be costly to send a technician onsite, whereas setting up and calibrating the system once would keep costs down.
Another similar application is distortion calibration. Information is not lost in an imaging system that has distortion – the information is simply moved. Distortion can be mapped or calibrated from your imaging systems to remove it. If pixel shift occurs, the distortion mapped is now incorrect; shifting the distortion calibration map can move your values affecting your accuracy.
Another recent challenge machine vision imaging systems face is an increase in traditional robotic imaging systems crossing over into autonomous systems, along with many embedded vision solutions beyond the relatively stationary conveyer line and robotic setups.
Vision enabled robots rely on imaging systems to know where they are in space. As robots move to perform their task, the constant movement can cause pixel shift, affecting their ability to know where they are and their accuracy. Machine vision is also expanding beyond the final pack-out robots on a line. After products are packaged and boxed up, vision-guided robots can transport and load them on and off of trucks – even the trucks themselves may be autonomous vehicles, guided by many sensors including vision.
Back in the factory and warehouses, there are more autonomous robots, driving or flying around to move products or inspect storage locations. Large distribution centers often carry a variety of goods, inspection and handling systems to handle a variety of weights, sizes, and packaging materials. As industrial machine vision systems make their way outside of typical applications and onto autonomous vehicles, once ‘controlled’ environments, no matter how challenging, are no longer controlled in many ways.
The future depends on monitoring and regulating air pollution, which is an essential step towards creating a cleaner environment.
MONITORING MARITIME POLLUTION WITH OPTICS
Monitoring and regulating air pollution is an essential step towards creating a cleaner environment. Danfoss IXA, a high-tech company based in Denmark, is developing a device called MES 1001, a marine emission sensor based on ultraviolet absorption spectroscopy which monitors the NO, NO2, SO2 and NH3 emissions produced by cargo ships to ensure that they are complying with all environmental regulations. The optical sensor is placed inside the exhaust system of ships, so the involved optics will be exposed to extreme conditions and must be able to withstand temperatures up to 500°C and very high pressures simultaneously.
Danfoss IXA was looking for a partner to develop optics fulfilling their demanding requirements, and in EDMUND OPTICS (EO) they found a partner who was prepared to take on this challenge which went beyond their normal capabilities. EO created custom test beds for verifying the unique requirements of the sensor, which enabled EO to develop a robust system to meet Danfoss IXA’s specifications.
Danfoss IXA develops sensors and systems for the maritime industry, focusing on energy optimization and the measurement of emission gases. They are a part of the Danfoss Group, a global enterprise which produces a wide range of technologies that address a variety of markets including food supply, energy efficiency, and climate-friendly solutions.
Smokestack emissions from international shipping are a severe problem for human health, contributing to the premature mortality of people all across the world from lung damage and cardio vascular diseases.
CREATING A CLEANER ENVIRONMENT STARTS AT SEA
The International Maritime Organization (IMO) has recently decided that commercial ships must comply with low sulfur fuel requirements globally by 2020. In addition, the current Nitrogen Oxide emission control area along the North American coastline will be expanded to cover the Baltic and North Seas in 2021. There currently aren’t convenient and reliable ways for the IMO to monitor ships’ emissions and enforce these regulations. A multitude of local and regional initiatives seeking to limit the air emission from ships further underline the fact that the industry needs to adapt to a world where strict emission requirements are part of the game. Danfoss IXA is developing the MES1001, which is a comprehensive marine emissions sensor suitable for accurately measuring a ship’s air emissions in real time.
Danfoss IXA approached several providers of optical components to jointly design the optical system for the new MES 1001 device. This project turned out to be very challenging due to the extreme high temperature and pressure requirements. High temperatures can cause optics to fail due to melting and thermal stresses, which severely limits the types of optical materials that can be used. High temperatures can also cause adhesives used in the optical assembly to outgas, contaminating the system. The high pressure requirements made the sealing of the optical system critically important. Most of the optics partners faced their limits in terms of design, metrology for these harsh conditions, or working across different continents and time zones.
EDMUND OPTICS (EO), with its global presence and large staff of optical engineers and designers, is always keen to face new challenges. One of the reasons that Danfoss IXA selected EO as a partner is their ability to ramp-up products from prototype to volume production. When approached by Danfoss, EO dedicated R&D and project management resources to developing an optical assembly for the MES 1001, even though EO had never designed systems to work at temperatures as high as 500°C before. EO investigated many different materials and mounting options, recognizing this project as a learning experience and opportunity to expand their capabilities. Custom testbeds for verifying the optical system’s unique requirements were created and proper sealants and optomechanics were identified to allow the assembly to survive these high pressures. The start of that development process was faced with many issues including cracking optics and outgassing adhesives, but by iterating the design process multiple times and researching in different materials these issues were solved and Edmund Optics eventually delivered an optical assembly that could survive the harsh environment inside a ship’s exhaust system. Edmund Optics is proud to be a part of this product which will positively impact the environment and support a global effort to reduce harmful emissions.
Danfoss IXA “greatly appreciated EO’s professional way of involving [them] along the development process as well as their ability to adapt to changing requirements as [Danfoss IXA] learned more about the exact conditions in which the sensor would be used.” During that time Danfoss IXA “found the support from EO’s project managers extremely fruitful and very efficient in bringing the development process to success.”
The robust optical system is a critical component of the new MES 1001 device, which was launched in 2017. It was exciting for EO to work on this cutting-edge technology in such a close collaboration with Danfoss IXA’s skilled research and development team. The MES 1001 will allow the IMO and other organizations to enforce maritime emissions requirements and help lead to a cleaner environment across the globe.
The speed of line scan cameras has greatly increased in the last years. MODERN LINE SCAN CAMERAS operate with integration times in the range of 15 µs. In order to achieve excellent image quality, in some cases illuminance levels of over 1 million Lux are required. One of the most important criteria for assessing image quality is noise disturbance (white noise). There are various noise sources in image processing systems and the most dominant one is called “shot noise”.
Shot noise has a physical cause and this has nothing to do with the quality of the camera. The noise is caused by the special essence of light, by photons. The image quality depends on the number of photons which hit the object and ultimately on the number of photons which reach the camera sensor.
In a set-up with a defined signal transmission there are three parameters which influence the 'shot noise' when capturing an image:
integration time (scanning speed)
aperture (depth of focus and maximum definition)
amount of light on the scanned object
The choice of lens aperture greatly determines the required light intensity. If, for instance, the aperture is changed from 4 to 5.6, twice the amount of light is required in order to maintain the same signal to noise ratio (SNR) – see fig. 01). By using a greater aperture, more depth of focus is achieved and the image quality is improved due to reduced vignetting effects with the majority of lenses.
LIGHT FOR ALL APPLICATIONS CURRENTLY
LEDs are available in various shades of color. You can get them in red, green, blue, yellow or amber. Even UV LEDs and IR LEDs are obtainable. The choice of a specific color and thus a specific wave length can determine how object properties on surfaces with diverse spectral response are made visible.
In the past, red light was often used wherever high intensity was required. However, relevant performance increase in LED technology today usually occurs with white LEDs. These high-performance LEDs are used for example in car headlights and street lamps. The core of a white LED actually consists of a blue LED. Using fluorescent substances, part of the light from the blue LED is converted into other visible spectral ranges in order to produce a 'white' light.
UV-LEDs are frequently used to make fluorescent effects visible. In many cases a wavelength of approx. 400nm is sufficient. UV-LEDs with shorter wavelengths may be suitable for hardening paint, adhesives or varnishes. In comparison to blue or white LEDs, UV-LEDs are less efficient. By focusing through a reflector however this can be improved. IR lighting is implemented for food inspection. Wavelengths of 850nm or 940nm are used. When sorting recyclable material, wavelengths from 1,200nm to 1,700nm are used to identify different types. Here however, IR-LEDs in this range are not as adequate as classic halogen lamps with appropriate filters where beam output is concerned.
The small design enables a very short warm-up phase. This circumstance presupposes good thermal dissipation, in order to maintain appropriate working conditions, i.e. temperatures. As a rule: the better the cooling, the longer the LED durability. Apart from durability, LED temperatures also influence spectral behavior (possible color shifting) and general output (luminance).
In systems where precise color reproduction is required, it is recommended to keep the lighting’s temperature steady at a predetermined value. At present, efficient control systems can regulate the LED temperature to within a spectrum range of less than 2°C.
Modern lighting systems, such as the Corona II lighting system developed by Chromasens, provide numerous cooling options. This includes passive cooling with thermal dissipation via convection, compressed air cooling, water cooling and ventilation. Active ventilation, compressed air or water cooling are good cooling methods for measuring applications situated in surroundings with high temperatures. By monitoring the temperature of the LEDs and regulating the cooling system, shifts in color reproduction can be completely avoided or at least greatly reduced.
FOCUS ON THE ESSENTIAL
If a flat object at a known and fixed distance is to be illuminated, selecting the adequate focus is relatively simple. Selecting the right lighting is more complicated, if the object is not at a predetermined distance from the light or has no flat surface. In such a case, assuring a permanently sufficient image brightness is a challenge. Here the use of reflector technology facilitates the accumulation of light from a LED (greater coverage angle of the reflected light) and a better light distribution from the depth.
In contrast to background or bright field lighting, focused lighting is normally used for top lighting. Customary lighting systems use rod lenses or Fresnel lenses in order to achieve the necessary lighting intensity. CHROMASENS adopts a novel and completely unique approach. While the use of rod lenses causes color deviations due to refraction, the mirror (reflector) principle developed and patented by Chromasens has no such trouble.
Shiny or reflective materials are a challenge for lighting. Unwished for reflections often appear in the image. In combination with a polarizing filter rotated 90 degrees in front of the camera, these unwanted light reflections can be prevented. When using such filters, certain factors have to be considered. The temperature stability of the filter is one point. In this respect, many polarizing filters can only be used to a certain extent. Another criterion is effectiveness: with such settings, only about 18-20 % of the original amount of light reaches the sensor. The amount of light provided by the lighting must therefore be great enough to minimize noise and yet achieve sufficiently good image quality.
The lense aperture and the light amount significantly influence the signal noise ratio
LED systems offer definite advantages compared to traditional lighting technologies such as halogen or fluorescent lamps + Good cooling ensures long durability, consistent spectral behavior and a high level of brightness
The use of reflectors assures optimal lighting, even from different distances
Color LEDs, UV- and IR-LEDs are extremely versatile
Polarizing filters prevent unwanted light reflection on shiny surfaces. The amount of light provided by the lighting must still be sufficient
Choosing the right illumination for the application is critical for acquiring the high quality images needed for calculating 3D data. We compare the imaging results of a directional coaxial brightfield illumination with a Corona tube light in terms of color image quality and height map for different samples. It could be shown that for material that exhibit considerable amounts of subsurface scattering, coaxial lighting geometry benefits the 3D measurement using 3DPIXA.In practice, it has to be kept in mind that introducing the beam splitter in the light path results in a shift of the working distance of the camera system, and a slight reduction of image quality.
An illumination scheme where the source rays are reflected from a flat sample directly into the camera is called a brightfield. With line scan cameras there are two possible ways to realize such a setup: either by tilting the camera and light source such that the angle with respect to the surface normal is the same but opposite direction, or by using a beam splitter. The first method is not recommended as it can lead to occlusion and keystone effects. Thus we want to discuss the brightfield setup using a beam splitter.
Figure 1 shows the principle of this setup in comparison to a setup with a tubelight. The tubelight is the superior illumination choice for a wide array of possible applications. It reduces the intensity of specular reflections and evenly illuminates curved glossy materials. Most of the time the tubelight should be your first choice and only some materials require the use of a coaxial brightfield illumination.
An example as such is material that exhibits strong subsurface scattering, which means that light beams partially penetrate a material in a certain direction, are scattered multiple times, and then exit at a different location with possibly different direction. Resulting from that is a material appearance that is translucent. Examples of such materials are marble, skin, wax or some plastics.
Using tube light on such materials results in a very homogeneous appearance with little texture, which is problematic for 3D reconstruction. Using coaxial brightfield illumination results in relatively more direct reflection from the surface to the camera, as compared to a tube light illumination. This first surface reflection contributes to the image texture; the relative amount of sub-surface scattered light entering the camera is thereby reduced.
There are some specific properties that have to be taken into consideration when using a coaxial setup with a 3DPIXA. Firstly, only a maximum of 25% of the source intensity can reach the camera as the rest is directed elsewhere in the two transits of the beam splitter. Secondly, the glass is an active optical element that influences the imaging and 3D calculation quality. In chapter 3 we have a closer look at these factors and offer some guidelines for mechanical system design to account for resulting effects. Prior to that, we discuss the effects of the brightfield illumination on a selection of a few samples to give an idea when this type of illumination setup should be used.
2.COMPARING BRIGHTFIELD AND TUBELIGHT ILLUMINATION
In this chapter we want to give you some impressions of the differences between using a coaxial illumination in comparison to a tubelight using different samples. As a tubelight we used the CHROMASENS CORONA II Tube light (CP000200-xxxT) and for the brightfield we used a CORONA II Top light (CP000200-xxxB) with diffusor glass together with a beam splitter made from 1.1 mm “borofloat” glass.
In figure 2 we show a scanned image of a candle made of paraffin, which is a material that exhibits strong subsurface scattering. With coaxial illumination (right image) the surface texture is clearly visible and the height image shows the slightly curved shape of the candle. In comparison the tube light (left image) contains very low texture and height information could not be recovered for most of the heights (black false colored region). The texture is only visible with coaxial illumination because under this condition the light reflected from the surface is more dominant in the final image than the subsurface scattered light. However, the ratio between these two effects varies with different surface inclinations. The more deviated the surface normal is from the camera observation angle, the less direct light is reflected directly from the first surface. Therefore, texture in the image gets lower. For the candle sample, more than 15° deviation resulted in failure in recovering height information. This can be seen in the right image at the outer edges of the candle.
3Fehler! Verweisquelle konnte nicht gefunden werden.. The substrate area in the tube light image (left) shows low texture, resulting in partially low performance height reconstruction (black points in false-colored image overlay). With coaxial illumination (right image), the amount of source rays reflected back into the camera from the surface of the material is larger than the subsurface scattered light. The image texture is higher and height reconstruction performance improves.
However, if the height of the balls is the focus in the application rather than inspecting the substrate, the situation becomes more complex as the coaxial illumination results in specular reflection on the ball tops. If these areas are saturated, it negatively affects height measurements as well.
The best illumination therefore strongly depends on the measurement task and materials used and can often only be determined by testing. If you are unclear which light source is best for your application, please feel free to contact our sales personnel to discuss options and potentially arrange for initial testing with your samples at our lab.
The beam splitter essentially is a plan parallel glass plate which offsets each ray passing through without changing its direction. The size of this offset depends on the incidence angle, the thickness of the glass and its refractive index. The thickness of the beam splitter should therefore be only as small as is needed for stability reasons. In the following analysis we assume a thickness of the beam splitter of d=1.1 mm “borofloat” glass.
The result of the beam splitters influence is a movement of the point from where the sharpest image can be acquired in all three spatial coordinates. The change along the sensor direction (called x-direction) leads to a magnification change of the imaging system that is negligible small (<0.4%, with a small dependence on camera type).
The change along the scan direction (called y-direction) only offsets the starting point of the image. If the exact location of the scanline is important (e.g. when looking on a roll) the camera needs to be displaced relative to the intended scan line by
Δy = d*(0.30n – 0.12).
The equation is valid for all glass thicknesses d and is a linear approximation of the real dependency on n, where n is the refractive index of the glass material introduced into the light path. The approximation is valid in the interval of n= [1.4, 1.7] and for all types of 3DPIXAs. The direction of the displacement is towards the end of the beam splitter that is nearer to the sample, so in the scheme in figure 1 the camera has to be moved to the left.
The change of the working distance is different along the x- and y-axis of the system because of the 45° tilt of the beam splitter leading to astigmatism. In y-direction the working distance is increased by
zy = +d*(0.24n +0.23).
As above, the formula is valid for all d and n= [1.4, 1.7]. The change of the working direction along the x-direction is not constant but also changes depending on the position of the imaged point which leads to field curvature. Both astigmatism and field curvature slightly lower your image quality which influences the imaging of structures near the resolution limit. But they should not influence the 3D algorithm as generally only height structures that are several pixels in size can be computed.
Additionally to the optical effects discussed above the beam splitter also changes the absolute height values computed by the 3D algorithm (i. e. the absolute distance to the camera). The exact value of this height change is slightly different for each camera. Generally the measured distance between camera and sample decreases, so that structures appear nearer to the camera than they really are. This change is constant over the whole height range (simulations show 0.2% change) and also constant over the whole Field of View. In summary, relative height measurements are not influenced at all, and absolute measurements are shifted by a constant offset.
As the precise change of the calculated height is not known, the zero plane of the height map can’t be used to adjust the camera to the correct working distance. We advise you instead to set up your camera using the free working distance given in the data sheet and correcting it with Δzy from above.
On certain translucent materials (those exhibiting considerable subsurface scattering of light), using coaxial illumination can result in a significant increase in image texture which greatly benefits the 3D height reconstruction. However, the additional glass of the beam splitter in the optical path of the camera system when using coaxial illumination influences the optical quality negatively. Further, the working distance of the system changes slightly and the absolute measured distances are set off by a constant value. This does not affect relative measurements, which are generally recommended with the 3DPIXA.
WHAT DO IMAGE PROCESSING SYSTEMS HAVE TO DO WITH KEEPING FOODSTUFFS IN GOOD SHAPE?
Everyone prefers foodstuffs that are fresh and outwardly attractive. Image processing systems are frequently used during the quality assurance process for these products to ensure that this is the case. The image data helps producers make informed decisions that would be otherwise be impossible.
But how are systems of this kind designed? What steps are necessary, what must be taken into account, and what options are available?
Selection of the camera, selection of the lens and lighting source, evaluation of image quality, selection of PC hardware and software and the configuration of all components – all of those are important steps toward an effective image processing system.
Imagine an apple grower asks you to design a machine vision system for inspecting the apples. He’s interested in delivering uniform quality, meaning the ability to sort out bad apples while still working fast. He is faced with the following questions:
What are the precise defined requirements for the system?
Which resolution and sensors do I need?
Do I want to use a color or monochrome camera?
What camera functions do I need, and what level of image quality is sufficient?
WHAT EXACTLY SHOULD THE SYSTEM DELIVER AND UNDER WHICH CONDITIONS?
This question sounds so obvious that it's frequently overlooked and not answered in the proper detail. But the fact remains: If you are clear up front about precisely what you want, you'll save time and money later.
SHOULD YOUR SYSTEM
Only show images of the object being inspected, with tools like magnification or special lighting used to reveal product characteristics that cannot be detected with the human eye?
Calculate objective product features such as size and dimensional stability?
Check correct positioning — such as on a pick-and-place system?
Determine properties that are then used to assign the product into a specific product class?
RESOLUTION AND SENSOR
Which camera is used for any given application? The requirements definition is used to derive target specifications for the resolution and sensor size on the camera.
But first: What exactly is resolution? In classic photography, resolution refers to the minimum distance between two real points or lines in an image such that they can be perceived as distinct.
In the realm of digital cameras, terms like "2 megapixel resolution“ are often used. This refers to something entirely different, namely the total count of pixels on the sensor, but not strictly speaking its resolution. The proper resolution can only be determined once the overall package of camera, lens and geometry, i.e. the distances required by the setup, is in place. Not that the pixel count number is irrelevant — a high number of pixels is truly needed to achieve high resolutions. In essence, the pixel count indicates the maximal resolution under optimal conditions.
Fine resolution or large inspection area — either of these requirements necessitates the greatest possible number of pixels for the camera. Multiple cameras may actually be required to inspect a large area at a high level of resolution. In fact, the use of multiple cameras with standard lenses is often cheaper than using one single camera with a pricy special lens capable of covering the entire area.
The sensor size and field of view dictate the depiction scale, which will later be crucial for the selection of the lens.
COLOR OR MONOCHROME?
Generally speaking, most applications do not really need a color camera. Color images are often just easier on the eyes for many observers. Realistic reproduction of color using a color camera necessitates the use of white lighting as well. If the characteristics can be detected via their color (such as red blemishes on an apple), then color is often — but not always — needed. Yet these characteristics can also in many cases be picked up in black and white images from a monochrome camera if colored lighting is used. Experiments on perfect samples can help here. If color isn’t relevant, than monochrome cameras are preferable, since color cameras are inherently less sensitive than black and white cameras.
Are you working with a highly complex inspection task? If so, you may want to consider using multiple cameras, especially if a range of different characteristics need to be recorded, each requiring a different lighting or optics configuration.
WHAT A CAMERA SHOULD ALSO PROVIDE: CAMERA FUNCTIONS AND IMAGE QUALITY
There’s more to a good camera than just the number of pixels. You should also take image quality and camera functions into account.
When evaluating the image quality of a digital camera, the resolution is one important factor alongside:
In terms of camera functions, one of the most important is the speed, typically stated in frames per second (fps). It defines the maximum number of frames that can be recorded per second.
THE EYE OF THE CAMERA: SCALE AND LENS PERFORMANCE
Good optical systems are expensive. In many cases, a standard lens is powerful enough to handle the task. To decide what’s needed, we need information about parameters such as
Image scale, meaning the ratio between image and object size. This corresponds to the ratio of the size of the individual pixels divided by the pixel resolution (The pixel resolution is the length of the edges of a square within the object being inspected that should fill up precisely one pixel of the camera sensor.
Focal length of the lens that determines the image scale and the distance between camera and object
Once this information is available, it becomes much easier to examine the spec sheets from lens makers to review whether an affordable standard lens is sufficient or whether a foray into the higher-end lenses is needed.
Lens properties like distortion, resolution (described using the MTF curve), chromatic aberration and the spectral range for which a lens has been optimized, serve as additional selection criteria.
There are for example special lenses for near infrared, extreme wide angle lenses ('fisheye‘) and telecentric lenses that are specially suited for length measurements. These lenses typically come at a high price, though.
Here too the rule is: Tests and sample shots are the best way to clear up open questions.
It’s hard to see anything in poor light: It may seem obvious, but it holds true for image processing systems as well.
High inspection speeds typically require sensitive camera and powerful lenses. In many cases however the easier option is to modify or improve the lighting situation to boost the image brightness. There are a variety of options for attaining greater image brightness: Increasing the ambient light and sculpting the light using lenses or flashes to create a suitable light source are two examples. But it’s not just the lighting strength that’s important. The path that the light moves through the lens to the camera matters too.
One common example from photography is the use of a flash: if the ambient lighting is too diffuse, then a flash is used to aim the light in a targeted manner — although then you need to deal with unwanted reflections off smooth surfaces in the image area that can overwhelm the desired details. During image processing, these kinds of effects may actually be desired to deliver high light intensities on straight, low-reflecting surfaces. For objects with many surfaces reflecting in various direction, diffuse light is better.
We look at photos by reflecting light on them, while a stained glass window only reveals its beauty when the light shines through it.
Which hardware is required depends on the task and the necessary processing speed.
While simple tasks can be handled using standard PCs and image processing packages, complex and rapid image processing tasks may require specialized hardware.
Software is required to assess the images. Most cameras come together with software to display images and configure the camera. That’s enough to get the camera up and running. Special applications and image processing tasks require special software, either purchased or custom developed.