As the plant floor has become more digitally connected, the relationship between robots and machine vision has merged into a single, seamless platform, setting the stage for a new generation of more responsive vision-driven robotic systems. BitFlow, Inc., a global innovator in frame grabbers used in industrial imaging, predicts vision-guided robots will be one of the most disruptive forces in all areas of manufacturing over the next decade.
"Since the 1960s robots have contributed to automation processes, yet they've done so largely blind," said Donal Waide, Director of Sales for BitFlow, Inc. "Vision-equiped robots are different. Now, just like a human worker, robots can see a specific part to validate whether it is being placed correctly in a pick and place application, for example. Cost savings will be realized since less hard fixturing is required and the robot is more flexible in its ability to locate a variety of different parts with the same hardware."
HOW ROBOTIC VISION WORKS
Using a combination of camera, cables, frame grabber and software, a vision system will identify a part, its orientation and its relationship to the robot. Next, this data is fed to the robot and motion begins, such as pick and place, assembly, screw driving or welding tasks. The vision system will also capture information that would be otherwise very difficult to obtain, including small cosmetic details that let the robot know whether or not the part is acceptable. Error-proofing reduces expensive quality issues with products. Self-maintenance is another benefit. In the event that alignment of a tool is off because of damage or wear, vision can compensate by performing machine offset adjustment checks on a periodic basis while the robot is running.
DUAL MARKET GROWTH
In should come as no surprise that the machine vision and robotic markets are moving in tandem. According to the Association for Advancing Automation (A3), robot sales in North America last year surpassed all previous records. Customers purchased 34,904 total units, representing $1.896 billion in total sales. Meanwhile total machine vision transactions in North America increased 14.8%, to $2.262 billion. The automotive industry accounts for appoximately 50% of total sales.
THE ROLE OF FRAME GRABBERS
Innovations in how vision-guided robots perceive and respond to their environments are exactly what manufacturers are looking for as they develop automation systems to improve quality, productivity and cost efficiencies. These types of advancements rely on frame grabbers being paired with high-resolution cameras to digitize analog video, thus converting the data to a form that can be processed by software.
BitFlow has responded to the demands of the robotics industry by introducing frame grabbers based on the CoaXPress (CXP) machine vision standard, currently the fastest and most powerful interface on the market. In robotics applications, the five to seven meters restriction of a USB cable connection is insufficient. BitFlow CXP frame grabbers allow up to 100 meters between the frame grabber and the camera, without any loss in quality. To minimize cabling costs and complexity, BitFlow frame grabbers require only a single piece of coax to transmit high-speed data, as well as to supply power and send control signals.
BitFlow's latest model, the Aon-CXP frame grabber, is engineered for simplified integration into a robotics system. Although small, the Aon-CXP receives 6.25 Gb/S worth of data over its single link, almost twice the real world data rate of the USB3 Vision standard and significantly quicker than the latest GigE Vision data rates. The Aon-CXP is designed for use with a new series of single-link CXP cameras that are smaller, less expensive and cooler running than previous models, making them ideal for robotics.
Machine vision is the incorporation of computer vision into industrial manufacturing processes, although it does differ substantially from computer vision. In general, computer vision revolves around image processing. Machine vision, on the other hand, uses digital input and output to manipulate mechanical components. Devices that depend on machine vision are often found at work in product inspection, where they often use digital cameras or other forms of automated vision to perform tasks traditionally performed by a human operator. However, the way machine vision systems ‘see’ is quite different from human vision.
THE COMPONENTS OF A MACHINE VISION SYSTEM CAN VARY, BUT THERE ARE SEVERAL COMMON FACTORS FOUND IN MOST. THESE ELEMENTS INCLUDE:
- Digital or analog cameras for acquiring images
- A means of digitizing images, such as a camera interface
- A processor
Credits : freepik.com
When these three components are combined into one device, it’s known as a smart camera. A machine vision system can consist of a smart camera with the following add-ons:
- Input and output hardware
- Light sources, such as LED illuminators or halogen lamps
- An image processing program
- A sensor to detect and trigger image acquisition
- Actuators to sort defective parts
HOW MACHINE VISION SYSTEMS WORK
Although each of these components serves its own individual function and can be found in many other systems, when working together they each have a distinct role in a machine vision system.
To understand how a machine vision system works, it may be helpful to envision it performing a typical function, such as product inspection. First, the sensor detects if a product is present. If there is indeed a product passing by the sensor, the sensor will trigger a camera to capture the image, and a light source to highlight key features. Next, a digitizing device called a framegrabber takes the camera’s image and translates it into digital output, which is then stored in computer memory so it can be manipulated and processed by software.
In order to process an image, computer software must perform several tasks. First, the image is reduced in gradation to a simple black and white format. Next, the image is analyzed by system software to identify defects and proper components based on predetermined criteria. After the image has been analyzed, the product will either pass or fail inspection based on the machine vision system’s findings.
Beyond product inspection, machine vision systems have numerous other applications. Systems that depend on visual stock control and management, such as barcode reading, counting, and store interfaces, often use machine vision systems. Large-scale industrial product runs also employ machine vision systems to assess the products at various stages in the process and also work with automated robotic arms. Even the food and beverage industry uses machine vision systems to monitor quality. In the medical field, machine vision systems are applied in medical imaging as well as in examination procedures.
Machine vision represents a diverse and growing global market, one that can be difficult to keep up with, in terms of the latest technology, standards, and product developments, as they become available from hundreds of different organizations around the world.
If you are looking for an example of how fast the market moves, and how quickly trends and new technologies emerge, our Innovators Awards provides a good reference point. In 2015, we launched our first annual Innovators Awards program, which celebrates the disparate and innovative technologies, products, and systems found in the machine vision and imaging market. In comparing the products that received distinction in 2015 to this past year’s crop of honorees, it does not take long to draw some obvious conclusions. First, let’s start with the most noticeable, which was with the cameras that received awards.
In 2015, five companies received awards for cameras. These cameras performed various functions and offered disparate capabilities, including pixel shifting, SWIR sensitivity, multi-line CMOS time delay integration, high-speed operation, and high dynamic range operation. In 2018, 13 companies received awards for their cameras, but the capabilities and features of these cameras look much different.
CAMERAS THAT RECEIVED AWARDS IN 2018 OFFERED THE FOLLOWING FEATURES:
Polarization, 25GigE interface, 8K line scan, scientific CMOS sensor, USB 3.1 interface, fiber interface, embedded VisualApplets software, 3-CMOS prism design, and subminiature design. Like in 2015, a few companies were also honored for high-speed cameras, but overall, it is evident that most of the 2018 camera honorees are offering much different products than those from our inaugural year.
There are two other main categories that stick out, in terms of 2018 vs. 2015, the first of which is software products. In 2015, two companies received awards for their software—one for a deep learning software product and another for a machine learning-based quality control software. In 2018, eight companies received awards for software.
THESE SOFTWARE PRODUCTS OFFERED THE FOLLOWING FEATURES OR CAPABILITIES:
Deep learning (three honorees), data management, GigE Vision simulation, neural network software for autonomous vehicles, machine learning-based desktop software for autonomous vehicle vision system optimization, and a USB3 to 10GigE software converter.
Lastly, the category of embedded vision looked much different in 2018 than it did in 2015. In the embedded vision category—which I am combining with smart cameras due to overlap—there were two companies that received awards in 2015, both of which were for smart cameras that offered various capabilities. This year, however, there were 12 companies that were honored for their embedded vision innovations, for products that offered features including: embedded software running on Raspberry Pi, computer vision and deep learning hardware and software platform, embedded vision development kits, embedded computers, 3D bead inspection, as well as various smart cameras.
Throughout the other categories, there was equal or similar number of honorees from both years, but there were several interesting technologies or applications that products that popped up in 2018 offered. This includes a lens for virtual reality/augmented reality applications, a mobile hyperspectral camera, a 3D color camera, and various lighting products that targeted multispectral and hyperspectral imaging applications.
This is all to say that, when looking back to 2015 to today, machine vision technology has grown quite a bit. With the rapid pace of advancements, the growing needs of customers and end users, the miniaturization and smaller costs of components, and so on; it is exciting to think about what machine vision products in 2021 might look like.
Vision systems for robotic manufacturing applications have significantly evolved over the last decade. While the vision systems of old were unreliable, clunky and expensive, today’s systems are anything but. Proper vision systems can make the difference between an efficient robotic system and one that is not working optimally.
HERE ARE 5 MYTHS AND TRUTHS ABOUT VISION SYSTEMS.
Credits : freepik.com
MYTH #1: VISION SYSTEM ARE COMPLICATED
In actuality, modern vision systems are very simple to install and use. Most of the algorithms and communications are built in, so it can be very easy and quick to make adjustments without the help of a trained engineer. New users are often surprised just how easy it is to use and maintain their vision systems.
MYTH #2: VISION SYSTEMS ARE NOT RELIABLE
If a vision system is properly applied, it will be highly robust, repeatable and reliable. Today’s vision system components are very robust, even in harsh environments. They are built to operate in rugged applications. Unlike a human, a vision system will see accurately every time. It never gets tired, takes a break or goes home for the evening.
MYTH #3: ALL VISION SYSTEMS ARE THE SAME
There is no truly out-of-the-box solution for vision systems. Each application is unique, and many factors of need to be considered. Anyone who tells you there’s a plug-and-play option for your operations is not selling you a solution that’s properly engineered for your needs. Customized vision systems are the only ones that will work efficiently and reliably.
MYTH #4: VISION SYSTEMS ARE ALWAYS THE BEST SOLUTION
While vision systems are helpful in many robotic applications, there are some jobs in which vision may not be the answer. For example, operations that have drastic changes from part to part moving quickly on a single line may not benefit from a vision system because more discriminating inspection may be necessary. In addition, a vision system helps provide tight tolerances, so applications with loose tolerances may be just fine with sensors and not need to be upgraded to a vision system.
MYTH #5: VISION SYSTEMS ARE TOO EXPENSIVE
Just 10 years ago, typical vision systems cost an average of $30,000. Today, that same system may cost only $5,000 to $15,000. The evolution of vision technologies have brought down the cost considerably. In fact, many companies can see an ROI relatively quickly because a vision system requires fewer special fixtures and conveyors, decreases downtime for fixture changeout, and increases operations overall.
An efficient manufacturer must get products in and out of a cell quickly and reliably. Vision systems paired with robotic operations can put an operation at a competitive advantage by providing opportunities to make more and streamline the process for optimum profitablilty.
MACHINE VISION TECHNOLOGY has found its way into applications inside and outside of factory settings, riding a wave of progress in automation technology and growing into a sizable global industry. Quite a bit of future technology will depend on machine vision, and the market will grow accordingly.
In 2017, according to a recent report, the global machine vision market was valued at $7.91 billion1. By 2023, the global market is expected to reach $12.29 billion – a compound annual growth rate (CAGR) of 7.61%. This robust growth is caused by a number of broader economic factors.
WHAT’S DRIVING LONG-TERM GROWTH IN MACHINE VISION?
The main drivers of growth in the machine vision market are the need for quality inspection and automation inside factories, growing demand for AI and IoT integrated systems that depend on machine vision, increasing adoption of Industrial 4.0 technology that uses vision to improve the productivity of robotic automation, and government initiatives to support smart factories across the globe.
Machine vision software will be one of the fastest growing segments between 2017 and 2023. The main reason for this is the expected increase in integration of AI into industrial machine vision software to enable deep learning in robotics technology.
PC-based INDUSTRIAL MACHINE VISION PRODUCTS, the oldest form of industrial machine vision, will retain a large portion of machine vision market share because of their ease of use and processing power.
Credit : freepik.com
WHAT TRENDS ARE WORTH WATCHING NOW?
While there are several main factors in the expected long-term growth of the global machine vision market, there are a few trends to keep an eye on now that are changing the way machine vision technology is deployed.
Industrial Internet of Things (IIoT): while AI and IoT technology are long-term drivers of growth, the IIoT is connecting production technology with information technology in today’s factories to increase productivity. The IIoT depends on heavily on machine vision to collect the information it needs.
Non-Industrial Applications: driverless cars, autonomous farm equipment, drone applications, intelligent traffic systems, guided surgery and other non-industrial uses of machine vision are rapidly growing in popularity, and often call for different functionality in machine vision than industrial applications. These non-industrial uses of machine vision are being deployed today and could be an important part of machine vision growth.
Ease of Use: Machine vision systems can often be complex from the user’s perspective. As mentioned above, PC-based machine vision systems will remain popular, despite their age, because of their ease of use. The desire for ease of use may drive further standardization in machine vision products, which could make them even easier to deploy inside and outside of factory settings.
The machine vision market is poised for long-term growth. The IIoT, growing non-industrial applications and ease of use are all helping buoy today’s machine vision market, but there are several other factors effecting long-term market expansion.
Many applications demand more from a lens than the performance delivered by STANDARD IMAGING LENSES, which introduces the need for ruggedized imaging lenses. A new imaging choice is the Stability Ruggedized imaging lens.
JESSICA GEHLHAR, VISION SOLUTIONS ENGINEER, AND CORY BOONE, OPTICAL ENGINEER, EDMUND OPTICS
Historically, Industrial and Ingress Protection Ruggedized imaging lenses have addressed many environmental and application challenges. But as imaging systems increase the number of moving elements and as products move through inspection systems faster for higher throughput, these movements require greater calibration and image performance. As applications like factory automation, measurement, robotics and autonomous vehicles continue to expand and develop, the need for Stability Ruggedized imaging lenses will increase along with the progression of the industry.
Each of these applications presents environmental challenges–such as shock, vibration, and contamination–to imaging systems. Unlike lab and observatory setups, which tend to have relatively controlled environments, manufacturing facilities can be rife with environmental operating difficulties.
To address these challenges, ruggedized imaging lenses have a number of features and benefits. But to determine the best ruggedized lens for an application, first let’s clearly define the various ruggedization techniques.
A STANDARD IMAGING LENS can be insufficient in some applications and environments due to the large number of moving parts within the lens assembly, such as the doubled threaded focus adjustment, the multi-leaf iris diaphragms, and their corresponding thumb screws. For example, the thin overlapped iris leaves are especially susceptible to high shock and vibration, which can cause them to easily spring out of place and be damaged. By replacing the iris leaves with a fixed aperture, the survivability of the lens can be greatly improved.
Another component of the lens that may come loose during shock and vibration are the thumbscrews. Although they may not completely fall off, they can be loosened enough that the focus changes, potentially degrading the image quality. On a machine vision inspection system, faulty image quality can increase the potential to reject passing units or pass failing units. Debris and contaminants in the area can compound these effects.
Historically, there have been two primary ruggedization techniques to address these environmental difficulties – Industrial Ruggedization and Ingress Protection Ruggedization.
IN AN INDUSTRIAL RUGGEDIZED IMAGING LENS, MANY OF THE MOVING PARTS OF A STANDARD IMAGING LENS ARE ELIMINATED:
the multi-leaf iris is replaced with a fixed aperture stop
the focus adjustment is replaced with a simple single thread
the thumb screws are replaced with set screws.
Industrial Ruggedization prevents many of the unintentional movements and focus shifts described above, and therefore maintains ideal image quality. Industrial Ruggedized imaging lenses can also prevent a user from accidentally changing the focus and iris settings.
In an Ingress Protection Ruggedized imaging lens, the lens assembly is either fully enclosed or sealed with O-rings (or RTV silicone) to withstand environmental contaminants. IP66 or IP67 environmental ratings are the most well known standards for particulate and water resistance.
Enclosures and seals can be especially critical for lenses used to inspect food quality. The lenses must withstand direct exposure to liquids and humidity in such wash down applications.
Many manufacturing, processing, and packaging applications are in unfavorable environments where dust, debris, dirt, adhesives or fluids are commonplace. Ingress Protection Ruggedized imaging lenses are designed to withstand these harsh environments.
Some of today’s more demanding applications in factory automation, measurement, robotics, and autonomous vehicles levy additional requirements on imaging systems beyond those of Industrial Ruggedization and Ingress Protection Ruggedization. In such situations, there’s another type of ruggedization: Stability Ruggedization. The EDMUND OPTICS TECHSPEC COMPACT RUGGEDIZED (CR) SERIES FIXED FOCAL LENGTH LENSES are an example of Stability Ruggedized imaging lenses.
While Ingress Protection Ruggedization prevents contamination and Industrial Ruggedization eliminates moving parts, Stability Ruggedization maintains (or stabilizes) optical pointing and positioning even after heavy shock, vibration and temperature change. In a Stability Ruggedized imaging lens, the individual lens elements are glued into place to prevent them from moving within the housing.
In an optical system, lens elements sit within the inner bore of the barrel. The space between the outer diameter of the lens and inner diameter of the barrel is typically less than 50 microns; decenters of even tens of microns are enough to significantly affect the pointing of the lens.
When using a Stability Ruggedized lens, if an object point falls on the exact center pixel, it will always fall there even if the lens has been heavily vibrated; therefore pixel shift is reduced and the image is stabilized.
Even in clean inspection environments with well-controlled robotic movements, there are challenges. Conveyer lines and robotic systems move at higher speeds and handle heavier products than ever before. Vibration comes from operation speeds and weight of the objects, or by the lines and systems located next to equipment. These high intensity environments create tough shock and vibration requirements for imaging systems, making the lens performance more critical.
Additionally, users have higher resolution and image quality expectations. As camera pixel sizes become smaller, even slight misalignments in imaging systems become apparent. Pointing and alignment changes that were once unnoticeable with a camera having 5.6 micron pixels may be very obvious with a camera having 1.4 micron pixels either over time or after a strong shock.
Stability Ruggedization is important in applications where the field of view must be calibrated, such as measurement equipment, 3D stereo vision, lenses used for robotic sensing, and lenses used for tracking object locations. These applications often require the pointing to be stabilized to values much smaller than a single pixel.
In 3D stereo vision, two imaging lenses are used to image a pattern that’s been projected onto a 3D object. The two images are then compared to extract the 3D information about the object, but to do this the angle of the two lenses and field of view must be well calibrated.
Once calibrated, any shift in the pixel mapping will offset the information in the 3D model, affecting the measurements. Many times these systems experience heavy shock and temperature shifts during shipping or relocation. If the system were to require recalibration for each relocation, it would be costly to send a technician onsite, whereas setting up and calibrating the system once would keep costs down.
Another similar application is distortion calibration. Information is not lost in an imaging system that has distortion – the information is simply moved. Distortion can be mapped or calibrated from your imaging systems to remove it. If pixel shift occurs, the distortion mapped is now incorrect; shifting the distortion calibration map can move your values affecting your accuracy.
Another recent challenge machine vision imaging systems face is an increase in traditional robotic imaging systems crossing over into autonomous systems, along with many embedded vision solutions beyond the relatively stationary conveyer line and robotic setups.
Vision enabled robots rely on imaging systems to know where they are in space. As robots move to perform their task, the constant movement can cause pixel shift, affecting their ability to know where they are and their accuracy. Machine vision is also expanding beyond the final pack-out robots on a line. After products are packaged and boxed up, vision-guided robots can transport and load them on and off of trucks – even the trucks themselves may be autonomous vehicles, guided by many sensors including vision.
Back in the factory and warehouses, there are more autonomous robots, driving or flying around to move products or inspect storage locations. Large distribution centers often carry a variety of goods, inspection and handling systems to handle a variety of weights, sizes, and packaging materials. As industrial machine vision systems make their way outside of typical applications and onto autonomous vehicles, once ‘controlled’ environments, no matter how challenging, are no longer controlled in many ways.
The future depends on monitoring and regulating air pollution, which is an essential step towards creating a cleaner environment.
MONITORING MARITIME POLLUTION WITH OPTICS
Monitoring and regulating air pollution is an essential step towards creating a cleaner environment. Danfoss IXA, a high-tech company based in Denmark, is developing a device called MES 1001, a marine emission sensor based on ultraviolet absorption spectroscopy which monitors the NO, NO2, SO2 and NH3 emissions produced by cargo ships to ensure that they are complying with all environmental regulations. The optical sensor is placed inside the exhaust system of ships, so the involved optics will be exposed to extreme conditions and must be able to withstand temperatures up to 500°C and very high pressures simultaneously.
Danfoss IXA was looking for a partner to develop optics fulfilling their demanding requirements, and in EDMUND OPTICS (EO) they found a partner who was prepared to take on this challenge which went beyond their normal capabilities. EO created custom test beds for verifying the unique requirements of the sensor, which enabled EO to develop a robust system to meet Danfoss IXA’s specifications.
Danfoss IXA develops sensors and systems for the maritime industry, focusing on energy optimization and the measurement of emission gases. They are a part of the Danfoss Group, a global enterprise which produces a wide range of technologies that address a variety of markets including food supply, energy efficiency, and climate-friendly solutions.
Smokestack emissions from international shipping are a severe problem for human health, contributing to the premature mortality of people all across the world from lung damage and cardio vascular diseases.
CREATING A CLEANER ENVIRONMENT STARTS AT SEA
The International Maritime Organization (IMO) has recently decided that commercial ships must comply with low sulfur fuel requirements globally by 2020. In addition, the current Nitrogen Oxide emission control area along the North American coastline will be expanded to cover the Baltic and North Seas in 2021. There currently aren’t convenient and reliable ways for the IMO to monitor ships’ emissions and enforce these regulations. A multitude of local and regional initiatives seeking to limit the air emission from ships further underline the fact that the industry needs to adapt to a world where strict emission requirements are part of the game. Danfoss IXA is developing the MES1001, which is a comprehensive marine emissions sensor suitable for accurately measuring a ship’s air emissions in real time.
Danfoss IXA approached several providers of optical components to jointly design the optical system for the new MES 1001 device. This project turned out to be very challenging due to the extreme high temperature and pressure requirements. High temperatures can cause optics to fail due to melting and thermal stresses, which severely limits the types of optical materials that can be used. High temperatures can also cause adhesives used in the optical assembly to outgas, contaminating the system. The high pressure requirements made the sealing of the optical system critically important. Most of the optics partners faced their limits in terms of design, metrology for these harsh conditions, or working across different continents and time zones.
EDMUND OPTICS (EO), with its global presence and large staff of optical engineers and designers, is always keen to face new challenges. One of the reasons that Danfoss IXA selected EO as a partner is their ability to ramp-up products from prototype to volume production. When approached by Danfoss, EO dedicated R&D and project management resources to developing an optical assembly for the MES 1001, even though EO had never designed systems to work at temperatures as high as 500°C before. EO investigated many different materials and mounting options, recognizing this project as a learning experience and opportunity to expand their capabilities. Custom testbeds for verifying the optical system’s unique requirements were created and proper sealants and optomechanics were identified to allow the assembly to survive these high pressures. The start of that development process was faced with many issues including cracking optics and outgassing adhesives, but by iterating the design process multiple times and researching in different materials these issues were solved and Edmund Optics eventually delivered an optical assembly that could survive the harsh environment inside a ship’s exhaust system. Edmund Optics is proud to be a part of this product which will positively impact the environment and support a global effort to reduce harmful emissions.
Danfoss IXA “greatly appreciated EO’s professional way of involving [them] along the development process as well as their ability to adapt to changing requirements as [Danfoss IXA] learned more about the exact conditions in which the sensor would be used.” During that time Danfoss IXA “found the support from EO’s project managers extremely fruitful and very efficient in bringing the development process to success.”
The robust optical system is a critical component of the new MES 1001 device, which was launched in 2017. It was exciting for EO to work on this cutting-edge technology in such a close collaboration with Danfoss IXA’s skilled research and development team. The MES 1001 will allow the IMO and other organizations to enforce maritime emissions requirements and help lead to a cleaner environment across the globe.