From dimensional and quality inspection through to optical path programming, from 2D robot guidance to the latest three-dimensional technologies: an overview of the artificial vision systems offered by SIR Soluzioni Industriali Robotizzate.
VISION AND ROBOTICS: THE WINNING COMBINATION
Of recent years, the use of artificial vision in industry has spread rapidly: image processing technologies have been successfully integrated into robotic systems, in operations involving the control and guidance of articulated robots in complex spatial situations. Amongst the many system integrators who have pursued this route, SIR Soluzioni Industriali Robotizzate has been particularly impressive, due both to the wealth of know-how acquired in 26 years of top-level automation and the complexity of the solutions it offers, which make it one of the industry’s main players at the international level. Proving itself a real problem solver, SIR designs and produces robotic solutions in a variety of applicational contexts, from assembly to process machining and from logistics to handling, in all modern industrial manufacturing sectors. At least 70% of the systems it designs and produces every year involve the use of image processing systems for the most widely varying requirements, and the diversity of the individual problems faced has led the company to develop an original hardware configuration and a family of software applications with functions covered by Italian and European patents.
THE HARDWARE AND SOFTWARE CONFIGURATION
Its familiarity with the use of robots has enabled SIR to fully exploit their precision and flexibility in the vision context as well: why not use the articulated robot itself as an image acquisition unit positioning tool? The solution involves the use of an integrated illuminator/camera unit consisting of a matrix of white, red or IR LEDs, depending on the application and the typical characteristics of the workpiece for localisation: material, shape and feeding scenario. The integrated assembly can be fixed on board by means of quick coupling/uncoupling modules, becoming one of the robot’s many end-effectors. This enables the articulated robot to use the image acquisition unit, consisting of B/W or colour cameras, only when necessary, avoiding all the restrictions of a permanent mounting. Cameras permanently fitted on the robot create problems of bulk, or the risk of damage to the cameras due to the dusts and liquids typical of industrial environments. In contrast, the SIR solution avoids these basic drawbacks and features an adjustable-angle LED matrix which allows the light to be conveyed into the Field of View (FOV) correctly. But the benefits do not stop here. The configuration also eliminates complex, rigid calibration procedures, and allows the piece to be examined from different angles and distances, allowing an impressive degree of flexibility. The positioning precision of modern articulated robots, of the order of 0.1 mm, guarantees the absolute reproducibility of the control conditions.
From the software point of view, SIR has developed a proprietary package, with Multi-Purpose characteristics, called VistaVision®. Built with a modular architecture, it combines the best of the sector’s most famous software libraries with a number of tools patented by the company, developed to resolve special problems. The system, which has now reached Release 4.1, is continually evolving, thanks to the constant implementation of new functions. The high calculation speed combines with user-friendly graphics and the ability to adapt to any type of scenario or application.
2D ROBOT GUIDANCE
The key application of the configuration described above is in robot guidance, meaning the localisation of objects and the calculation of their coordinates (X, Y and their angle of rest on the plane), to allow them to be picked by a robot or another device. Pieces are generally fed by means of indexing tables with universal smooth tops, or on pallets or in bins, tidily arranged with or without tier sheets (2D and 2½D conditions). The robot performs N scans of the layer divided into lines and columns (viewing grid), correlating the individual FOVs in a single, high-resolution calibrated image. Using state-of-the-art multi-model geometrical Pattern Matching mechanisms, the system is able to locate all the pieces in the layer, lying in different positions (front, back, etc.), or sort different objects in case of mixed feeding: the gripping position during picking is accurate to a few tenths of a millimetre or a degree, more than enough for reliable picking. The system’s most interesting features are the strategies for establishing the correct picking sequence, and a number of very useful additional tools: first and foremost, the algorithm for checking the gripper descent zone, which prevents collisions if the robot locates between the object for picking and those next to it, and continuous, instantaneous control of the lighting level in the area for inspection, with feedback action on the camera exposure for total or local correction of the under and over-exposed areas.
QUALITATIVE AND DIMENSIONAL INSPECTION
Thanks to the combined use of the VistaVision® software and special lenses and lighting equipment, (Backlights, Domes, special LEDs, etc.), SIR has designed and built complex machines for the inspection of mechanical components. These are able to check correct assembly, identify significant details, check for blocked holes and damaged threads, measure fundamental distances or the diameters of holes or bevels, and, last but not least, perform surface inspections. These may include inspection for cracks, porosity, dents, unturned residues, burrs, excess materials such as sandblasting residues or burns caused by worn tools. The specifically dimensional inspections have provided impressive results in terms of precision, with tolerances which may vary from 0.1 mm down to the order of hundredths of a millimetre, depending on the FOV set, the lens and the hardware used. Similar results have been achieved in the analysis of small-sized dimensional flaws, including on concave and convex areas or pipeline entry zones, inspected with pinhole lenses. Thanks to the use of the latest segmentation techniques, combined with a new technology for dynamic adaptation to variation in pieces, reliability of over 99% can be achieved. It is worth pausing a moment on the inspection of threads, with examination of the individual crests using endoscopic probes with coaxial illumination, capable of delivering a good image even on diameters of the order of 6 mm. From the logistical/construction point of view, SIR inspection systems may be of two different types, depending on the cycle time required and the problems to be resolved. The first type comprises a carousel with a number of control stations which may range from a minimum of 4 up to a maximum of 8, one of which provides the robot-tended loading/unloading bay. The other stations are used for one or more types of inspection, meaning that hardware configurations may vary widely: each station has several cameras mounted in different positions, often with controlled axes. The most recent implementations include the dimensional and quality inspection of automotive transmission components (65 inspections in 12 seconds) and the testing of household appliance motor components (58 checks in just 7.5 seconds).
In the alternative configuration, small articulated robots are used to position the cameras, using the interchangeable end-effector concept. Although the cycle time is slightly longer, this solution is more flexible than a rigid machine, with lower hardware costs: the robot can be programmed differently from piece to piece, by modifying the inspection type and viewing positions. This solution has been successfully applied in the inspection of Diesel and petrol engine turbochargers, where the articulated robot performs thirty or so tests, ranging from thread inspection to porosity checks: in this case, the piece’s three-dimensional nature renders the use of fixed cameras highly problematical. The robot is also able to acquire data with interpolated motion, with synchronisation of the acquisition triggers and strobe lights, a technique adopted for inspecting the lapping of through pipes: in this case, the robot moves in synchrony with a backlight which counter-rotates at the opposite end. A number of applications using the “infinitely small” concept, processing images obtained directly from electron microscopes, are currently under development.
VISION AS A TOOL FOR THE SELF-PROGRAMMING OF MACHINERY
Amongst the many modules that make up the VistaVision® system, the EasyPath is definitely one of its most distinctive features. It involves genuine optical programming of paths, a useful tool that provides time savings of 80% over the traditional method which requires the manual acquisition of hundreds of points and a plant stoppage. Unlike robot CAD simulations, which need the 3D mathematical drawing of the piece, EasyPath uses an ordinary image acquisition unit and allows calibration to be performed on the real piece, with adaptation to any variations in the castings or the quantity of burrs. Here again, a vision grid of lines/columns generates a single, extremely high resolution image, eliminating the optical deformation zones. A profile extraction tool establishes the contours for machining. When the tool data (diameter and form) and the feed speed and approximation are then set, the software calculates the machining path, discarding the points which are not required or which generate inconsistency in the trajectory (areas with sharp recesses). Once programming is complete, EasyPath can also be used in RunTime mode: the tool is able to recognise the sections of path which vary from the nominal contour, due to burrs or particularly significant variations, on the piece currently being processed. The next step is feedback adjustment of the robot’s behaviour, involving local modification of the path, variation in tool feed speed or performance of more passes to remove the surplus material. The process’s quality and precision, generally around 0.2 mm, are guaranteed by suitable path alignment methods.
THREE-DIMENSIONAL VISION
In spite of the ambitious goals achieved with EasyPath, SIR continues to invest considerable resources in R&D to equip its systems with original functions: the most interesting recent development is the launch, scheduled during the next few months, of CubicVision, an application developed for 3D robot guidance in different contexts. Thanks to the joint use of complementary localisation technologies, such as camera/laser reconstruction, stereoscopy and the use of three-dimensional Pattern Matching systems with monocular image acquisition units, the user will be offered a multi-purpose system capable of generalising the whole concept of three-dimensional vision, regardless of pieces’ feeding systems, type and form, restrictions which have often hindered the adoption of 3D vision in an industrial context. This is combined with the implementation of appropriate picking strategies, developed for the most widely varying operating contexts. The aim is to develop software, not yet available on the market, which is reliable and applicable in the vast majority of cases, capable of acquiring the spacial position (X, Y, Z and Euler angles) of pieces in a state of random disorder.
Thanks to the concept of vision as a robot end-effector, SIR has also introduced stereoscopic systems, in which just one “virtual” camera is moved into the acquisition positions. Unlike fixed image acquisition units, this Hand-Eye method relieves the operator of the need to perform complex calibrations by regulating the working volume on the basis of the dimensions of the component to be identified and simply saving the new localisation positions. Stereoscopic vision requires the acquisition of at least 3 details of the piece, used for the subsequent triangulation: this function is provided by an algorithm for recursive positioning in 2+N positions and appropriate tools for the correction of optical aberrations. In scenarios in which stereoscopy might be restrictive, the system is able to switch to the use of a camera rigidly connected to a laser beam emitter. This pair of devices, calibrated for a specific range of action, is able to provide reconstructed images in which the intensity of each pixel represents the height of the component displayed. It will be possible to filter the pieces with specific height characteristics on this image, or users may opt for dedicated Pattern Matching algorithms capable of defining spacial positions using information about the object’s deformation compared to the reference model. This algorithm is the heart of the third technology under consideration, monocular 3D vision. In this case, the spacial position and angle can be established from a single viewpoint by observing the object’s planar characteristics or measuring the relative perspective and distance deformation of areas on the same plane. Another algorithmic approach allows the import of the 3D mathematical drawing of the piece for localisation, for automatic processing of a finite series of two-dimensional models obtained through rotating traverse; these are then compared with the RunTime image in order to identify their homography. The skilful combination of software algorithms and hardware configurations will be able to solve the specific problems of every single application: SIR robots will therefore be able to use a standard system capable of adapting the various contexts and overcoming the restrictions imposed by the need for orderly piece feeding. This innovation, of fundamental importance for the sector, will satisfy a growing demand from the market and open out new scenarios in the field of intelligent automation.
Davide Passoni
R&D Department
SIR spa
