Teaching robots to read
Large-scale production requires highly scalable assembly processes and detailed quality inspection. Fully automated robotics systems put parts together, guide them through the factory, and assure that quality standards are met. Dr Rainer Pausch of ABBYY looks at implementing text and barcode reading in machine vision systems.
Algorithms relying on visually acquired information use modern machine vision and computer vision systems, where the combination of an industrial camera and a computer-processing unit simulates the function of human visual control. Machine vision-based robotic systems are able to ‘see’ and make decisions based on the visual input rather than just executing pre-determined actions like, for example, a static welding robot on a production line.
By incorporating sophisticated text recognition Optical Character Recognition (OCR) technologies, machine vision systems can gain ‘reading’ capabilities. Text-based information printed on labels, displayed on control panels or visible on computer screens can be captured, interpreted and processed.
While information on paper-based documents such as service reports or product documentation can be converted as part of a scanning process, automation systems capture this information with cameras that have been implemented directly in the production line or testing facility. In both scenarios, the information is available as ‘image-only’ information. OCR technology is then needed to convert this and make it machine readable, searchable and usable.
Like humans, the systems can ‘read’ information and subsequently react. Such information includes text and barcodes on packaging as well as images of text on electronic displays. With integrated data capture technologies, robotics, computer vision and quality control systems can process the extracted data and compare it with entries in the database to initiate the system to take the appropriate action. Converted information can be exported to applications and systems in the form of XML data and plain text. With these new capabilities, robotics can conduct a range of completely new tasks.
By implementing sophisticated technologies for text recognition, such as OCR, hand print extraction (ICR), barcode recognition (OBR) and optical mark recognition (OMR), independent software vendors and hardware providers for the manufacturing industry can significantly extend the functionality of machine vision systems – and offer more value to their customers. Armed with these abilities, manufacturers can put their machines to work to answer some of the more challenging questions that occur at the end of their production lines.
For example, touch panels and user interfaces in various consumer devices can be automatically tested by simulating the individual steps. During the screening process of the touch-enabled device, the individual areas of the screen are activated by the robotic platform and response messages are captured. The extracted text is automatically validated by a computer with integrated OCR technology.
Domestic appliances, such as washing machines, microwaves or dishwashers contain such panels, as do car infotainment systems, which are increasingly becoming the norm in modern vehicles. In order to enable communication between driver and vehicle to take place through the car’s infotainment system, a team of engineers ensures that these systems in the car functions accurately and reliably. For Volkswagen, the testing department was looking to automate rigorous testing processes. To improve the automated testing processes, they introduced OCR technology to verify the content appearing on the screens – even for complex display messages and in various languages.
During the automated production process, individual parts or partially finished products need to be routed to different locations. By integrating text and barcode recognition technologies, automation systems in the production line can then be programmed to ensure that parts automatically follow pre-defined routes.
By applying OCR to photographed labels, packaging or documentation, text contained in them can be extracted and checked against databases. This way, products with incorrect information can be singled out automatically before they leave the manufacturing site, thereby reducing the margin for error. This process can be applied to anything from information on pharmaceutical packaging, labels on beverage bottles or information about allergy or expiration dates on food packaging.
Even processes in which human interaction is necessary – such as during service and maintenance tasks – can be significantly accelerated by the help of text recognition and data capture technologies. These technologies automatically extract service steps information from scanned paper documents, for example notes and tick box information on repair slips, and make this data available in the central repository.
The benefits of teaching a machine to read and react for itself across industries where quality and accuracy are key, will ultimately save time, money and help achieve customer satisfaction.
Latest news about Vision systems