The benefits of smart cameras have made them an ideal match for a manufacturing environment. Combining a camera, processor, software and communications in one small, low-power package, smart cameras are highly integrated optical inspection tools for controlling quality and increasing productivity—required tasks in manufacturing. Smart cameras were developed in the 1990s but only became practical in the last decade due to improvements from consumer electronics and the development of easy-to-use software.

Two levels of smart camera exist, based on capability and price: 

A vision sensor is a specialized smart camera that provides limited functionality in a single view (image). In a bottle inspection application, this limited function might be reading a barcode on a bottle, reading the date and lot code, looking for the position of a label or checking that the bottle’s closure (cap) is seated by measuring its vertical position. Single view means that only one image is used per inspection cycle.

In addition to the elements of a smart camera, a vision sensor might have options for built-in lighting and inexpensive lenses. In machine vision, lighting is a critical part of any vision system. Integrating these elements and having only a single function in a single view keeps system cost low and simplifies setup.

A smart camera vision system (SCVS) has a single view but provides many functions on that view. In the bottling example, a single SCVS could check the presence and position of the label, read the barcode and verify the date and lot code. If an item, such as the barcode, is not in the SCVS’s field of view, a second SCVS could be used to read the barcode.

The smart camera’s ultra-small form factor (44 x 44 x 39 mm) is ideal for space-limited applications. Source: Teledyne DALSA.

However, for very demanding tasks, such as high-speed printing inspection or functions requiring integration of multiple views, a standard vision system should be used, rather than a smart camera. A vision system consists of a larger, higher-performance processor connected to one or more standard machine vision cameras. It can process multiple views (camera images) with many functions for each view. For example, with four cameras in a vision system, you could combine four views of a bottle to inspect the “lip” of the bottle for damage.

Smart Cameras in a Bottling Inspection

On a bottling line, bottles are labeled, filled, and closed (capped) at speeds that are well beyond human vision. In approximate order of difficulty, here are some machine vision tasks—the more basic tasks can be handled by a smart camera, but the more complex would require a traditional vision system.

• Verify the product by reading a barcode
• Check the presence and quality of a label
• Verify the date and lot code
• Check for proper closure
• Check printing or seal on each closure
• Check the “lip” of the bottle for damage
• Look for foreign material in the bottle

Financial and practical considerations drive the amount of machine vision inspection applied to a bottle, or any manufactured product. For instance, faded printing might only cause a consumer to pick another bottle, but foreign matter in a pharmaceutical might cause injury and an expensive lawsuit.

How Software Plays a Role

While a smart camera and a smart phone might have similar hardware, software shapes what you perceive as the product. Early machine vision systems were typically provided with a subroutine library and required extensive expertise in both programming and machine vision algorithms. 

This smart camera is a highly integrated optical inspection tool for controlling quality and increasing productivity. Source: Teledyne DALSA.

Modern machine vision software uses three techniques to allow easy development of machine vision applications. First, a graphical user interface (GUI) visually presents the components and algorithms in a machine vision system in a way that is easy to use and easy on your memory. With a GUI, you get what you see and do not have to remember obscure commands and function calls.

Second, individual algorithms have been gathered into tools that incorporate expert knowledge. A common example is a barcode tool that incorporates knowledge of what barcodes look like, the various types and patterns of one and two-dimensional barcodes, and what image processing operations are needed to read the codes. You simply draw a rectangle in the image that surrounds the barcodes, specify the type of barcodes and how many barcodes to find. The processing and reading is done by the tool, with no additional knowledge required on your part.

Third, for sequences of tasks, graphical programming is provided. In graphical programming, you use a GUI to insert functions and control operations into a program—a processing sequence. In our bottle inspection example, a smart camera vision system (SCVS) might first read a barcode to determine which product is present. Then it might measure positions to check if the closure is seated and the label is in position. Graphical programming provides excellent ease and flexibility in solving vision tasks when combined with GUIs and tools.

How Smart Cameras are playing a role in IoT, Deep Learning, Big Data and the Cloud 

There are interesting and important connections between smart cameras and current technology trends. The Internet of Things (IoT) is a catchphrase for internet-connected “smart things” such as home thermostats and security systems. Smart cameras are IoT-ready devices, as they “talk” to production control systems using internet protocols. As such, a smart camera can be discovered and configured to support “on demand” manufacturing. However, exposing your machine vision system to the wider internet is usually not necessary and is a security risk.

Deep learning refers to a multi-layer, artificial neural network where all layers are “teachable.” It would be great if a vision system could be taught to recognize good and bad parts from example images of, say, good and bad carrots, a challenging application because the definition of “good” varies, unlike a machined part. 

Big Data—in this case, a large set of training images—are needed to train deep learning networks as the network must “learn” both the critical elements (features) to extract from the image and how to make good and bad classifications based on those elements.

However, there are challenges with this approach for machine vision. First, gathering a large set of training data is usually difficult to do, particularly if product defects are rare. Second, training the neural net takes a lot of computing power, perhaps from distributed processing in the cloud, and so is impractical on a smart camera. However, once the network is trained, it could be transferred and run on a smart camera. For instance, vision tasks such as face recognition, currently run on a smart phone using hardware similar to smart cameras.

The combination of dramatically improving hardware, driven by consumer demands, and modern software technology has made smart cameras capable, easy-to-use and inexpensive. We look forward to smart cameras and vision systems becoming even smarter and easier to use.