“It’s the software, stupid,” quipped an industry executive regarding the importance of software relative to hardware components in machine vision systems (paraphrasing a U.S. presidential candidate’s widely-repeated comment from years ago about the economy). It is oftenand correctly stated that acquiring a high-quality, consistent image by means of proper illumination, optics and camera selection can account for 85% of the success of a machine vision application. This is the important “image delivery” side of machine vision system integration.

Ultimately though, a machine vision system must execute competent software algorithms or “tools” with the acquired image to process, analyze, and produce data that can be used to reliably evaluate the status of the part or feature to be inspected. It is the software that drives a system’s capability, and in the machine vision marketplace, the software also has an important role in product differentiation and selection.

Machine vision software and system architecture

The term “software” in a machine vision system can refer to 1) the functional tools that perform operations on the acquired image and control the system hardware, and 2) a graphical user interface (GUI or UI) that is used to configure parameters and settings. Depending upon the component architecture, a product might provide one or both of these software features. To clarify, let’s review how machine vision software fits into some of the most widely-used machine vision system architectures currently found in products for industrial automated inspection.

Smart cameras and similar systems

Most smart cameras and similar vision components implement a system architecture where only the software for the functional tools resides on the inspection device.  A proprietary GUI program for system and tool configuration, unique to that product or product family, runs separately on a “development” PC connected to the machine vision device, usually over a high-speed Ethernet interface.  In general, this is an efficient architecture that allows the machine vision device to dedicate all processing to the execution of the inspection task without the software and hardware overhead required for user interfacing.  

In almost all of these configurations, the development computer can be disconnected in production to allow the component to run standalone. However, the development computer might be included as part of the plant-floor system integration to act as the operator interface for one or more of the smart cameras in the system. Alternately, manufacturers of smart cameras and similar components often offer other configurable user interface devices and/or software libraries to interface to third party UI displays or computers intended primarily for results display, selection of inspection configurations, and other tasks related to the operation of the system. In most cases, the implementation of this type of interface is done via other configuration tools or by code using standard software languages.

Systems with an integrated computer

Certain machine vision products combine a camera or cameras with a full featured computer that directly interfaces with a standard or proprietary monitor and a keyboard/mouse or other input device to provide an interface for development and display. In some components, the system is built around an industrial PC (IPC) as a computing platform running a Windows or other third-party operating system. In some cases the computer is proprietary to the specific product, and may also use a proprietary operating system. In yet another related architecture, a standard IPC is packaged with camera sensor and electronics in a device that physically resembles a standard smart camera as described above. In all cases, for this architecture the GUI is a part of the machine vision component, and the software for the machine vision tool functionality and the GUI for configuration and interfacing are, from the viewpoint of the end user, combined.

A complement to this type of system architecture is the ready availability of standard “off-the-shelf” software packages that combine the complex machine vision software tools with a standardized GUI for application and user interface development. Running on an IPC, these packages can offer a robust system, with scalable hardware configurations, usually without the need for low-level program coding.

Machine vision software libraries

Machine vision integrators, OEMs, and end-users who need a more advanced or customized inspection solution may turn to the wide variety of machine vision software libraries available. A machine vision software library provides a deep selection of tools and algorithms which must be implemented in low-level code like C, C++, .NET or other standard programming languages under Windows and other common operating systems.

A machine vision software library provides only the tools and algorithms for the execution of a machine vision application. The programmer must develop a GUI program as required, although many library packages do offer some form of “Rapid Application Development” feature that might speed the system programming process.

It is generally accepted that the depth and scope of algorithmic capability found in most separate software library products exceeds that offered in the “integrated” products described above. Of course that expanded capability comes with additional integration complexity.

Important features and functions of machine vision software

As noted earlier, inspection software tends to drive differentiation and selection of a machine vision component for a general purpose application. (Hardware architecture, resolution, features, and product cost of course are important factors outside of this discussion.) Machine vision engineers often have a degree of comfort with and “loyalty” to a particular machine vision system, in part due to familiarity with the 1) software tools and functions, and 2) the configuration offered in a graphical user interface (GUI).  Certainly, there are systems which provide some very unique and powerful tools that perform a specific function for a targeted application or result. In those cases, that capability alone may dictate the selection of that product above all other considerations. Likewise, the use of a system with a familiar interface can reduce the learning curve on new applications. In any case, it is important for new and even current users of machine vision technology to understand key features and functions in the machine vision software that may be critical for the target application.

What to look for in tool capability

Most general purpose machine vision products contain at least a base set of well-implemented algorithms that can successfully perform many general inspection applications. Having a variety of tools is important; even machine vision experts might find, as a project progresses, that inspection functions other than those originally considered might be needed to make the application robust and reliable. In a general purpose machine vision system, one would expect to find the following tools and functionality.

Camera configuration: The system should easily interface with the camera(s) (or image sensor) to control image acquisition parameters such as exposure time, gain, and perhaps provide the ability to acquire partial images. Functions that perform automatic exposure or gain adjustment are a plus.

Image processing: Image processing (or “preprocessing”) is a step in the inspection task where the image can be manipulated to enhance features that will be processed by image analysis tools.  There are many standard tools for image processing, and at the minimum a system likely would offer simple filtering and some morphological operations.

Image analysis: These tools extract information from the image about features in the image, and do the bulk of the work of the inspection task. A well-designed machine vision system would certainly offer a wide variety of image analysis tools including: histogram analysis, blob (connectivity or particle) analysis, edge and edge-pair location, and geometric searching. It is common and very valuable to have “composite” image analysis tools available in a system. These tools are pre-configured processes that perform a standard task and are intended to be easier to use than configuring basic tools from scratch to perform the same task. In some cases, as noted earlier, these tools might be unique to a specific product, and could provide a significant benefit to the targeted applications. Examples of this type of tool would include surface analysis (defect detection on a homogeneous background) or print quality inspection.

System functions: The ability for the machine vision software to perform a variety of system functions is an absolute requirement for the system. In particular, access to I/O (discrete, analog, and network protocols like TCP/IP, Ethernet/IP, DeviceNet and others) is necessary for the exchange of control and inspection information. In particular, the ability to seamlessly communicate with other automation devices like robots is a must for many applications.

A final word about inspection tools: Every tool must be configured or “tuned” for a specific inspection. This is accomplished through the use of various tool parameters that control the execution of the tool algorithm. Different machine vision software packages provide varying levels of access to the tool parameters for adjustment. It might be important in any given application to be sure that the software selected provides sufficient parameterization to “fine-tune” a particular tool for a given task.

What to look for in a machine vision GUI

It is not only the software tool capability, but also the operator interface ease of use and function that is key to the machine vision application execution. The GUI program configures tool parameters and organizes the execution of the tools. This is a critically important function, and perhaps is the most visible differentiation between general purpose machine vision systems.

Machine vision application configuration GUIs feature a wide variety of methods to order the execution of the machine vision tools, and to direct the sharing of information between tools. The “look and feel” of machine vision configuration software packages varies widely as does the method that tools are manipulated in the software (spreadsheets, wired links, icons, lists, and so on). All are intended to provide the end user with a means to set up a sometime complex series of machine vision tools to perform an inspection, without a significant need for programming in a cryptic language. If the interface is too simple, it might not expose tool parameters that could be important to a specific application, or it might not allow logic functions like looping, branching or conditional execution which might make the inspection more efficient. Alternately, if the interface is too complex with extensive options and features, it might be considered no longer “easy to use.”

Key differentiating factors for machine vision software graphical user interfaces will include:

Ease of use: the interface method through which the user configures the inspection program—often a very subjective concept.

Configurability/parameterization: this is key to flexibility in the software and the inspection application.

Conclusions

 Machine vision inspection software, while in many cases tightly linked to a hardware component, is in itself a critical part of every machine vision system. Many choices exist, and it is important to understand both the architecture and the features of machine vision inspection software when selecting and implementing an integrated inspection solution.