A “false color” 3-D image of concrete railroad ties shows tie degradation. The custom analysis software must isolate the tie from the gravel and then evaluate its condition. Source: Nagle Research
Smart cameras are nothing new. They have been around since as far back as 1985 and perhaps even earlier. Their value proposition is both immediately apparent and enticing: inspect and measure automatically without the complexity, cost and hassle of adding a separate computer system and writing custom software. But for all the promise that they hold, and often fulfill, they are not a one-size-fits-all solution. Smart cameras represent an evolving sector of the machine vision industry, but as with most things, tradeoffs must be considered before deciding to use a smart camera technology in a vision application.
Hardware Performance
With smart cameras, on-board image processing hardware has evolved over the years. The smart cameras of today are sometimes orders of magnitude faster than their counterparts of years past. But for all their improved speed, they will never be able to match the performance of a separate PC dedicated to processing images in real time.Modern PC hardware has a decided advantage in this regard. Multiple CPU cores (processors), memory cache to expedite data access, and voluminous redundant array of independent disks (RAID) hard drive configurations for nearly endless data storage allow a finished solution to be more than a basic vision system. It is often advantageous for a final solution to store statistics, a history of processed images and possibly provide a human machine interface (HMI) to allow credentialed operators to adjust inspection parameters, which also can be tracked and logged.
It is the nature of electronic devices to become smaller and more powerful, and computers are the quintessential example of this phenomenon. Computers with two, four or more processors now can efficiently be squeezed into a space scarcely larger than a paperback.
A “false color” 3-D image of the bottom deck of a wooden pallet is shown here. Each board must be isolated and graded individually, and any of more than a dozen defect types identified and measured. Source: Nagle Research
Software Performance
Some smart cameras can be programmed using traditional programming languages such as C or C++, but more common of late is the trend toward flowchart-driven programming methods designed to insulate the implementer from the complexities of the underlying code. In general, a palette of tools is presented like building blocks to be assembled procedurally to solve a specific task. And again, many applications can be broken down into these coarse steps.Children, for example, often enjoy playing with plastic building blocks, building houses, castles and animals, as the plastic blocks allow them to achieve a certain level of detail in their designs. On the other end, video game artists use sophisticated 3-D tools and paint software to create detailed models for the games they create.
These two very disparate tool sets accomplish-at least superficially-the same goal, that is, to model reality artificially. Plastic building blocks are great for children, as they render reality precisely at the level of granularity children care about. But ultimately they are unable to provide the detail and nuance required by those in the game industry.
So it is with handcrafted C or C++ code as compared to building-block software. The required image processing algorithms can be refined and polished in solution-specific ways. The HMI can be customized without limitation. Interactions can occur with any number of other system components. In practical terms, there are few limits that cannot be overcome with the proper application of appropriately powered hardware and well-designed software. With that in mind, here are some examples of advantages PC-based vision has over smart cameras.
This screen shot of the 3-D pallet load inspection system was created from two separate sensors, which were stitched together prior to analysis. Source: Nagle Research
This is really the greatest value proposition of the PC-based vision solution. Software development is, by its very nature, a heuristic endeavor. A programmer with a depth of knowledge and experience in a particular problem domain can work with the data in an almost organic way. Custom software developers have a toolbox they use as well, and each tool can be customized for the task at hand.
A typical workflow for custom software developers involves gathering massive amounts of actual data at the customer site early in the project. The developers then can work with that data off-line and refine the algorithms without unduly imposing on the customer’s productivity.
Where smart cameras shine is when the items to be inspected are presented in a deterministic or predictable way, and when the items look largely the same from one to the next. But when there is a high degree of variability in the part itself, it becomes much more difficult to solve with the smart camera.
For example, one project involved developing an automated pallet inspection system. When developing a high-speed pallet inspection system using several 3-D cameras, it is important to deal with the variability of the pallet. In short, they’re like fingerprints: no two are exactly alike. The analysis algorithms have to be flexible enough to identify and measure more than a dozen different defects on these pallets. It would have been impossible to do that with smart cameras.
The issue here is having various cameras contribute data to the final image that is then analyzed. As an example, another project involved high-speed 3-D inspection of molded carbon fiber sheets and the resolution required was too much to encompass with a single camera. Developers had to engineer seven 3-D cameras side-by-side and write software to combine the seven images into one seamless 3-D image.
After the algorithms are selected and implemented, developers refine and optimize the code as necessary to meet the customer requirements. When every millisecond counts, this kind of intense effort pays dividends. The extra performance can be used to obtain a more detailed image to analyze, or even allow the manufacturer to increase throughput.
The 3-D pallet load inspection system combines the inputs from two separate sensors to perform the required analysis. Source: Nagle Research
Custom Software
After all the research, team whiteboard sessions and implementation, the result can be a powerhouse solution. But just because a solution is created in code does not automatically mean that it is an optimal, or even a good, solution. Along with the power and flexibility that modern software tools afford, so too comes the responsibility to apply those tools correctly.Seasoned software developers see things like this all the time. It is the inevitable product of improper planning, tool selection and inadequate evaluation of the chosen developer’s skill level. Smart camera-based solutions are no different in this regard, except, to paraphrase a popular programming text, you get less rope with which to shoot yourself in the foot. The availability of simplified development tools does not eliminate the need for foundational planning and a clean implementation.
Smart cameras are a great way to solve many common vision challenges, and some skilled integrators have implemented impressive solutions with those tools. In fact, sometimes it is difficult to draw that line and say conclusively that these tools are or are not enough to get the job done. It takes an honest assessment of what will be required both now and in the future. If someone decides to use a smart camera and he hits a brick wall, the work done up to that point is largely unsalvageable.
Smart cameras will always trail dedicated computers in raw computing power and flexibility in the final solution. Regardless of which development path is chosen, however, the quality of the final solution will be a direct reflection of the developer’s preparation, experience and skill. V&S
