Machine vision projects have a higher frequency of these types of problems:

• Projects that do not move forward or move forward too slowly
• Difficulty in getting a suitable quote
• Larger cost and timetable overruns
• Failure to accomplish the mission or retreating from the planned mission
• Unreliable operation or difficulty in keeping it running on a long-term basis

These issues arise even when common good project management techniques are used. A major step towards avoiding them is recognizing and adapting to the differences with machine vision. One might ask “Different than what?” The answer is “different than application-level project types that most assume are similar” but we’ll need to expand on that. This could easily be a “how to do machine vision” set of books but since it’s just an article we’ll need to narrow that a bit. We’ll do that by constructing and comparing a pair of examples, one machine vision example and one non-machine vision example.

Our Machine Vision Example

Our chosen example has these attributes:

• A medium-complexity medium-difficulty automatic inspection application (vs. other major types such as robot guidance)
• The mission is to inspect for “mechanical” defects on a surface ... “mechanical” meaning defects which are disturbances of the surface that don’t change the lightness/darkness or color. And our (internal or external) customer’s customer has specified “reject defects greater than 2mm in length”. Using considerations discussed in a previous article we’ll plan for a rules-based machine vision solution rather than a Deep Learning based solution.
• A fully automated system, including inspection while the product is moving. This is compared to much simpler manual or semi-manual scenarios such as “hand set hand removed” workpiece flow or handheld scanners. So, for example this means that it has to work on the first-and-only try 99.9999% of the time vs. multiple tries being allowable.
• The system/solution is to be added to an existing production process. The alternative is when the machinery is being designed to facilitate the machine vision, such as with a new production machine. So rather than the process design helping the machine vision, the machine vision must adapt to the process’s particulars and constraints.
• A desire is to do it “in house” to whatever extent is prudent in order to achieve cost savings and empowerment, while recognizing that outside expertise may be needed (vs. a turnkey outside solution)
• A requirement to confirm that the project is feasibly doable prior to making the main investment. The alternative is an explicit or implicit “let’s just try it and see if the mission or a portion of it can be accomplished” basis.

Our Non-Machine Vision Example

Now let’s pick an example application-level project type that most assume is similar regarding the required approach. Like most machine vision, our example’s place in automation is to monitor a process, the same general place in automation as instrumentation. Many knowledgeable people would intuitively assume that it can be handled in a similar manner. So, we’ll choose a typical instrumentation application where one might assume that…measuring the temperature of a process and determining whether or not it is within set limits.

The main underlying differences

In this section we’ll discuss the main underlying differences that affect project management. We’ll do this by describing attributes of the non-vision project that are relevant to project management which are NOT true of our machine vision project example.

• A technical non-specialist can do a pretty good job at defining the requirements of the application. The functional requirements are usually the survivability and safety of the equipment, plus an easily defined main mission. Generally, these decisions don’t have heavy interactions between them and so they can be determined separately. As a result of this, it’s relatively simple to project and confirm that the application will be successful. When making operational spec decisions, costs of the ramifications of those decisions is determinable (sometimes with help from equipment vendors, and usually quickly) by a technical non-specialist to use as feedback to guide that decision-making process.
• The solution is centric on one piece of equipment. Creating the solution is primarily a matter of properly selecting, installing and setting up and programming that piece of equipment. As a result, an application engineer from a potential supplier for that type of equipment usually has enough scope to create a solution for the overall application. And the cost of that narrower-scope work is typically low enough (e.g. 5% of the equipment) that potential supplier can afford to and does the solution creation work for “free” for potential customers as a part of their sales process.
• Installation and startup consists of a well-defined installation and configuration process followed by a brief stage of identifying and solving any errors or problems.
• Successful long-term ownership is centric on routine maintenance procedures plus having spares or spare parts on hand.
• The main pre-purchase step for a successful solution is making a good selection of that central piece of equipment.

We already made the foundation statements of this article, which are that all of the above are relevant to project management, and that NONE of the above are true for our example application. Let’s expand on some of the project flow steps on our example that illustrate this.

A successful approach recognizes that a much larger and higher-expertise part of the work of the project occurs much earlier in the process.

The main “rocket science” part of the solution will be creating a physics and lighting solution that makes the defects reliably visible for “simple rule” differentiation. We covered this in a previous article. Note that expertise and help in the simpler area of selecting and applying the vision unit (e.g. a smart camera) is not applicable to this.

Development of the defect detection spec

A successful approach includes handling development of the defect detection spec as a continuous sequence, team effort, with the team including a vision expert and customer personnel. But for illustration we’ll split this into three phases:

Phase 1: Not enough has yet been decided. The only performance statement we have is to catch defects larger than 2 mm. To confirm a solution requires knowing the width, depth, subtleness and nature of the defects that must be detected and evaluated.

Phase 2: In this phase the customer generates more details for the detection spec. They just gave the safe wide-ranging answer: “Detect all such defects, and pass the ones with a length under 2 mm and fail those with a length over 2 mm.” And while they have discussed the typical products and conditions, they haven’t narrowed / nailed these down. The result requires a physics solution than makes even the subtlest defect reliably visible and a system so accurate that it can detect the difference to pass a 1.9999 mm defect and fail a 2.0001 mm defect. It also requires verification of a solution for all of the permutations and combinations of all of the variables that haven’t been narrowed/nailed down. So, at the end of phase two, the spec is unfeasible to accomplish if taken seriously. But phase 2 is still progress.

Phase 3 is continuation of a team effort that includes the customer and a vision expert to guide the functional specification process to a spec which can be taken seriously and which meets the actual requirements while bringing the price down out of the stratosphere. This involves the expert’s ability to “see into the future” (sometimes combined with some lab work) to know the ramifications of functional spec decisions as guidance and feedback to the decision-making process. This same process implicitly is confirming the feasibility of solving the toughest aspects of the application, thus satisfying the requirement to confirm this prior to making the main investment.

Designing the solution

Solution design is largely outside of the scope of this article but here are a few underlying elements relating to project management:

• Our operational spec included a (narrowed) definition of all of the conditions that the solution must successfully handle, and the solution must be robust enough to handle all combinations of all of those variations.
• The solution design includes much crucial work that is completely outside of any single component (such as a vision unit or smart camera)

Installation and implementation

Rather than just installation of a piece of equipment, the physics (including lighting) part of the solution must be designed to achieve reliable simple-rule differentiation of the defects. This deals with complex variables of the product line, products and defects. This may require some on-site adjustments of aiming and location of light sources, and temporary mounting of them until that is finalized. Then all software settings adjust to the resultant sight picture. Generally, the solution will require finalizing / tweaking while running hundreds or thousands of products. This is different than just installing the equipment. Prior lab or simulation work can reduce the amount of this work required on the line if that reduction is needed.

Long term successful ownership requires a different focus than just prescribed maintenance and spares. Most long-term failures arise from failing to create and maintain good documentation of the solution and failure to have someone available who understands the equipment and solution. Without these things in place, what is typically an otherwise easily resolved problem such as a variation/ change in the product or a cleaning crew moving a camera becomes a mysterious problem that nobody is able to fix.

In addition to the usual areas, documentation should include tagged stored images from the system, and a description of the design approaches used for the programming and physics/lighting solutions.

Even with our narrowed example, there are too many variables to give exact solutions, but for persons who already manage technical projects, an understanding and acceptance of the counter-intuitive differences often is sufficient to adapt. For example, in obtaining the much larger amount of expert work and expert help needed early in the project requires different commercial arrangements than a similar-appearing application that is centric on one piece of equipment.

For the counterintuitive areas of machine vision, the learning curve might be five years consisting of five minutes to hear it and then five years to believe it. Hopefully this article provided some information to assist and expedite that process.