Most manufacturers are faced with an increasing number of instruments requiring calibration, while simultaneously reducing the resources available to maintain them. One popular method for minimizing the resources needed for maintaining instrumentation is the implementation of a calibration management software (CMS) system. Under current Good Manufacturing Practices (cGMP) and 21 CFR Part 11 Electronic Signature/Electronic Records requirements, this software application must be validated, because it is part of the quality assurance of the manufactured product.

CMS systems have increased in functionality. The typical calibration software system not only tracks equipment inventory, calibration schedules and histories, but increasingly ties into calibration procedures, contains calibration interval analysis capabilities and other advanced features.

With the increased capability of this software, manufacturers have come to rely on it. Validation provides assurance that the system is reliable and demonstrates that the system operates in a state of control.

With current staff reductions and increased workloads, it becomes a matter of prioritizing validation projects and identifying the level of detail needed for the particular system to be validated. A current topic among validation specialists revolves around the identification of the level to which a system must be validated. A useful technique for doing this is the Risk Assessment Analysis (RAA). This method identifies the relative risk and the corresponding level of validation required.

It is therefore prudent to select a validation process that produces the highest level of compliance with a minimum of resources.

User requirements specifications

When validating a CMS system, verify the functionality of the software against the user's application requirements. Because the purchaser of a CMS system does not have control of the specifications the developer uses to design the software, the user must identify his own requirements and match these to the available systems.

The most common method for doing this is through generating User Requirements Specifications (URS). This document describes the application from a functional viewpoint. The emphasis should be on the required functions, not on the method of implementing those functions. A requirement specification should be written for each function of the software. According to Good Automated Manufacturing Practice Guide 4 (GAMP), the following guidelines are given for establishing URS.

• Each requirement statement is to be uniquely referenced and no longer than 250 words.

• Requirements statements should not be duplicated or contradicted.

• The URS should express requirements and not design solutions.

• Each requirement should be testable.

• The URS must be understood by both the customer and supplier; ambiguity and jargon should be avoided.

• When possible, the URS should distinguish between mandatory and regulatory requirements and desirable features.

• There may need to be a formal review of the URS between the customer and supplier to check understanding and that requirements have been met, or not, in the functional specification.

For a calibration management system, some general areas for requirements include:

• Ability to add, maintain and retrieve master equipment records.

• Ability to add, maintain and retrieve calibration history records.

• Ability to track schedules of future calibration due dates.

• Ability to print calibration forms.

• Ability to compile reports for calibration interval analysis.

• Multiple level password protection capability.

• Audit trail capability to track changes to the database.

• Internal backup and restore capability.

It is the purchaser's responsibility to ensure that the software system selected meets the URS. A comparison of the software specifications to the URS should be made and documented. Because not all software systems meet every user's requirements, the user must also document any discrepancies between the software and the requirements, including any work or procedures that eliminate the discrepancies.

The preparation and verification of the URS is vital for demonstrating that the system is suitable for use in its intended application.

Software vendor qualification

Because CMS software systems are intended for use in a regulated environment, it is the vendor's responsibility to use an appropriate development quality assurance methodology. It is the purchaser's responsibility to verify that the vendor has the appropriate methodology in place and that it is capable of developing high quality software.

There are many software development models commonly accepted as being capable of producing high quality software. Among these are the Life Cycle model defined by the Pharmaceutical Manufacturers Association's Computer Systems Validation Committee and GAMP 4. It is the purchaser's responsibility to understand which model the vendor uses and whether it is sufficient to develop the quality of software for a given application.

Verification of a vendor's development process can be accomplished in a number of ways, including vendor surveys, audits and review of vendor-supplied documentation. The method with the highest level of assurance is the vendor audit. The vendor audit is costly, and for a calibration management software system, is rarely justified. The need for an audit can be eliminated by a combination of a written survey the purchaser sends to the vendor, as well as reviewing documentation the vendor supplies. Some vendors make copies of their software quality assurance procedures and records available to their users.

Another important method for determining a vendor's software development capability is through examination of the vendor's reputation in the industry. Check with other users, including those who have audited the vendor to see what their experience has been.

Validation

Once the software system and vendor have been qualified for use in the user's application, the software can be implemented. The implementation of the software involves many aspects including installation, training, procedure writing and validation.

The key to validating a CMS system is preparing and implementing a validation project plan. The validation project plan provides a prospective plan that when executed, produces documented evidence that the system operates in a state of control. With a correctly prepared validation project plan, the work is planned in advance, carried out in logical order and easily documented.

There are several, commonly accepted formats for organizing a software system validation project plan. An acceptable format to both manufacturers and regulatory agencies has an appropriate level of detail, while not encumbering the manufacturer with unnecessary work. Such a format includes:

Scope. This section identifies the goals and objectives of the validation. It typically contains information on where the system is implemented and the products or processes it affects.

Validation team approvals. It is important to list key individuals involved in the validation effort. Their responsibilities during the validation should be listed for reference. Typically, one individual is designated the validation manager. This section should contain a form to be signed by all individuals with project and system oversight responsibility. The corporate organization structure and the scope of the project, in conjunction with internal policies, will determine the highest level approval required.

Validation certificate. The validation certificate is a concise summary of the basis for determining that the system has been successfully validated. It should be designed to identify the benchmarks, initials and dates.

System validation plan. This section contains the overall plan and basic validation requirements. It should include a system characterization and reference to any supporting documentation, such as user requirements, vendor qualifications and software user manuals. It can be in outline form with numbered steps identifying the aspects of the plan.

System validation protocol. The protocol provides an outline of expected test outcomes that would indicate that the software's performance is satisfactory. It also can include information on how the test results should be documented. This section is an overview of the tests to be done in the operational performance qualification.

Installation qualification. For a CMS system, the installation qualification can be broken down into four subsections.

• Standard operating procedures. These procedures describe the rules under which the software will be operated. The key areas for a calibration management system are security, operator training, system and data backup and recovery, and use of customizable software functions.

• Hardware configuration. This is a list of hardware used to run the software. It should include system processor, operating system and version, and any network connections.

• Software configuration. This is a list of information about the software being used, such as version, release ID and file dates. The date installed and who installed it should also be noted.

• Documentation library. This section should indicate all relevant documentation pertaining to the software installation. This can include user manuals, standard operating procedures and validation documentation.

Operational qualification. This portion of the validation program describes how the critical aspects and functions are to be tested. It contains test scenarios that challenge the system in situations which it would be used according to the software vendor's specifications. Once the tests and acceptance criteria are defined, they are executed and documented. The documentation should, at a minimum, include the test results, a pass and fail indication, and signature. Many companies include screen captures as added confirmation the tests were actually completed. Most vendors in the FDA market have pre-packaged validation protocol scripts and these can greatly decrease test script writing time. The vendor should keep these updated on a regular basis to reflect changes in cGMP interpretation.

Performance qualification. When implementing commercial off-the-shelf software and using predefined validation scripts, it is important to test and document the software. This is typically the last step before going live. This involves writing usage standard operating procedures and testing them under actual circumstances. It also tests the software through all of the processes a typical calibration group faces, such as creating current due lists and handling out of tolerance conditions. As with each area in the validation process, documentation is paramount. Complete description and documentation of this step will streamline audits and inspections.

Validation currency maintenance. There are various situations that will require revalidation and these should be addressed before they occur. These situations include moving the software to a new system and installing new versions. Procedures should be identified that describe implementing these changes and what revalidation should occur.

In most cases, the majority of time spent implementing the plan will be in the operational qualification stage. This stage is critical, it is where the actual tests that challenge the software occur.

When documenting the tests, screen captures of the software in action can eliminate some of the documentation demands. Each test should be dated and signed or initialed. It is suggested that notes, check marks and dates be hand written, showing active participation by the validation team. The plan and resulting documentation should be kept in a three-ring binder, with each page numbered and dated.

On completion of the validation plan, vendor qualification and user requirements, all of the resulting documentation should be organized in a central location. Plans should be prepared to identify what is to be shown to an auditor and the order in which it is to be presented. If this is identified up front, it will save time in the audit and help demonstrate that the software system is under control.

By proving the software meets the application requirements, the vendor meets industry accepted development criteria and the software meets the validation plan criteria, the manufacturer can be confident that the software system is suitable and that it will hold up under regulatory scrutiny. Q