You Make It— I Break It

The Importance of Validation Testing.

Validation testing plays a significant role in product development. Have you ever wanted to rip the radio out of your car and smash it on the ground? I’ve taken things out of my car and smashed them – and it’s quite satisfying! I didn’t do it out of frustration, I did it as part of a structured automotive interior test plan. Having spent several years in automotive validation, some of the most interesting aspects of my role have involved dropping, freezing, and crushing parts to ensure that they still function. This may sound like fun and games, but validation testing is a critical step in the product development process. Finding failures as soon as possible allows for proper troubleshooting and product evaluation prior to further testing and production. The purpose of this blog is to communicate the significance and the elements of a good test plan.

The Significance of a Good Test Plan

It acts as the main script to follow during the validation phase. A good plan shows the overall strategy as well as the day-by-day (sometimes hour-by-hour) schedule of tests and deliverables. Having a thorough test plan in place far in advance of testing provides the opportunity for all roles involved to be on the same page in terms of requirements, desired results, equipment, personnel, and timing. It provides a means for tracking test completion as well as tester issues or product design issues.

It ensures production-readiness and customer durability. The plan must be comprehensive in its scope. The product designer spends a significant amount of time considering the potential failure modes of a product. The validation plan includes any tests necessary for evaluating those risks. A good plan delivered to the customer prior to testing assures them that the end product will not only be fully ready for production but will also be durable and satisfying for the end user. While accuracy of the test plan is important, it is also necessary to allow room for changes as lessons are learned during the early phases of testing.

It acts as a tool for estimating the resources and effort needed. As the test methods are documented and the timing evaluated, a good plan helps the team determine how many people are required, what equipment needs to be purchased, and what will be outsourced. The program manager will use this information to estimate a final cost for full validation.

A test plan team will increase your ability to create and maintain an effective test plan. The team members brought together for test planning often include a:  Validation Engineer, Product Engineer, Mechanical and/or Electrical Engineer, Systems Engineer, Validation Technician, Test Development Engineer, and Program Manager. DISHER has experience in each of these roles. We also can help guide you through the development and execution of any type of test plan.

Elements of a Good Test Plan

1. Clear document format. In my experience, customers often require the use of their DVP&R (Design Verification Plan & Report) form. These are excellent for test tracking as they include: order of testing; test name; test specification and section which is referenced; test sample quantities per test; timing of each test; and a section for reporting test results. If you need to create your own form, multiple examples are available online for reference.

2. Solid understanding of test specs and methods. The validation engineer must fully comprehend the test methods outlined in the customer’s specification. Having this knowledge will allow the validation engineer to ensure that the test is set up correctly and train anyone performing the test.

3. Proper equipment. The validation team must understand the capability of their lab’s equipment in order to plan for testing. If equipment is purchased, it must be properly configurable to meet the test requirements and perform consistently for the duration of testing as environmental tests can involve months of continuous usage. New equipment, software, and processes should be proven out in advance of testing. Some tests can be outsourced to companies that specialize in certain types of testing, but it is important to understand their capabilities and accreditations prior to sourcing them. Timing of outsourced tests can also be a major concern or roadblock; therefore, multiple sources may be necessary.

4. Repeatable tests. Repeatability of a test is vital in gathering good data. If the test cannot be performed consistently, then it is often difficult to know if an issue is caused by a part failure or the tester itself. Temperature and humidity cycling is common in environmental testing. Equipment must have the ability to repeatedly achieve the specified test parameters. Other examples include: using fixtures for drop testing or chemical applications; using robotic or automated equipment to actuate buttons, levers, etc.; using software programs for electrical test applications; and using recording equipment and sound measurement software for noise tests in order to eliminate subjectivity.

5. Clear description of test method. Customer test specifications often describe the test method desired. However, some specs reference other test specs, which can reference more test specs, and so on. Therefore, it is the responsibility of the validation engineer to decipher all the different components and include only the necessary test steps clearly within the test plan.

6. Clear acceptance criteria. Upon completion of a test, there must be no ambiguity regarding the outcome. The customer’s specification will almost always show exactly what constitutes a pass or fail for any given test. This information must be clearly shown within the test plan. However, I have seen multiple instances where a spec will simply read, “Must meet functional status class X”. This does not provide enough information for the person performing the test to make a determination. The validation engineer must research exactly what is needed per the customer and include that information within the test plan’s acceptance criteria. This should be done in advance of setting up the test.

7. Updated timing. The overall testing timeline is developed at the beginning of each program, typically with input from the program manager. The customer’s end date is referenced along with the estimated duration of each test to complete the test plan timeline. The ADVP&R will be presented to the customer for approval prior to the start of testing. Any deviation from the timing shown within during testing needs to be reported.

8. Timely reporting of results. Regardless of what your test plan may look like, at some point you will need to report the results to your customer. The requirements for this reporting vary by customer. Some customers don’t want to see any results until all testing is complete. Others want the test reports monthly or perhaps weekly. In one case, a customer required that they be contacted within 24-hours of any test failure. This could become an “all hands on deck” exercise if the validation team needs to determine if an issue was a part failure or a tester issue. Parts certainly can fail during testing, but I have seen many instances where it was determined that the test was not completed correctly or the equipment malfunctioned. This absolutely needs to be evaluated prior to contacting the customer.

The amount of effort put into advance planning of your validation testing will directly correlate to the success of the product. A good test plan will meet your customer’s needs, be easy to understand and follow, and ensure that the product is the best it can be. It’s wise to start the planning process as soon as possible! Integrate the right people from your cross-functional teams. Make sure everyone understands the customer’s specs and agrees with the test methods. Gather equipment and start training far in advance of any testing. Then communicate with the customer regularly and report the results clearly. Your customer can rest assured at your thorough product validation. As a result, you won’t feel the need to rip out a radio and smash it onto the ground in frustration! Instead you can turn up the tunes and simply rock on to the next project.

Looking for a proven validation process, validation training, or an experienced validation engineer? DISHER would be glad to help you with any phase of the product development process.

Written By: Aaron Denbow, Electronics Engineer
Aaron has 20 years of experience in the automotive electronics industry, including printed circuit board design, prototyping, and production. As a Program Manager, his passion for customer service helps him build positive, effective relationships.