We’ve written about the importance of pilot runs before here and here, so in this post, I’m mainly going to focus on how to plan for a successful pilot run.
What is a pilot run?
Before we get started, a quick recap. A pilot run is where you do a test run of manufacturing the product using the same production processes, materials, components, equipment, etc, as to be used in mass production, but before mass production begins. The goal is to find and fix issues that would otherwise be a big problem if replicated in, say, thousands of units being mass-produced. It allows you to perfect the production process, debug issues, calibrate the equipment, etc. Not doing a pilot run means you’re going into mass production blind, and any number of damaging problems could occur which I’ll come on to next.
Benefits of doing a pilot run
Pilot runs are beneficial because they allow you to find issues before you start producing thousands of products which could all otherwise end up as scrap. Logically, what business can risk that happening?
By doing a small run of maybe even less than 100 pieces, you will find and be able to avoid all kinds of problems like:
- Issues with pick and place machines
- Pasting and stencil issues
- Parts missing or not fitting
- Soldering issues, such as it being out of place or the wrong temperature
- Testing stations not where they need to be on the line (for some complex products, they needed to be tested at different points during production before the entire unit is assembled because once assembled the modules and/or components are so tightly packed inside the housing it is impossible)
- Automation and jigs are in the wrong position or need to be added to the line
- …and more
It allows the testing of process controls
Typically the factory should be using a process control plan, statistical process control for tracking each process, control charts that tell you if processes are in or out of control and their timings, and monitoring yields on a regular basis. The pilot run lets you test all of these in action and make adjustments where needed.
…and the testing of automation and jigs
Automation and jigs also need to be planned as these will increase the consistency of work coming off the production line, as without them operators’ output might vary depending on if they’re tired, sick, in a bad mood, etc. Jigs are tools that will assist operators to do a process correctly, for example showing a green light if a component has been attached properly, but red if it is not and needs to be done again. Automation is important for many reasons, for example, testing RF can be done very quickly by automatically moving products into the testing chamber where the equipment can check the RF emissions in seconds and then move on to the next product. Once again, the pilot run allows the testing of jigs and automation before mass production starts.
Pre-pilot run activities
Before you’re ready to get started, consider these points first:
- Plan your pilot run ahead of time – done in advance a pilot run will help you decide a number of things you need for manufacturing, such as what equipment, how much space, how many staff, etc.
- Make sure all teams are trained – do your different teams, such as production, assembly, QC, warehousing and logistics, etc, have training on the equipment, the ability to assess what is good and bad, testing procedures, jigs and fixtures, etc, before mass production starts? The pilot run is a great time to do training and work out where more is required.
- Make use of the lag between hardware and software readiness – software usually lags behind hardware by some weeks, so once the hardware design is locked the hardware team can be moved to focus on the pilot run while the software is being finished and can help arrange the production line for it to take place on. Once the software is complete, a test with it on the hardware will be needed, but this is probably not as involved as the pilot run.
- Be sure that equipment is ready – manufacturing requires multiple teams to do multiple tasks and is complex. Sometimes it’s easy to forget to calibrate the equipment, but this could lead to product reliability failures in the field or defects in thousands of pieces. For example, you would need to make sure that the solder temperature is correct and consistent when producing PCBAs.
- Prepare the processes and make them ready – you need to select the most efficient process with the best yield, so you need to examine the process on paper in advance and devise improvements which could mean using automation, adding staff in an area where there might be bottlenecks, etc. Process engineering in this way is one of the most critical tasks for successful mass production.
5 planning steps for your pilot run.
- Consider the scope of the pilot run. For example, you may run a pilot for different reasons and the scope may be quite narrow or broad, perhaps just to test a component or PCB, supplier qualification, software testing, testing a new piece of manufacturing equipment, or testing the full production process before mass production.
- What are the main objectives? For example, if you’re doing a pilot run to test components, your objectives might be to know if the component is good enough quality, if the pick and place machines can place it properly, etc. You might have to do this many times if you have multiple components in the product.
- Who will perform it? You’ll make a pilot run team, so who is in it and what are their responsibilities? The person in charge needs to be knowledgeable about the product, design, manufacturing, components, etc, depending on what is being tested. Ideally, the team will be led by a senior staff member, such as a high-level engineer or project manager who understands the process very clearly from end to end and has the authority and experience to coordinate the different team members.
- Do you have the plan to carry it out? The pilot run plan may include schedules, team, equipment, and components involved, as well as details for second suppliers in case the primary supplier runs out of stock, and anything else important, too.
- How will you document and evaluate the pilot run? In order to assess the pilot run’s effectiveness, you need to evaluate every step of the process and document any issues found and how they were resolved. The documentation may include lessons learned analysis, corrective action plans, and more, and should be retained so it can be used to run future pilot runs more easily. The documentation is also living, as it will be updated over time as more pilot runs are done and improvements to processes are made.
Will your Asian manufacturer do a pilot run? Don’t count on it!
It’s not uncommon for manufacturers to skip the pilot run and rush into mass production as they’re incentivised to ship your products as soon as possible in order to get their payment. For the reasons mentioned here, that’s not good, but less-experienced buyers often fall into the trap of trusting their supplier too much because ‘they know what they’re doing.’ This can lead to important elements being skipped to save them tie and money, such as quality and reliability testing, pilot runs, using adequate packaging that actually protects the products during shipping, etc.
Here at Agilian, it’s an essential part of our new product introduction process.
How does the cost of doing pilot runs compare with not doing them?
Pilot runs vary considerably depending on the products, goals, etc, outlined here, so it’s hard to give a general ‘cost’ for them. However, if you do not do pilot runs you risk schedule and funding overruns during mass production because you’ll run into so many issues (that could have been found and fixed during the pilot) that you’ll need to spend time and money on activities like reworking defective products, overtime pay for operators, etc. There are also costs associated with brand damage attributed to unreliable products, customer warranty returns, and lawsuits or product recalls due to unsafe products, for example.
The pilot run should not even be expensive to set up and run, especially if the manufacturer has already made similar products on a similar production line which can simply be adjusted to the new requirements by adding relevant machines, jigs, automation, operators, additional lines, etc, where needed for the pilot. Not doing a pilot could easily cost 10x more in terms of negative effects.
Conclusion.
Why take the risk of skipping it? A successful pilot run is very beneficial, resulting in smoother mass production, fewer risks (those risks that remain will be known and manageable), and better yields.
So, if at this point you’re wondering how to assure that your new production includes a pilot run, let us know, and we’ll be happy to give you some advice.
Editor’s note: This post first appeared on QualityInspection.org as Pilot Run Best Practices and has been adjusted slightly for Agilian readers.
