How I Achieved an 85% Decrease in Downtime

Probably the most rewarding job of my career was building assembly line control systems. While there are many aspects of the job that I valued, what made it most rewarding was the radical improvements I was able to make to line reliability.

When I joined the organization, they were experiencing as much as 20% downtime for their top level printer assembly lines. When I was given the responsibility for building line control systems for a new series of production lines, I took a very different approach from what had been common wisdom, resulting in average downtime of less than 3%, and the majority of that was normal operator rotation on the line.

How did I achieve an 85% decrease in downtime? Mostly by breaking the rules.

The common wisdom at the time was that since process engineers design the assembly process, they should write the control code for their assembly station. The result was that each assembly line segment was designed and controlled differently than all other segments. Since each line segment was unique, and because the technicians who had to maintain the operation of the line, had no understanding of the control systems, what should have been minor downtime events would take significant time to troubleshoot.

I took a very different approach, I worked with process engineering to establish a common base workstation design and associated line control. This design was termed a “standard stopper set.” This became the base on which all assembly stations were built. Some stations, particularly those which included automation, might include substantial additional logic, but it was always built on top this standard stopper set so anyone familiar with that code always started from a position of base knowledge. I then negotiated with the technical support team to arrange training in programmable logic coding for a cross shift group of technicians. Those technicians then participated in workstation design, implemented the code for those assembly stations, and coded any customization needed. In addition, the line design included a portable line control workstation that could be take directly to each workstation for development and troubleshooting.

In doing this, the technicians gained ownership of the line, including line control, and developed into a community of skilled troubleshooters.

I broke other established rules in the design of those assembly lines. The common wisdom was to maximize utilization of high capital assets like vision inspection and final test stations. However these were complex systems, and analysis of downtime indicated that they were a major cause of line stoppage. So, after a fair amount of haggling, I was successful in getting installed spares added to the line. In this way, if the throughput needs indicated we needed two final test stations, I added a third. Yes it would be idle most of the time, but on the failure of one of the running stations, we could bring the spare online with the flip of a switch, and troubleshoot the failure while the line continued to produce product. These installed spares paid for themselves within two months of line startup.

Existing lines included multiple custom configured PCs. However a failure of any PC would result in significant downtime because a new PC would have to be reconfigured for that station. When I configured the new lines, I established a standard PC configuration which could, with a simple software switch at startup, do any task on the line. With this, and an installed spare, a PC failure resulted in an immediate swap with the installed spare. The boot sequence would pause long enough for the technician to indicate what task the PC was to do, and within a few minutes, the line would back in operation.

While existing lines required a 20 minute labor intensive startup process, the new lines were started in under three minutes with the flip of a breaker. Where once we never powered down an idle line, operations got into the habit of opening the master line breaker if a line was going to be idle for more than an hour.

These approaches, proved so successful that we retrofitted all existing assembly lines using the same concepts, with similar reliability results.

I used a lot of tools from techniques like lean, six sigma, and agile (before that was really a thing). For example, before I was designing line control, I was supporting the line control. I could fix most automation failures remotely, but I always walked the line afterward. These lines were about the length of a football field, and I walked all the line at the beginning of each shift and usually a couple of times during a shift. I would speak with operators and support technicians, but mostly I watched, watched how people interacted with the automation, watched what broke, and how people went about fixing them. In the Lean world, these would be called gemba walks. I called them customer engagement.

The standard configured PCs were based on Toyota’s SMED, single minute exchange of dies. The PCs weren’t stamping dies, but if Toyota could swap out stamping dies in 60 seconds, why couldn’t we do the same thing with PCs. All digital communication on the line was routed to a switch panel. On older lines, these looked like rats nests of wires. On my lines, all patch cables were neatly routed through Velcro ties. The Lean folks would call this 5s, I called it facilitating easy tracing of patch routs.

Probably most importantly, most of the actual work implementing these ideas was done by a group of very talented engineers and technicians. My role was to keep the focus on a target, facilitate effective communication, engage people to take ownership, step in to prove an idea when others said “it can’t be done,” and cheer them on when they succeeded.

Success, for me, has never come from following the rules or applying so called best practice. Where I have been most successful is when I understanding the target, analyzed for causal factors, applied countermeasures to specific intervention points, observed effects, and rapidly applied course corrections.

This entry was posted in On Process Improvement. Bookmark the permalink.

Leave a Reply

Your email address will not be published.

This site uses Akismet to reduce spam. Learn how your comment data is processed.