Manufacturing Reflections

Last week NPR interviewed the author of Why Bad Things Happen.  In an automation world we diagnose causes to improve prevention, but ultimately without going into MPC religion, “stuff happens”.  My engineering brain focuses on remedies.

Part of the solution is good control design – spending the time to figure out what can happen.  Look at reams of past data (electronically) and test a control design against significant process history, interview experienced and not-so-experienced operators about what they worry about, watch a lot (and why) and compare a problem with similar solution designs in your or peers experience.  Make the design sustainable within the imperfect data and processing world of it’s environment.

The other step is to test the solution through each designed function and conceivable events.

1. What will the controller do if important controllers are unavailable, what will if do if a possible plant section is not operating
2. Is the controller designed to recover smoothly from loss and re-connectivity to your control system
3. Is the controller designed to recover smoothly and automatically from a computer reboot or loss/recovery of power.
4. How does the controller respond to projected bad data or flaws – can you test it with clearly bad data sent to it’s systems (like a zero to a controlled variable)

Your final catch-all is to confirm that the controller limits are appropriately set and at levels your controller can survive.  Manipulated Variable high and low, rate-of-change limits are to keep you out of the ditches and limits on the control system itself, required for plant, equipment and operator safety should be tested.  Confirm that if computer based control sends zeros out (which has happened on uninitialized controllers) that a lower-level control would, where appropriate reject or maintain it’s limits even with wrong values from a computer-based control system.

Operator training needs to include basic troubleshooting and required monitoring training.  An operator should know the basics of what your controller is designed to do and be able to reasonably question things that don’t seem right.  The operator should be able to turn off a suspicious control function (with on/off control swithes), but be required to write down the whats and whys.  This log is critical in your troubleshooting (along with uptime monitoring) and determining if a controller patch or operator training is warranted.  Then each event can rapidly be resolved and repeats avoided.

Mike T.