Anatomy of a Support Call

Feb 4

Learn more about Corso Systems’ Process Support Options

With the Corso Systems team continuing to take on more customer and technical support over the last few years, we thought it might be beneficial to share how we handle support calls. The particular example call in this post will cover a lot of bases. It includes PLC support, Ignition Support, and how we start to build in better tools so operators can take quick action and will eventually not need to rely on calling us for ongoing issues.

This particular customer in the example below uses Corso Systems for 24/7 support. We primarily help them with PLC and Ignition issues. As part of their support package, they have full access to our internal Ignition-based ticketing system to track support requests, a dedicated support phone number to reach us, along with a support email address. The customer has one dedicated Controls Engineer on their team and needed more coverage as they have seven unique facilities with individual support needs.

The Initial “Heads Up”

Before the customer called, we received an email from their Controls Engineer that let us know about a workaround for a current issue at one of their facilities. A faulty valve in one of the systems was keeping the plant from starting up. While an electrician was scheduled to troubleshoot the field device and PLC connection, they needed this workaround in the meantime.

During startup, the system had a multiple minute timer in the PLC based on various process conditions. The faulty valve would need to force the timer to complete, so they could restart the process without having to wait on the timer. Typically, it would take 2-3 restarts before other process conditions would override the interlocks from the faulty valve before the process could continue.

First Support Call

About a month after the initial email arrived, we received our first call from the plant around 8:00PM. The operator was running into the issue with the faulty valve and needed the timer reset.

No problem there. We created a support ticket to track the event and started time tracking. We logged into the system, verified everything was communicating between Ignition and the PLC, went online with the PLC, and found the relevant timer.

We gave the operator the go ahead to start the process, saw the timer going, and forced it to complete by typing in the correct time in the PLC logic. We told the operator they were good to go again, they started the plant, and they were off to the races.

Second Support Call

Around 3:00AM the following morning after the first call, the plant tripped offline due to another process interlock around 3:00AM. The operator tried to start back up, but ran into the same issue again, and needed the timer to be reset. They the support line, went through the same process as before and we helped them get back up and running.

Empowering Operators

Sadly, this type of scenario is common in manufacturing. A critical piece of equipment will have a mechanical or electrical issue which takes time to troubleshoot, more time waiting for parts to arrive, and then even more time for wiring or equipment replacement. In these situations, it’s also common to have workarounds like forcing the timer to complete—especially when everyone on the team knows what’s happening and that the issue isn’t an emergency. In this case, the malfunctioning valve was more of a nuisance. If the operator had chosen to wait for the timer to complete (without the workaround), and try again to get up and running, the process could easily add an hour or two of downtime each time the issue occurs.

While we are happy to take calls for this type of situation at any time, and will get to the bottom of the issue, because it had happened twice and was a known problem, there was an easy fix. Fixing the issue and empowering the operator at the same time would reduce the number of support calls. It would also reduce support calls where the operator is already frustrated because the plant is down and they are under the gun to get it running as fast as possible.

In this case, since we only needed to set a value in the PLC so that the operator could start up the plant in Ignition, so it was an easy fix.

In the Ignition Designer, we found the relevant screen for the start up process, and added a button. This button opened up a popup window where an operator could reset the timer themselves with a single button press. To make sure no one could abuse the system by clicking the button ten times in a row, we added a numeric entry field so the operator could enter a PIN number that would effectively password protect this function.

The plant supervisors also wanted to limit the number of resets within a given period of time, so we added a tag in Ignition to track how many resets had happened, and then rate-limited this function to alleviate any issues that could damage hardware in the field.

Once the operator entered the PIN number, Ignition automatically checked the number of previous resets. Then, assuming that more reset tries were available for the time period, the reset button was enabled. Once the operator pressed the button it would add to the total reset count, reset the PIN number, and force the timer to complete. As the time period elapsed, we would drop previous resets from the rate limiting feature, so that the operator could start the process over as needed during their shift.

Wrapping Up

Our solution empowered the operator to solve the problem themselves by clicking a button we added to the screen they were already using the start the plant. We included guard rails with password/PIN protection and rate limiting to make sure they couldn’t accidentally click the button—this also ensured that they had to stop to think before resetting. These features made the operators extremely happy that they didn’t need to call support for a known issue, especially when they were already stressed about the plant being down.

Our goal with customer support is to eventually work ourselves out of the job. Much like using templates in Ignition, if we find ourselves faced with the same issue more than once, we get to the root of the problem. We give operators the tools they need to manage the issues for themselves where possible. And we do our best to make sure that the support calls coming in are for new issues as they arise.

The best support we can provide is no support because production keeps running smoothly. We work hard to make sure our customers are running as much as they want to, our support staff isn’t bogged down with repeat issues we can resolve, and we can help reduce overall stress and anxiety for everyone involved.