I have a pretty annoying problem that has been consuming most of my time over the past days.
My configuration is a cRIO 9014 with a 9013 TC module and a 9472 PWM module.
The only signals I have are three PWMs (using FPGA) being sent to three eurotherm solid state relays (which then control the three different bands of a furnace) and 6 thermocouples positioned at various points along the furnace.
I can control the furnace and measure the temperatures without any problem up to about 850 oC.
Once at least three of the bands are close to this temperature range I SUDDENLY start getting huge thermocouple noise with a period of 25 seconds (please see attached image).
I have tried using different TCs, different cables, different NI 9213 modules, different cRIO power supplies (currently using a 24V, 5Amps one, which should be OK for just 3 PWM signals). I realised that the problem occurs only when my solid state relays (SSRs) receive a pulse width of larger than 0.2 to 0.25.
The only "cure" of the problem was when I split the thermocouple readings into two 9213 modules (i.e. 3 TCs each).
The noise appeared to go away independently of the pulse width sent towards the SSRs. However, this is not a neat solution as the particular RIO is designed for doing more jobs (with lots more TCs required) and I cannot afford to have one 9213 for every 3-4 TCs or so.
Finally note that when measuring the TCs with T > 850 oC using STD TC measurement equipment e.g. Fluke TC reader, the signal appeared to be OK. Well, I did get a random 1-2 oC noise but I guess thats perhaps acceptable.
Please, any suggestions? Has anybody come accross such a problem in the past?
Thanks for your time,
I suspect that you have some coupling between the power circuit (heaters, solid state relays, and PWM) and the small signal circuit (thermocouples). The reason the problem appears to suddenly start is that the SSRs are fully on or fully off until the temperature gets close to the setpoint at which time they begin switching states frequently (about every 25 s!). Plot the power or voltage to the heaters at the same time scale as the temperature images you posted. I expect that you will see strong correlation.
Look for ground loops, inductive coupling, and power supply coupling. Are the TCs grounded, ungrounded with grounded sheaths, shielded or unshielded, in the same conduit as the power cables to the heater? How long are the TC wires and the power cables? What is the power and voltage rating of the furnace?
Very good answer Lynn
To add I would consider using a switching Power supply if you can find one for the TC reader circuits.
Linear Power supplies are suseptible to noise spikes from Relays and such.
But the main thing to look for is a ground loop as mentioned.
Has anybody come accross such a problem in the past?
Yep. 850C. One of the weirdest problems I've ever had to diagnose. It was because our furnace was made of Alumina, and it turns out the resistance of Alumina DROPS at higher temperatures (had a reference for it somewhere but I've lost it). Around 850 the resistance is low enough for the SSR power signal to couple directly to the TC's (heater and TC's touched the Alumina). If I remember right, the TC input (cFP-TC-120) could mostly filter this out, but had problems when different TC's went to different furnaces with SSRs out of sync. We fixed the problem by switching to a TC input module with higher common-mode isolation (cFP-TC-125's in place of -120's).
I should add that the exact problems we had were different than yours (the TC's would start to wander all over the place if at least TWO furnaces with TC's going to the same input module were >850C), but I'll bet it's the same problem, differing only because you have different TC input modules.
thanks a lot to all of you. I will start looking into it asap. I can verify that the correlation between power voltage and TC signal is very very high. Changing the PWM changes the noise of the TCs accordingly. We already had a look for DC couplings or ground loops. Maybe we need to spend some more time and just make sure all is OK.
I am very alarmed by what James said. We had suspected such a situation but ruled it out because the phenomenon was pretty much digital. That is it started suddenly close to a T of 850 oC. But now that James has come accross it in the past we will have to think of alternative solutions to perhaps increase the isolation between thermocouples and furnace walls. Also, another possibility could be the usage of NI 9219 4ch TC input that appears to be
more robust when it comes to signal noise.
Thank you all very much for your answers and your time.
Please post back later and let us know what you find. It may be of help to someone else with a similar situation.
just as an update for this topic. After confirming the problem as described above, we tried several things such as trying to better isolate the thermocouples (e.g. thicker ceramics) or changing their position and moving them further away from the from the oven resistances. It is possible that the interference taking place inside the oven is electromagnetic so we did not really have any luck. We then tried using an external RC filter for the NI 9213 module but there was no difference whatsoever.
So we said, instead of the 3000 EUR + 900 EUR setup for the RIO+NI 9213 module, why not try a 50 EUR arduino module build to read TCs. Never expected to improve the situation but it was a common sense step prior to giving another 1000+ EUR to get the NI 9219 RIO module which offers 4 isolated TC channels..
Guess what. The little arduino, incl. a very std RC filter, works like a charm. No problems whatsoever. I am not an electronics engineer (i am a mechanical engineer) but I was deeply unimpressed.
Thanks for keeping the thread updated through your process. This is a reasonably difficult issue to try and troubleshoot over an internet forum, due to the number of components that are interacting and the large number of possible problem spots. I think that this is an issue where utilizing NI's phone support might be a quicker method to getting a solution out. Everything that's been mentioned in this thread is a reasonable suggestion to trying to solving this problem, and I'm not sure why the Arduino was able to work where the Rio and 9213 were not working, but since they have different programming interfaces, I wonder if a difference in the coding of the Arduino could be contributing to this? In either case, I'm glad to hear that your system is working. I also hope that his experience doesn't turn you off to NI in the future.
I didn't see you mention using the common mode detection that is available on the 9213 to determine if the thermocouples are indeed outside the module's common mode range limit. This would at least give you more of a clue as to the source of the issue.
It would also be useful to know how you have the 9213 configured. It has a high resolution and high speed acquisition mode. High resolution will give you 50/60Hz rejection. High speed will be more noisy in general, though from your graph (if I'm reading it right) it appears to be more of an offset instead of actual noise which I believe suggests a common mode issue. Also your statement about reducing the number of channels causes a decrease in the issue seems to also suggest a common mode issue as well. Have you tried connecting COM to mitigate the issue? This will allow the module COM to float to the voltage offset of your thermocouples, but all TCs must be within the full common mode range of the module.