We have an openPDC server concentrating info from 2 other openPDC servers (via GEP) and trying to send information via TCP. I start the openPDC service and it starts getting info, but never outputs any info. I believe this issue has to do with the initialization of the output adapter, but I have not seen this before. Before the openPDC service stops, I get the Warning: “Initialization of adapter <OUTPUT_STREAM_ACRONYM> has exceeded its timeout of 15 seconds. The adapter may still initialize, however this may indicate a problem with the adapter. If you consider this to be normal, try adjusting the initialization timeout to suppress this message during normal operations.”
Please let me know if I can provide more information.
Is this a custom output adapter that you have developed, or is it one of the adapters provided with the openPDC installation?
I personally did not setup the system, but there is nothing on the “Manage Custom Outputs” page in openPDC Manager.
Our outputs are on the “Concentrator Output Streams” page. I don’t think this answers your question. Is there a good place to go to find the answer?
Oh, I see. I guess I misunderstood. It’s not an output adapter that is failing to initialize, it’s your output stream.
I am unable to reproduce the issue. Can you tell me what version of openPDC you are using on the server with the failing output stream? Also, what type of database are you using to store the configuration for that instance of the openPDC?
The failure seemed pretty random, but I did notice that the OPENPDC drive on one of the openPDC servers that was feeding the one I am having issues with was full and that service had stopped. I cleared some archive files on that server, and started the service there and it started without crashing. That server only outputs using GEP though.
Obviously this instance on this server isn’t as straightforward.
In the openPDC folder, there should be a folder called ConfigurationCache. In that folder, there should be a file called SystemConfiguration.bin. Can you send me a private message and attach that file to it?
It looks like your output stream configuration is fine. It takes about 20 seconds to process its configuration in my debug environment, so that would explain the message about the initialization timeout. Feel free to add InitializationTimeout=30000 to the output stream’s connection string.
I still can’t reproduce the crash. Can you send me the ErrorLog.txt from the openPDC folder? It may have information about the crash.
I see a lot of unhandled exceptions, but they’ve all got basically the same info. The important part is in the stack trace: InterSiteData.Internal.EmpSocket. We can narrow the cause of the crash down to your ISD adapters. Whichever adapter this is, it’s running a DNS query to find the IP address of a hostname that it’s trying to connect to and failing. Hopefully, that should be enough information to help you troubleshoot.
Okay. Thanks for the help! Ill post my findings.
So basically we had custom action adapters setup to send information to old servers. The DNS names of those servers were recently removed, and that is what caused the issue. To solve, I just disabled the obsolete action adapters and this fixed the problem.
A possible improvement to the software would be to throw an error, but not have openPDC crash due to this. Obviously easier said than done.
Thanks for the help!
Thanks for the suggestion! However, that improvement would need to be brought up with GE. Unfortunately, because of the way the .NET Framework was designed, there is no way to for the openPDC to prevent modules from starting their own threads, throwing unhandled exceptions, and crashing the whole application. There isn’t even any reliable way for our code to determine which adapter spawned the exception. The only halfway reasonable option for us is to log the exception on the way by.
1 Like