Forwarding measurements limitation?

jeffhill · August 22, 2019, 6:24am

Hi all,

I am having success if i forward measurements from
OpenHistorian -> SIEGate(internal subscription) ->internet -> SIEGate(TLS Subscription)->OpenHistorian(Internal)
with a mixture of internal subcription and gateway using TLS subscription.

I am having trouble forwarding measurements to a 3rd SIEGate as in this example
SIEGate->SIEGate(internal subscription)->Internet->SIEGate(TLS Subscription)
.
In this case the 2nd SIEGate receives all the data, but the 3rd one receives nothing, but there are no errors and there are messages about metadata refresh being successful etc. Note this is almost identical to the working example above with data originating from SIEGate instead of OpenHistorian.

Now this works all the way to the last SIEGate
OpenHistorian->SIEGate(internal subscription)->SIEGate(Gateway)

But this doesn’t
OpenHistorian->SIEGate(internal subscription)->SIEGate(Gateway)->Internet->SIEGate
The last SIEGate doesn’t receive any data, but no errors or warnings.

This also doesn’t work
OpenHistorian->SIEGate(internal subscription)->Internet->SIEGate(TLS)->SIEGate(TLS)->OpenHistorian(internal)
The SIEGate with the 2nd TLS subscription doesn’t receive data.

If I pick subscriber status, all the connections have the green light and Stream statistics shows that connection is TRUE.

This all seems to indicate you can only forward data once to another SIEGate but no more?
Is forwarding to a 3rd SIEGate possible, and if so is there any difference in setup after you go past 2? Ive basically been doing the same publisher/subscriber routine for each of the 2 pairs of SIEGates. That is the 1st publishes to the 2nd, and the 2nd publishes to the 3rd, the 3rd subscribes to the 2nd, and the 2nd subscribes to the 1st.

I tried this with OpenHistorian as well(latest stable build) and that also didn’t work, but also gave some errors, so I have gone back to SIEGate.

StephenCWills · August 22, 2019, 1:41pm

Hi Jeff,

There shouldn’t be any one-subscriber restriction in SIEGate. Rather, the whole point of SIEGate is to share with all your partners so the multiple-subscriber case is something we’ve tested extensively. It’s best to proceed under the assumption that what you’re attempting to do will work.

Based on the behavior you described, I feel it’s prudent to bring up what may seem like an obvious consideration. After you’ve set up the subscriber gateway to successfully connect to the publisher gateway, there are two things you need to do.

Configure the subscriber’s permissions on the publisher gateway. (What measurements are available to that subscriber?)
Configure the subscriber’s measurement subscriptions. (What measurements does the subscriber want to receive?)

These steps are not necessary for internal subscriptions because, by default, internal subscribers are implicitly trusted and automatically subscribe to everything. If you’ve been dealing with a lot of internal subscriptions, it’s easy to forget one or both of these steps.

Thanks,
Stephen

jeffhill · August 22, 2019, 10:30pm

Thanks Stephen

I did check that the Subscriber Measurement Access if filled in correctly, and that Measurements Subscriptions on the subscriber are all set.

I think the crux of my issue at least is that the devices on the last SIEGate do not appear when i pick Measurements Subscriptions so I am unable to tell the subscriber what to subscribe to. The publisher has given permission to this measurement. What made me think it was a limitation is I have measurements also coming from a OpenHistorian using the same publisher setup which DO appear at the last SIEGate. It’s devices also show up on the Measurements Subscriptions too, it’s only forwarded SIEGate measurements that aren’t being acknowledged.

Sorry I should have described the issue a little clearer. I have also set TLS!Datapublisher to be
securityMode=TLS; allowSynchronizedSubscription=True; useBaseTimeOffsets=true; AllowMetadataRefresh=True

I don’t really know what to try next short of merging the goose code with your latest stable release 1.7.4?

StephenCWills · August 23, 2019, 4:56pm

It sounds to me now like what you are describing may be a data flow configuration issue. This has to do with the Internal flag on your measurements, and it’s designed to help avoid internal feedback loops when routing data around your network. Looking at a simple data flow for an individual PMU, the internal flag is designed to work something like this.

       Internal    Internal                External    External
PMU >> openPDC  >> SIEGate  >> Internet >> SIEGate  >> openHistorian

In other words, for any given measurement routing device on you network, data that originated from a device on that network should be considered internal, and data that originated from a device on another network and entered via an external gateway should be considered external. This helps the system identify which data flows on the network are owned by you and which data flows are owned by others.

When setting up internal subscriptions, you may have seen the following diagram. It was made in order to guide users who don’t yet understand how to configure data flows. The guidelines described by this diagram help to get internal and external data flows going in the appropriate directions on your network, and also to ensure that the internal flags are applied to measurements appropriately throughout the network.

The reality is, all data flow settings can be overridden if you know what you’re doing, and these guidelines don’t work universally for every possible setup. The subscriber connection string defines three parameters:

internal=true/false determines whether the measurements received by the subscriber should be marked as internal.
receiveInternalMetadata=true/false determines whether the publisher should provide metadata to this subscriber for the publisher’s internal measurements.
receiveExternalMetadata=true/false determines whether the publisher should provide metadata to this subscriber for the publisher’s external measurements.

A TLS subscriber will use the defaults of internal=false; receiveInternalMetadata=true; receiveExternalMetadata=false. An internal subscription sets these flags based on your selections when setting up the subscription. You can manually tweak these flags to give you some more fine-tuned control over how data flows into, through, and out of your network.

NOTE: The internal flag on measurements is stored in the Measurement table of the database. These can be directly modified, but if the measurements are received via GEP, then I would recommend against it because the subscriber’s metadata synchronization process will overwrite the flag with the value in the subscriber’s connection string.

Thanks,
Stephen