What is Data Publisher? and What is Subscriber?

#1

Hello All,

I need some clarification over the some functionalities used in openHistorian. There are some doubts i have which i want to clear.

1.)
What is Data Publisher?? and how it works ??
When i have Configured Subscription to Internal Gateway Publisher, then what i am configuring is a Data Publisher Configuration or a Subscriber Configuration?? As in Sample Test code of Data Publisher and Data Subscriber what i have understood that from Data Publisher You transmit Data and from Data Subscriber connect to Same channel and you receives the Data.
So my query is if i am configuring Internal Subscription to Internal Gateway Publisher it was saved into Device table. So it treat this configuration as Device which is communicating on GEP and through which we can received the Data but where is my Subscriber.
Also need some explanation and significance of All the fields are used in Manage Device Configuration
for better understanding.
… What is Concentrator whether we need to configure it as a Concentrator??

2.)
Or Subscriber is a different functionality which is used to fetched the Data from openHistorian Server.???Then can you brief about how Subscriber works??

3.) While configuring Subscription to Internal Gateway Publisher it is what are the use of option
Receive Internal Metadata and Receive External Metadata. As i had checked both option?? So if i got some clarity then i can configure it properly.

4.) While configuring Measurement for can i have brief explanation of all fields specially Adder, Multiplier, options Internal and Subscribed and Measurement Type.
For my sample application I have created Measurements with Check option Internal, Subscribed and enabled. Own Point Tag, Signal Reference and Measurement Type = Statistics.
Measurement Type has multiple Values. So what is the significance of Measurement Type for the Measurements for which data come from another application. Whether it is used while storing Data?? Need to understand Measurement Type.

I know there are many queries ?? But if i have clarity on these stuff then i can build proper application

Thanks in Advance

#2

Hello ankitshah-it,

I’ll do my best to clarify these points.

  1. Fundamentally, nearly every piece of functionality in the openHistorian is either an input adapter, action adapter, or output adapter. Every device in the Device table is converted into an input adapter, the type of which is determined by the Protocol field. You can see the mappings between protocols and their respective code modules by looking at the Protocol table. Specifically, the DataSubscriber class itself is the input adapter for a GEP data subscriber.

    A data concentrator is an action adapter that acts like it’s an output adapter. The type of a concentrator’s output adapter is one of the implementations of PhasorDataConcentratorBase. Concentrators are specifically designed to publish data via one of the frame-based synchrophasor protocols. The only protocols that support concentration of data from multiple devices are IEEE C37.118, BPA PDCstream, and IEC 61850-90-5.

    GEP is not a frame-based protocol so it does not require a concentrator setup in order to publish data. The out-of-the-box configuration for openHistorian includes three data publisher adapters in the CustomActionAdapter table, and these are the GEP data publisher instances used for publication.

  2. No, subscriber is a GEP Data Subscriber just like the one used in DataSubscriberTest.

  3. Take a look at the following explanation to better understand Receive Internal Metadata and receive External Metadata.
    Forwarding measurements limitation?

  4. Adder and Multiplier define the linear adjustments used when determining the AdjustedValue of the measurement: AdjustedValue = Value * Multiplier + Adder.

    Internal defines whether the measurement should be considered internal or external to the openHistorian’s network (see the explanation I linked in #3).

    Subscribed indicates whether the measurement is coming from a GEP subscription or not.

    Measurement Type should be pretty self-explanatory. I’m not sure what else I can add, except that the measurement types defined in the system were largely inspired by the frame-based synchrophasor protocols. Others were added later as needed. The valid values for Measurement Type can be found in the SignalType table in the database.

Thanks,
Stephen

1 Like
#3

Thankyou Stephen

(1) One issue i am facing right now. That from My Application (Data Publisher) when i am writing a Particular Value( 989873.365478564) to the specific Measurement. But when it was received by openHistorian Server the value is changed to 989873.375.
Signal Type (Measurement Type) is STAT

My query is that why it is changed to this and how the calculation applies.??
I want to store the value as it is in historian and want to fetch as it is ?? Is there any configuration required to make change?? Please do the needful in this matter as i have many devices which produces 0 1 (status), 98989898.11213121399 double, long and integer values.

(2) What is the Limitation of Data size and Type of Data to store into openHistorian??

(3) Also for Measurement Object I have found that AdujstedValue, GSF.Timeseries.ITimeSeriesValue.Value, Value and m_value and in which the 3rd value is little different then others. What are they and When storing into openHistorian which value is stored and How to retrieve it.

(4) And I need use of each and every fields of Device and Measurement for better Understanding, if there is any Document then Please Provide the Link of it so I can understand significance of each field of Device and Measurement Table.

Thanks in Advance
Ankit Shah

#4

  1. The issue is that the protocol converts your original 64-bit floating point value into a 32-bit floating point value for transmission. The same thing happens when I run this in PowerShell: [System.Convert]::ToDouble([System.Convert]::ToSingle(989873.365478564)). Basically, GEP was designed to compete with existing synchrophasor protocols in terms of bandwidth and performance, and it didn’t make a lot of sense to transmit a full 64-bit floating point value when all the existing protocols were already passing around 32-bit floating point values already anyway. Currently, all values are transmitted as 32-bit floating point numbers and there is no way to change the data type used for transmission when using GEP.

  2. I’m not incredibly familiar with the true limitations of the underlying SnapDB archive used by the openHistorian. What I do know is that the HistorianKey and HistorianValue classes each define three 64-bit long integer values for a total of 24 bytes each, and these are the data types used when setting up the SnapDB archive in the openHistorian’s LocalOutputAdapter. Also, I just noticed the LocalOutputAdapter explicitly forces the value into a 32-bit floating point number before fitting it into HistorianValue.Value1.

  3. The Value property simply exposes the value stored in the m_value field. This is a common pattern in C# to have a public property expose the value of a private member variable. I’ve already explained what the AdjustedValue property is. If I recall correctly, when synchronizing metadata via GEP, the publisher does not transmit the values for Multiplier and Adder to the subscriber and simply transmits the AdjustedValue in the data stream. If you apply an Adder or Multiplier to the signal on the subscriber side, openHistorian will store the AdjustedValue.

  4. Unfortunately, I don’t think there is any such document, and the meaning of various fields in the database often depends entirely on context. For instance, the FramesPerSecond field in Device is used differently depending on the Protocol of the device, and also has relevance in context of the reporting processes.

1 Like
#5

Thankyou Very Much @StephenCWills.

As you have explained Point No. 1 that it is converting 32-bit floating point values while using GEP. Then is there any other way to store 64-bit floating point values into openHistorian???
Because my application will have different devices’ different data type values, so that it has to go with 64-bit floating point values.

Is there any plans for enhancements in openHistorian which can store 64-bit floating point values etc…??? So i can use that functionality for my application???

Thanks in Advance
Ankit Shah

#6

I think if you bypass GEP and the measurement routing system and instead use the openHistorian socket API to send HistorianValue objects directly into the archive, you can use System.BitConverter.DoubleToInt64Bits() to save the full-resolution 64-bit floating point value as a long integer. The only problem here is that the value wouldn’t show up properly in any of the displays because every one of them assumes the value was stored as a 32-bit floating point number.

Here’s the code used in LocalOutputAdapter that converts the measurement into a HistorianValue.

// Since current time-series measurements are basically all floats - values fit into first value,
// this will change as value types for time-series framework expands
m_value.Value1 = BitConvert.ToUInt64((float)measurement.AdjustedValue);
m_value.Value3 = (ulong)measurement.StateFlags;

Given that the LocalOutputAdapter is not utilizing Value2, you could get the best of both worlds if you utilize use that to store the full resolution value.

m_value.Value1 = BitConvert.ToUInt64((float)measurement.AdjustedValue);
m_value.Value2 = (ulong)System.BitConverter.DoubleToInt64Bytes(measurement.AdjustedValue);
m_value.Value3 = (ulong)measurement.StateFlags;

Note that any option you choose here will likely adversely affect the archive’s compression and cause the archive to be larger than what would typically be expected.

As for whether there are plans for future enhancements to support other data types, the answer is yes. However, as of yet there is nothing concrete, and therefore also no timeline. We may get to it soon, and we may get to it never.

1 Like
#7

@StephenCwills Thank you very much for your valuable guidance.

I will Develop Application according to it and will let you query if i have doubt regarding this

Thanks
Ankit Shah