From any Outside Application can i Save data to openHistorian

#1

Hello All,

I have one query that I have an application which continuous generates some Data which need to be stored in Historian (In File, RDBMS etc…). How can i stored these generated data into Historian. What are the possible ways??

Thanks in Advance

#2

It’s hard to provide recommendations given so little information, but here’s a handful of solutions off the top of my head.

  1. If the application supports any of the real-time streaming protocols supported by openHistorian, simply configure an existing input adapter.
  2. If the application supports its own real-time streaming protocol, write a custom input adapter to decode that real-time stream and pass measurements into the openHistorian routing engine.
  3. If the application supports writing real-time streaming data to an RDBMS, write a custom input adapter to load batches of records from the RDBMS, convert them into measurements, and pass them into the openHistorian routing engine.
  4. Modify the application with DataPublisher code from GSF so that it supports publishing data to openHistorian via GEP.
  5. Modify the application with code from the openHistorian socket interface to push data directly into the openHistorian archive.
  6. Modify the application with code from the historian API so that it can write its own archive files directly.
#3

Sorry for providing Very Little Information.

But as you have given the options I think No. 4 and No. 2 is suitable for the Application and I have chosen option 4 and modified application and added DataPublisher Code (From DataPublisherTest Code) but how can we use GEP (Gateway Exchange Protocol) to insert the records into RDBMS ??

I have created One Custom Output Adapter which is storing Data into RDBMS directly Now I want that the Data from Application also should be inserted into RDBMS

And among 2 and 4, Which Option is better in terms of Performance ??

Thanks In Advance
Ankit Shah

#4

I’m guessing the measurements don’t make it to the custom output adapter because the openHistorian doesn’t have any metadata associated with the data points entering the system via GEP. Your simplest option would be to manually define the metadata in openHistorian and then use matching signal IDs when generating measurement keys for the data publisher.

Alternatively, you can attempt to go through the steps to automate the exchange of metadata between the application and openHistorian. I suspect that the process of getting that working would be prohibitively complicated, but here’s a rough outline of what you’d need to do:

  1. Organize the metadata in your application into devices and measurements.
  2. Assign GUIDs to each of your application’s devices and measurements that will not change. This needs to be a static metadata field associated with these objects. openHistorian will need to use these to match remote objects to local objects when synchronizing configuration.
  3. Create a new class which is a subclass of DataPublisher. In this class, override the AquireMetadata function to provide a DataSet with the configuration from your application. Try to match the schema of the resultset defined by the DefaultMetadataTables expression, which is defined in DataPublisher.
  4. In your application, use an instance of your DataPublisher subclass instead of using the DataPublisher class directly.
  5. Check for error messages in the openHistorian console to troubleshoot issues with the metadata synchronization.

Here’s the definition for the DefaultMetadataTables expression mentioned in step 4.

public const string DefaultMetadataTables =
    "SELECT NodeID, UniqueID, OriginalSource, IsConcentrator, Acronym, Name, AccessID, ParentAcronym, ProtocolName, FramesPerSecond, CompanyAcronym, VendorAcronym, VendorDeviceName, Longitude, Latitude, InterconnectionName, ContactList, Enabled, UpdatedOn FROM DeviceDetail WHERE IsConcentrator = 0;" +
    "SELECT DeviceAcronym, ID, SignalID, PointTag, SignalReference, SignalAcronym, PhasorSourceIndex, Description, Internal, Enabled, UpdatedOn FROM MeasurementDetail;" +
    "SELECT ID, DeviceAcronym, Label, Type, Phase, DestinationPhasorID, SourceIndex, BaseKV, UpdatedOn FROM PhasorDetail;" +
    "SELECT VersionNumber FROM SchemaVersion";

As for your question about performance, it really depends on the characteristics of the data and the protocols supported by your application. GEP was designed to be a very low-latency, high throughput, pub/sub streaming protocol, and stress testing indicates that it scales very well. We consider GEP to be a very widely applicable solution in terms of performance. However, your application’s protocol may be better suited to your data or the performance requirements of what you’re trying to accomplish.

Both solutions will use an input adapter to receive measurements and pass them to the routing engine, so there is no difference in terms of what the openHistorian has to do once the measurements arrive in the system.

#5

Thank you Thank you very much Stephen for your valuable guidance.

I have made a Class which instantiate DataPublisher.

  1. Created measurements into measurement table with Protocol GEP
  2. Created Subscriber
  3. Now through My Class which instatiate DataPublisher added first measurements and sent them for further processing through procedure “QueueMeasurementsForProcessing”.
  4. While openHistorian Service Started subscriber connected with mine DataPublisher and receives the Data (Signals).

And openHistorian engine is storing the specified published Data.

Now my query is that how can i retrieve those Data

  1. All
  2. With specified signal ids and within Timestamp Range
  3. Data from some Timestamp to till now
  4. Is it compulsory to connect to openHistorian Service for Data retrieving. i.e. if Service is stopped then can we retrieve Data or not. if yes then how???

One thing also i have noticed that my Timezone is set for India but the Timestamp of Data stored is 5:30 hours behind the current Timestamp. So i think need to configure Timestamp as per the Timezone. From where and how i can change it so i can get realtime Timestamp as per the Timezone Set.

Thanks in Advance

#6

The way you access data will depend most on how you plan to use the data. Here’s a relatively full list of options off the top of my head.

  • To simply visualize the data on graphs and widgets…
    • Grafana dashboard visualization
    • Trend / Export page in the web UI
  • To export data into CSV or COMTRADE data sets…
    • Trend / Export page in the web UI
  • To access the data within a .NET program…
    • The Gateway Exchange Protocol
    • The openHistorian socket API
    • The HistorianQuery web API
  • To access the data programmatically on another platform…
    • The Gateway Exchange Protocol
    • The HistorianQuery web API

As for the timezone, we don’t actually support storing the data in any other timezone besides UTC. There are a handful of good reasons for this, most of which may actually be irrelevant in your case. Regardless, if you must store and/or display data in your local time zone, then you will need to come up with your own solution.

Thanks,
Stephen

#7

Thanks @StephenCWills

#8

Hello StephenCWills Can you explain how can i Access Data within .NET Program or on another platform through the methods as you have mentioned.

  1. The Gateway Exchange Protocol
    2.The openHistorian socket API
    3.The HistorianQuery web API
    So it helps me to create application for data retrieval and which way is best suit for my application.

Thanks in Advance
Ankit Shah

#9

For the Gateway Exchange Protocol, refer to the following code sample.
https://github.com/GridProtectionAlliance/gsf/blob/master/Source/Applications/DataSubscriberTest/Program.cs

For the openHistorian socket API, I recommend this code sample because it’s fairly simple.
https://github.com/GridProtectionAlliance/openXDA/blob/master/Source/Libraries/openHistorian.XDALink/Historian.cs

For the HistorianQuery web API, the only code sample I know of is in JavaScript.
https://github.com/GridProtectionAlliance/openHistorian/blob/master/Source/Applications/openHistorian/openHistorian/wwwroot/TrendMeasurements.cshtml#L1287