Exporting openHistorian data to Python

GPA Products openHistorian
1

Is there any way to extract open Historian data into Python for better analytics using openHistorian.db instance?

2
3

Thanks Ritchie for providing the openHistorian Python API. Just a quick question do we need to configure snapDB separately as when I am running the code to connect to my localhost server where openHistorian is hosted I am getting this error : -

Connecting to openHistorian…
Failed to connect: Unexpected server response: 72

which is actually an exception thrown after this execution:

def readTest():

# Create historian connection (the root API object)

historian = historianConnection("localhost:8182")

instance: Optional[historianInstance] = None

try:

    print("Connecting to openHistorian...")

    historian.Connect()
4

Try leaving the port number out:

historian = historianConnection("localhost")

The openHistorian API, by default, listens on 38402 not 8182.

Thanks,
Ritchie

5

Hi Rithcie,
Thanks for your response. I implemented this too by opening the port 38402 on my system too but then too I am receiving this message:

Connecting to openHistorian…
Failed to connect: [WinError 10061] No connection could be made because the target machine actively refused it

6

Check the following:
(1) openHistorian service is running
(2) Windows firewall is turned off (or allow openHistorian.exe or ports as an exception)

7

Does the openHistorian Python API allow us to read statistics (STAT) values such as average latency?

I’ve tried running “readTest.py” and was able to get PPA type values, but was unable to read STAT values. Here is the code I modified for this:

records = metadata.GetMeasurementsBySignalType(SignalType.STAT, instance.Name)

I currently have 2 Historian instances (PPA, STAT) under “Manage Historian Instances”, both of which are local.

image
image917Ă—550 65.7 KB

I noticed that in the “readTest.py” code when I try to change the index for initialInstance = historian.InstanceNames[0] to anything greater than 0, I receive the exception “Failed to connect: list index out of range”.

8

As far as I know we can only extract the PPA historian instance. Secondly STAT is not a signal type and hence it would not return any value.

9

No, statistics are archived in a separate local repository (e.g., C:\Program Files\openHistorian\Statistics) - however, there is a REST-based web-service for getting these values which should be easy to query from python (see below). FYI - examples are small subset of functions available, see IMetadataService.cs and ITimeSeriesDataService.cs for full REST method listing.

For metadata use: http://localhost:6055/ - examples:

  • Read all metadata in XML format:
    http://localhost:6055/historian/metadata/read/xml

  • Read all metadata in JSON format:
    http://localhost:6055/historian/metadata/read/json

  • Read metadata points from 10 to 100 in XML format:
    http://localhost:6055/historian/metadata/read/10-100/xml

  • Read metadata points from 10 to 100 in JSON format:
    http://localhost:6055/historian/metadata/read/10-100/json

  • Read metadata points 10, 11 and 20 in XML format:
    http://localhost:6055/historian/metadata/read/10;11;20/xml

  • Read metadata points 10, 11 and 20 in JSON format:
    http://localhost:6055/historian/metadata/read/10;11;20/json

For data use: http://localhost:6056/ - examples:

  • Read current value for data points from 10 to 100 in XML format:
    http://localhost:6056/historian/timeseriesdata/read/current/10-100/xml

  • Read current value for data points from 10 to 100 in JSON format:
    http://localhost:6056/historian/timeseriesdata/read/current/10-100/json

  • Read current value for data points 10, 11, and 20 in XML format:
    http://localhost:6056/historian/timeseriesdata/read/current/10;11;20/xml

  • Read current value for data points 10, 11, and 20 in JSON format:
    http://localhost:6056/historian/timeseriesdata/read/current/10;11;20/json

  • Read archived values for data points from 10 to 100 and specified start/end times in XML format:
    http://localhost:6056/historian/timeseriesdata/read/historic/10-100/2021-04-21%2023:50:00/2021-04-21%2023:51:00/xml

  • Read archived values for data points from 10 to 100 and specified start/end times in JSON format:
    http://localhost:6056/historian/timeseriesdata/read/historic/10-100/2021-04-21%2023:50:00/2021-04-21%2023:51:00/json

  • Read archived values for data points 10, 11, and 20 for specified start/end times in XML format:
    http://localhost:6056/historian/timeseriesdata/read/historic/10;11;20/2021-04-21%2023:50:00/2021-04-21%2023:51:00/xml

  • Read archived values for data points 10, 11, and 20 for specified start/end times in JSON format:
    http://localhost:6056/historian/timeseriesdata/read/historic/10;11;20/2021-04-21%2023:50:00/2021-04-21%2023:51:00/json

1 Like
10

FYI - some history:

The STAT archive is an instance of the older openHistorian 1.0 that was deployed with the original openPDC (see GSF.Historian and LocalOutputAdapter) - fast writes, not as fast reads.

The PPA archive is an instance of the newer openHistorian 2.0 based on SNAPdb (see GSF.SortedTreeStore, openHistorian.Core and LocalOutputAdapter) - fast writes and fast reads.

Here was a good presentation explaining some of technical differences:

11

Thank you Ritchie, the information was very much helpful. I am able to get the data that I was looking for now.

12

Hello! In order to avoid opening another trend of this topic, I will write in here:

Can you tell me what is the compatible versions of openHistorian releases that works with python code available here: openhistorian · PyPI (openhistorian 1.0.5)

We were able to successfully download data based on OH v2.8.395, but on the stable release (v2.8.383) I received the message

image
image868Ă—72 19.3 KB

Is there any special configuration were made in OH side? Both OHs are in the same LAN with no firewall restrictions.

Thanks!

13

No specific version requirements for Python API, looks like you are seeing a connection issue. Could be related to firewall, non-standard port configuration or incorrect IP/hostname for OH instance. Also make sure OH service is running

Thanks!
Ritchie

1 Like
14

Understood, thanks for the guidance. But still I don’t see this failure point yet… While I’ve verified the IP addresses are correct, there seems to be an issue with the connection itself. Here’s what I’ve investigated so far:

  • Configuration: I’ve double-checked the configurations of both HIS1 and HIS2, and they appear identical. However, it’s possible there might be a subtle difference causing the issue.
    Logs: I haven’t found any relevant error messages related to the connection failure in the HIS logs.
  • Python Code: The error originates from the sttpSocket line in the Python code.
    image
    image710Ă—174 80.6 KB

To assist in debugging further:

  • Port Verification: How can we confirm if port 7175 is indeed the standard port used for HIS communication?
  • Configuration Comparison: how to perform a more meticulous side-by-side comparison of the HIS1 and HIS2 configurations to identify any potential discrepancies, even minor ones?

The connection attempt occurs within the same sub-network, so firewall rules are unlikely to be the issue, but I will double check this again.

Thanks!

15

You can verify that the openHistorian STTP port is on 7175 by running the “XML Configuration Editor” (from the openHIstorian application group on the Windows start menu). Expand the sttpdatapublisher section and check the ConfigurationString setting:

image
image594Ă—598 16.1 KB

You can also try re-initializing the STTP Publisher adapter. Open the manager application for the openHistorian and navigate to “Actons > Manage Custom Actions”. Next, select the STTP!DATAPUBLISHER adapter and click the “Initiaize” button.

Optionally, you can try restarting the openHistorian service - or - as a last resort, trying restarting the host machine.

There is always the possibility that another application has the port locked. Check the Task Manager to verify there is only one openHistorian executable instance running (technically, only one is allowed, but it is possible a rogue instance is still running).

To verify which application has port 7175 in use, run this command from a command prompt that has administrator permissions:

netstat -abnp tcp
1 Like
16

Indeed the port was different in HIS1 installation (stable build), using port 7177. HIS2 is the nightly build, using port 7175. While this worked at the the attempt (yey!) it started generated another error claiming that “an existing connection was forcibly closed”.

image
image790Ă—239 103 KB

HIS1 netstat result:
image

HIS1 is in production and reseting it should be avoided.

Any next tip to progress towards the resolution?

17

This is commonly associated with a TLS connection issue. Port 7177, by default, is associated with the TLS instance of STTP, or “STTPS” found in the sttpsdatapublisher section in the config file, not the sttpdatapublisher. This is a subtle, but important, difference. Can you verify the port number for the standard, non-TLS configuration section that should be labeled without the “s” suffix on STTP, i.e., sttpdatapublisher:

image
image594Ă—598 20.1 KB

Additionally, corresponding to the sttpdatapublisher, there should be a custom action called STTP!DATAPUBLISHER - can you let me know its configuration string from the manager application? Can you verify the record is enabled?

Thanks!
Ritchie

18

Good news! Now it is sorted. Indeed I made a mistake looking at the wrong field but also HIS1 has a different port than HIS2. Now both are working. Thank you for the support here!