Metadata Endpoint Limiting Results

GPA Products openHistorian
1

We have an application that relies on the response from the metadata endpoint, as shown in the following example request:

localhost:6155/historian/metadata/read/json

However, the response appears to be limited.

Note 1: We have 20,581 measurements registered in the OH2 under the Metadata > Measurements tab, but only the first 422 are being returned when calling the 6155 port.
Note 2: We are observing the same behavior with openPDC (port 6151) on this server.

I would like to receive all measurements, if possible, when using localhost:6155/historian/metadata/read/json.
Does anyone have any ideas regarding the cause of this limitation?

Thank you in advance for your support.

2

Hello Renan_Pires,

I have a feeling this is due to a gap in the historian metadata IDs. Likely, you do not have a measurement configured in your system with ID 423, and this is causing the metadata query to stop at that point, thinking it’s reached the end of the archive.

This throws a bit of a wrench in my theory, though. Where is the openPDC’s metadata coming from? Is it hosting its own openHistorian 1.0 archive? The openHistorian 1.0 archive is supposed to insert blank records into the data set wherever there is a gap in historian IDs.

Thanks,
Stephen

3

Hello StephenCWills,

Thank you for your quick response!

There is indeed a gap in the historian metadata IDs. We don’t have a measurement configured with ID 423.

What additional information can I provide to help identify the cause of this gap and assist in finding a solution?

Best regards,
Renan Pires

4

The issue occurs due to a mismatch between the file formats used by openHistorian 1.0 and the scalability improvements we made in openHistorian 2.0. The legacy code returns blank metadata records whereas the new code returns null. The code basically looks like this…

int historianID = 0;
while (true)
{
    Metadata metadata = Archive.ReadMetadata(++historianID);
    if (metadata == null)
        break;
    metadataRecords.Add(metadata);
}

(The actual code: MetadataService.cs#L183-L196)

As you can see, it’s reading metadata records sequentially from zero, and at the first gap it encounters it’s going to bail. Ideally, we would just read all the metadata that exists instead of blindly reading sequentially and filling in gaps with blank records.

If you’re proficient with C#, you can try to code up a fix and submit a pull request. Otherwise, you’ll just have to wait for us to come up with something ourselves.

5

I would like to issue a small correction to my previous message.

In Note 2, I mentioned that we were observing the same behavior with openPDC (port 6151) on this server. This was a mistake.

The correct statement is:
Note 2: It’s not possible to test the historian/metadata response from openPDC on this server because openPDC is configured not to store historian data. It directly forwards the received STTP stream to openHistorian 2.

I apologize for any confusion caused, and thank you again for your assistance!

Best regards,
Renan Pires

6

Hi, Stephen.

I have another question about the metadata response.

I’m retrieving metadata from OpenHistorian 2 using the following URL:

http://server_ip:6155/historian/metadata/read/17321/xml

In the XML response, I noticed that the <SystemName> field appears as:

<SystemName>SE_EST_MI!PAXNGB_500_PM_</SystemName>

However, I was expecting the full name to be:

image
image1851Ă—461 184 KB

image
image1863Ă—540 239 KB

SE_EST_MI!PAXNGB_500_PM_01

Is there any specific reason why <SystemName> might not include the full name? Could it be related to a character limit or formatting rule?

Thanks,
Renan Pires

7

That particular older web interface is related to the 1.0 version of the historian whose serialized metadata file has some field size limits that the openHistorian 2.0 does not.

Thanks,
Ritchie

8

Hi Ritchie,

Thanks for the clarification!

Is there a recommended endpoint (or other way) in OpenHistorian 2.0 that would return that full metadata field?

Thanks again for your help!

Best regards,
Renan Pires

9

Could try the one used for Grafana support. This API call is a “POST” operation:

http://localhost:8180/api/grafana/Search

You would need to send a JSON request similar to the following:

{
  "dataTypeIndex": -1,
  "expression": "SELECT * FROM ActiveMeasurements WHERE True ORDER BY PointTag"
}

Note that this is a SQL adjacent expression, i.e., it is working on a read-only in-memory data table, not directly accessing database - so available SQL query syntax is limited – but DISTINCT and TOP n expressions are supported.

FYI - this example is for the latest nightly build of openHistorian. Earlier versions will support something slightly different.

There are other options as well, other APIs, plus you could use STTP get metadata…

Thanks,
Ritchie

1 Like