openHistorian: old archive data has been overwritten by new data

GPA Products openHistorian
1

Hi,

We have an old archive data, and it has been over written or merged by new data.
How is it possible? Any settings changed?

E.g.
We had an archive data for the month 2020-Sep at the folder structure c:\archive\2020\09*.d2 files.
Now it has been overwritten/merged by new data which are created for the month 02-2021.

E.g. File name:

ppa-2020-09-25 19.06.04.065_to_2021-02-19 03.46.15.033-637493031784135023.d2

But the filename date looks like between 2020-09 --2021-02.

What does it mean? whether this file contains data for both months?

If I try to fetch those data using Grafana, then no data is coming for month 09-2020.

Please help on this. It’s very critical.

Regards
Logu

2

That just means that a device or import adapter pulled in data from 2020 at the same time as 2021, most likely this is due to a bad timestamp (GPS offline for example). So this file contains data from both 2020 and 2021 - nothing is overwritten. Data from 2020 in the file with both years will be read along with existing 2020 data.

Note that you can throw out data that is “old” by enabling the EnableTimeReasonabilityCheck=True in the openHistorian connection string. Related settings are PastTimeReasonabilityLimit and FutureTimeReasonabilityLimit in seconds that can be adjusted to determine what is “reasonable”.

Ritchie

3

Hi Ritchie,

Thanks for your response.
Actually this file contains the data for month Sep-2020 . But I could not read that data in Grafana portal by specifying date range or month as well.

Do we have any option to read that file content, without time stamp? so that I can understand the contents of that file.

Logu

4

I suggest looking at the “ArchiveTools” application in the openHistorian source code, e.g.:

This tool opens individual .d2 files for analysis. In the code you can see it check the table.FirstKey for example. This is the first HistorianKey in the file that contains the ID, Value and Timestamp.

You can use the table.BeginRead() function to iterate over all values in the file, e.g:

using (SortedTreeFile file = SortedTreeFile.OpenFile(fileName, true))
{
    SortedTreeTable<HistorianKey, HistorianValue> table = file.OpenTable<HistorianKey, HistorianValue>();
    HistorianKey key = new HistorianKey();
    HistorianValue value = new HistorianValue();

    using (var reader = table.BeginRead())
    using (var scanner = reader.GetTreeScanner())
    {
        while (scanner.Read(key, value))
        {
            Console.WriteLine($"ID: {key.PointID}, Timestamp: {key.TimestampAsDate:yyyy-MM-dd HH:mm:ss.ffffff}, Value: {value.AsSingle}");
        }
    }
}
5

Hi Ritchie,

As you suggested, I’m trying to read the archive file which has the name “ppa-2020-09-25 19.06.04.065_to_2021-02-19 03.46.15.033-637493031784135023.d2”.

I’m able to see the meta data from this file.

…\ppa-2020-09-25 19.06.04.065_to_2021-02-19 03.46.15.033-637493031784135023.d2
ID: 9aebe6e9-7ab7-441a-98df-da89faf4a628 Commit Number: 2 Start Time: 25/09/2020 7:06:04 PM End Time: 19/02/2021 3:46:15 AM

But, when reading the file using scanner.Read(key, value), got the below exception.

Message = “This node is not supposed to access the underlying node level.”

Could you please clarify, what could the wrong?

Thanks & Regards
Logu

6

Can you send more code - I will make sure everything looks OK.