It's not possible to change the statArchiveFile.FileSize

mnagostini · January 30, 2019, 12:14am

Hi,

I’m trying to change the size of the statistics historian files (“stat_archive_…d”). I stop the openPDC service, change and save de configuration file “C:\Program Files\openPDC\openPDC.exe.Config”, session “”, parameter “FileSize”, but when I restart the openPDC service the parameter returns to its old value. I have tried to change this parameter with “XML Configuration Editor” too, but the result is the same.

I’m using the version 2.6 (release 2.6.13), but I could see the same behavior in the version 2.4.

Is there some trick for that? My main goal is to adjust the size and number of the statistics files to have an equivalent number of files in the statistics and archive (measurements) historians.

Thanks!

StephenCWills · January 30, 2019, 3:11pm

There is a data operation that, by default, executes whenever the system does a ReloadConfig called OptimizeLocalHistorianSettings. This data operation is responsible for a number of different tasks to tweak those configuration file settings automatically. As you can see from the following link, the system is automatically updating those settings to force a DataBlockSize of 1 KB and a FileSize based on the number of statistics being archived.

github.com

GridProtectionAlliance/gsf/blob/master/Source/Libraries/Adapters/HistorianAdapters/LocalOutputAdapter.cs#L999




// Apply settings optimizations to local historians
foreach (DataRow row in historians)
{
    acronym = row.Field<string>("AdapterName").ToLower();
    name = $"local \'{acronym}\' historian";


    // We handle the statistics historian as a special case
    if (acronym == "stat")
    {
        // File size of the statistics historian is tuned based on the number of statistics being archived (two data blocks per statistic per archive file).
        // This number is rounded up to the nearest MB to prevent rounding errors when serializing the settings to the configuration file.
        // The data block size is set to its minimum of 1 KB which works better for archives with a low sample rate and a large number of signals.
        const int DataBlocksPerSignal = 2;
        int statisticsCount = Convert.ToInt32(database.Connection.ExecuteScalar("SELECT COUNT(*) FROM ActiveMeasurement WHERE SignalType = 'STAT'"));
        int fileSize = (int)Math.Max(1.0D, Math.Ceiling(statisticsCount * DataBlocksPerSignal / 1024.0D));


        // Set rollover preparation such that the historian will not be creating a standby file and rolling over simulataneously.
        // Common.Mid is used to ensure that the setting is between 1 and 95, which are the upper and lower bounds of the rollover preparation threshold.
        // The actual number of data blocks per signal is calculated because it may be different since the file size was rounded up to the nearest MB.
        double actualDataBlocksPerSignal = fileSize * 1024.0D / statisticsCount;

If you’re not careful when adjusting these settings, you can create situations where the archive is constantly rolling over, which leads to extreme disk and CPU usage, and it makes the archive unusable. This code helps to ensure that the size of the stat archive files grows appropriately as the system scales up.

If you still feel the need to modify the file size manually, you can disable the data operation by removing it from the DataOperation table in the configuration database. Alternatively, you can change the OptimizeLocalHistorianSettings parameter in the configuration file to False. Note that this will also disable the other optimization tasks handled by this DataOperation as well, which includes additional configuration file adjustments for non-stat archives and the automatic creation of an archive’s corresponding input adapter to support temporal subscriptions.

mnagostini · January 31, 2019, 11:12am

Nice, Stephen. Understood.

Thanks.