Option to upload raw Synchrophasor data from Phasor Measurement Unit

Is there any option available (other than comtrade) to import/upload raw synchrophasor data from the phasor measurement unit?.

Hi,
I have captured the xx.pmuCapture file from PMU connection tester and tried to create a new device in OpenHistorian selecting xx.PmuCapture file. But i am not able to see all the phasor data except status and freq. Could you please advice if i am missing any settings. What is the supported protocol used in .PmuCapture file. Is it support ICE 61850_90_5 standard.

Hi,
My PMU sends data to connection tester as per IEC 61850 90_5 standard. I captured the .PmuCapture and tried to see the phasor data in OpenHistorian. But i am not able to make it successful in openHistorian.
But i could able to replay it in connection tester itself.
Could you please guide me how to use the data from .pmuCapture file in OpenHistorian.

Hi Ritchi and Stephen,
I could able to create a device with configuration and pmu capture captured from connection tester successfully. i could able to see the changes in phasor values. But though the time stamp shows the older time, it is still shown as real time. Is there any way to see the phasor data instead of playing the pmu capture file?.
Also how can we decode pmu capture file to understand how data is encoded?. Please guide me.

Hi amalorpavam,

The PMU capture files are created by literally taking bytes from the network protocol and writing them to a file. These files have the same format as the network stream they were captured from. This makes them ideal for replaying a real-time data stream, but perhaps unwieldy as a file format. Their primary purpose is for testing and troubleshooting.

Databases and historians are designed for long-term storage and queries. They have a fundamentally different format, optimized for this different purpose. openHistorian will not be able to read the PMU capture files except to replay the network stream as an input adapter. Therefore, the capture data can be streamed into the archive, but it cannot be queried directly.

Thanks,
Stephen

Thanks Stephen.
In this case, though i feed .pmucapture file to open Historian, i can only replay them but will not be able to query to get any useful data from it. Is my understanding correct.

I am actually trying to take backup of pmu output when the connection between PMU and PDC is lost. Later i will have to feed the phasor backup to openHistorian to understand what had happened when the connection is lost. If .pmuCapture is not helping me to feed the backup data to openhistorian, will back phasor data in comtrade format will help me?. Please suggest me what could be be the useful format to feed the backup of phasor data to Openhistorian.

Regards.

Hi Stephen,
I have tried few things today in open historian with the data captured (.PmuCapture file) using connection tester. I am able to set the alarm on phasor data. Also i am able to query on phasor data in Grafana Dashboard ex: maximum, add etc.

I am new to open historian. Could you please explain me what exactly cannot be achieved in OpenHistorian when we use the data stream in file with some real time example to help me understand the issue.

Regards.

Hi amalorpavam,

If what you need is a secondary archive, I recommend setting up a short-term historian archive on the PDC’s local network. Use GEP/STTP to move the data between the PDC and openHistorian, and then make sure you configure data gap recovery in the openHistorian’s subscriber. If you are not using openPDC as your PDC, you can deploy substationSBG to the PDC’s local network for exactly this purpose.

See: https://www.youtube.com/watch?v=wcUWMv2iCyk


As you said, the PMU capture files are only really useful for replaying streaming data. On top of that, there are a few limitations that make the format rather unwieldy.

  1. Existing processes for capturing and replaying PMU capture data require manual intervention. An automated processes would need to be developed for capturing. Replay likely could remain as a manual process, but the process should be refined for the sake of usability.
  2. Some of the typical features of a proper archival process would need to be reproduced in order for this strategy to be successful. This includes automatically partitioning the data into separate PMU capture files, timestamping the partitions, and automatically truncating the archive.
  3. All the frame-based synchrophasor protocols use configuration frames to define the metadata that is required by a decoder to properly interpret the data frames. In order for any given PMU capture file to work, the configuration frame must be captured with the data. The automated capture process would need to identify configuration frames, hold them in memory, and write them into each PMU capture file. The decoder typically has to read ahead to find the configuration file before going back to the start of the file and reading the data frames.

Thanks,
Stephen

Hi Stephen,

Actually my problem is that there is a PMU and it is constantly producing phasor data with some sampling rate defined. But No PDC connected to it.

I need to do the following two steps.

  1. Capture the PMU output (both configuration and data frames) and store it in a file continuously for few hours.
  2. Later Give the stored file (in step 1) to OpenHistorian to see the phasor details/its waveforms.

Kindly suggest me a way to achieve this.

Regards,
Amala

Hi Amala,

If you are connecting an app to the PMU to capture the data in a PMU capture file, then you should be able to replace that app connection with an openHistorian instance to save archive files directly. You’ll be storing the data to your disk either way. Can you explain why you feel you specifically need the PMU capture?

Thanks,
Stephen

Hi Stephen,
In my customer site, when the PMU is not connected with PDC, I want to save the PMU output in a file. In this scenario, there wont be pmu connection tester to capture the pmu output. I will have to write a code in my device to capture the pmu output.
You have mentioned that .PmuCapture file is the bytes from network protocol. Also it is an acceptable format to play the stream in open historian. Hence i feel that i can store the PMU output from my device in the same format (as .PmuCapture) in a file when the connection is lost with PDC. Later, this file can be downloaded from the customer site and it can be played in OpenHistorian to check for phasor details.

Hope I have mentioned the details to give you more clarity. Kindly provide your feedback on my approach.

Thanks in advance.

It sounds like you are creating an app on the PMU itself, is that correct? If there truly is a way you can determine whether the PDC is connected to the PMU in order to start capturing data into a PMU capture file, and also a way for you to retrieve the raw bytes that would have been sent to the PDC if it were connected, then your approach could work. Assuming you have solved those problems and produced the PMU capture file, here’s what you need to know from our end in order to replay it.

Fundamentally, TransportProtocol=File is used to read data from a PMU capture file. It’s used in connection strings to configure the frame parser. See: https://github.com/GridProtectionAlliance/openPDC/blob/master/Source/Documentation/wiki/Getting_Started.md#configuring-a-connection-string

The most straightforward way to import data from a PMU capture file is to create a new input device using the input device wizard in the openHistorian Manager. The problem is, the wizard will create all new measurements to associate with the data. That is to say, it will not be merged with the rest of your data associated with the same measurements. Fundamentally, the PhasorMeasurementMapper adapter that normally handles all the frame-based network streaming protocols supported by the openHistorian does not share its input measurements. However, you may be able to get around this by using the SharedMapping=ACRONYM connection string parameter to associate the file-based input with the real-time device’s configuration.

Therefore, your strategy should be to use the Inputs > Add New Device Based Input page in the openHistorian Manager. The connection string should be something like file=C:\Projects\openHistorian\Build\Output\Debug\Applications\openHistorian\Sample1344.PmuCapture; definedframerate=30; simulatetimestamp=false; autorepeatfile=false; transportprotocol=file; sharedMapping=SHELBY, making sure to use the appropriate PMU capture file path as well as the appropriate real-time device’s acronym in the shared mapping. Fill in the appropriate values for the ID Code, Protocol, and Historian fields. Check the Enabled flag and then Save.

Good luck!

Thank you so much Stephen.

Could you please explain me how do we manage to supply the configuration frames to openHistorian in this case.

Thanks.

You can serialize the config frame to the PMU capture file alongside the data frames, or you can serialize it to an XML format and load it using the configurationFile=C:\path\to\configurationFile.xml connection string parameter.

Thanks Stephen for the valuable information.
I will try and get back to you.

Regards.

Hi Stephen,
I would like to get your feedback on the below queries. Kindly provide your comments.

  1. When I use .PmuCapture file, currently OpenHistorian plays it like real time stream and archive the data. Is there any future plan to import the data from .PmuCapture file instead of playing it. In case if there is any plan at your end, could you please let me know when the feature will be available with openHistorian. What is the process to be followed to add any new feature with OpenHistorian.

  2. Is there any way to speed up the play back of Pmu capture file in openHistorian.

  3. Could you please comment on “Import COMTRADE option” for few hours of synchrophasor data. I dont see any information on how the PMU STAT filed have to put in COMTRADE synchrophasor file to convey the status information to openHistorian. Could you please share with me if you have any real time sample for COMTRADE file, which has all channel information including status field.

Sorry that i asked you many queries. Kindly update your feedback.

Hi Amala,

  1. No, there are no plans to directly import from PMU capture files rather than replaying them. Most users are content with buffering data in an archive using a PDC or SBG on the PMU’s local network, and that is the approach we will continue to recommend in the future.

  2. You can speed up playback by tweaking the sampling rate of the input device that reads from the file.

  3. I actually don’t really know anything about the COMTRADE import option as I was not involved in its development and have never used it myself. Sorry I can’t be of any help there. Maybe Ritchie can comment if he finds some time amidst his travels.

Thanks,
Stephen

Thanks Stephen for your feedback.

I have tried to increase the frame rate while selecting the pmu capture file in the connection string and observed that the time taken to payback and archive the data is decreasing. Thanks.

I need to save the configuration frames also with data frames while storing it in the file. Hence As you have suggested previously, I have tried to add the configuration frame and data frames together in pmu capture file and start the new device. but it didnt work. Here are the steps i followed.

  1. Open a pmu capture file and write the configuration frame, which will be sent to OpenHistorian as a response to configuration request. Configuration frame starts with 0xAA, 0x31, 0x06, 0x26 etc…
  2. Then start appending the data frames continuously in the same pmu capture file. protocol: IEC 61850 90-5. Data frame starts with 0x01, 0x40, 0xA2, 0x18 etc…
  3. Then in openHistorian, selected the option “Input/AddNewDeviceBasedInput”, then created new Acronym “TEST_CONFIG” and selected the proper protocol and the pmu capture file. Then Saved it.
    But it didnt play the capture file, instead it was showing TEST_CONFIG in red colour in Graph measurements page.
    Could you please suggest how to solve this problem.

Regards

If you add the device using the Inputs > Add New Device Based Input page, it will not automatically create child devices and measurements based on the configuration frame. You need to set up the connection string to define a shared mapping with the actual device connection so that the playback device uses the child devices and measurements of the actual connection to decode the data from the file. I wouldn’t normally suggest this, but it will be necessary because you are using these capture files to backfill missing data in the archive.

Thanks,
Stephen

Hi Stephen,
Thanks for your reply.
Today i tried to use sharedMapping=ACRONYM in the connection string and updated with actual device’s acronym name. But it didnt work it seems. It was still in red colour and didnt play anything.
my capture file will have change of values in IA:MAG measurements. i tried to view IA:MAG measurement of the actual device in grafana visualization. But there is no value for this measurement at expected time as per my capture file.
Could you pls suggest some more thing to test and make it work.

Regards.