# Unable to make output with CSVInputAdapter

**URL:** <http://discussions.gridprotectionalliance.org/t/unable-to-make-output-with-csvinputadapter/1020>\
**Category:** openPDC\
**Created:** [April 2, 2022, 7:03pm UTC](http://discussions.gridprotectionalliance.org/t/unable-to-make-output-with-csvinputadapter/1020 "2022-04-02T19:03:40Z")\
**Posts on this page:** 12\
**Page:** 1

<div class="post-metadata">

**Author:** ![ali94](http://discussions.gridprotectionalliance.org/letter_avatar_proxy/v4/letter/a/e274bd/32.png) [@ali94](http://discussions.gridprotectionalliance.org/u/ali94)\
**Post date:** [April 2, 2022, 7:03pm UTC](http://discussions.gridprotectionalliance.org/t/unable-to-make-output-with-csvinputadapter/1020/1 "2022-04-02T19:03:40Z")

</div>

Hi,  
OpenPDC works great for reading a CSV file. However, once I use real timestamp of CSV file measurements instead of simulated timestamp (when I put SimulateTimestamp=False in connection string of manage custom input adapters), PMU connection tester is unable to receive data. I am not sure where the mistake occurs. I would appreciate if you could help me. Here are screenshots that might help:

![image](http://gridbits.gridprotectionalliance.org/uploads/default/original/1X/78b9c36eb66fa2947f54a4d248e8694613f2403e.png)

![image](http://gridbits.gridprotectionalliance.org/uploads/default/original/1X/4b69873ccc6911ffe6a0cccde3bed8a7cc3642cd.png)

Thanks,  
Ali

---

<div class="post-metadata">

**Author:** ![Korunekosama](http://discussions.gridprotectionalliance.org/user_avatar/discussions.gridprotectionalliance.org/korunekosama/32/708_2.png) [@Korunekosama](http://discussions.gridprotectionalliance.org/u/Korunekosama)\
**Post date:** [April 5, 2022, 11:52am UTC](http://discussions.gridprotectionalliance.org/t/unable-to-make-output-with-csvinputadapter/1020/2 "2022-04-05T11:52:47Z")

</div>

HEllo,

I guess there is a control on the accordance between your local system time and the timestamps in your input file. If this difference is too large (lag time \> few seconds) then they may be rejected by OpenPDC as it may consider them as obsolete.  
I’m not sure that it’s the real cause of the problem ; but it could be.

Hope that could help

Stephane

---

<div class="post-metadata">

**Author:** ![StephenCWills](http://discussions.gridprotectionalliance.org/user_avatar/discussions.gridprotectionalliance.org/stephencwills/32/20_2.png) [@StephenCWills](http://discussions.gridprotectionalliance.org/u/StephenCWills)\
**Post date:** [April 5, 2022, 2:37pm UTC](http://discussions.gridprotectionalliance.org/t/unable-to-make-output-with-csvinputadapter/1020/3 "2022-04-05T14:37:34Z")

</div>

Hi ali94,

Stephane is correct about the time difference between the captured CSV data and your local clock. It’s almost impossible to synchronize these. You should consider checking your output stream configuration. If you are not using simulated timestamps, then you must disable `Use Local Clock As Real-Time` under `Advanced Properties`. Also, on the CSV adapter, consider disabling `AutoRepeat` or else the timestamps will wrap back to the beginning and your output stream will start dropping data.

You may also want to check the openPDC Console and the error log (ErrorLog.txt) when initializing the CSV adapter to check for errors in the CSV adapter. Note that when `SimulateTimestamps` is set to false, the timestamp column must be specified in ticks, which is a long integer format representing the number of 100-nanosecond intervals since January 1, 0001.

> **[DateTime.Ticks Property (System)](https://learn.microsoft.com/en-us/dotnet/api/system.datetime.ticks?view=netframework-4.6.1)**
>
> Gets the number of ticks that represent the date and time of this instance.

Thanks,  
Stephen

---

<div class="post-metadata">

**Author:** ![ali94](http://discussions.gridprotectionalliance.org/letter_avatar_proxy/v4/letter/a/e274bd/32.png) [@ali94](http://discussions.gridprotectionalliance.org/u/ali94)\
**Post date:** [May 4, 2022, 10:28am UTC](http://discussions.gridprotectionalliance.org/t/unable-to-make-output-with-csvinputadapter/1020/4 "2022-05-04T10:28:08Z")

</div>

Thanks for the reply, now it is possible to receive data with no problem. But there are other problems and I would appreciate if you could help me:

1-Some measurements (Analog Values, Digital Values, Calculated Values) appear to be broken, and the PMU connection tester is unable to receive them in this scenario.

2- Some time stamps and measurements are failed to match correctly in receiving point (PMU connection tester)

3- In the case of a miss data, the PMU connection tester deletes all measurements in a row of input data.

4- Some stats and measurements in Graph Real-time Measurements and Stream Statistics do not work.

![image](https://gridbits.gridprotectionalliance.org/uploads/default/original/2X/3/34c12db3cb35a49fb770e4d16e289563f0770ed6.jpeg)  
Input data in CSV Input Adapter (problem 1)

![image](https://gridbits.gridprotectionalliance.org/uploads/default/original/2X/e/eee3c7c627ab3ffad6d9173e677c441bdc3a9672.jpeg)  
export data in PMU connection tester (problem 1)

![image](https://gridbits.gridprotectionalliance.org/uploads/default/original/2X/e/eaa01ac206cf9c91c9c3fd4759c6357a660768eb.jpeg)  
Input data in CSV Input Adapter (problem 2,3)

![image](https://gridbits.gridprotectionalliance.org/uploads/default/original/2X/5/575813bf2d698424273d9433b3ffe76f9e421ef9.jpeg)  
export data in PMU connection tester (problem 2,3)

![image](https://gridbits.gridprotectionalliance.org/uploads/default/original/2X/2/2713fa63ea266beae45257d67a1ab2b5d60a417f.png)  
Graph Real-time Measurements (problem 4)

![image](https://gridbits.gridprotectionalliance.org/uploads/default/original/2X/5/553f2a972c4bf9a1f5a23e03ef3a7a0e3858977c.jpeg)  
Stream Statistics (problem 4)

---

<div class="post-metadata">

**Author:** ![StephenCWills](http://discussions.gridprotectionalliance.org/user_avatar/discussions.gridprotectionalliance.org/stephencwills/32/20_2.png) [@StephenCWills](http://discussions.gridprotectionalliance.org/u/StephenCWills)\
**Post date:** [May 4, 2022, 3:22pm UTC](http://discussions.gridprotectionalliance.org/t/unable-to-make-output-with-csvinputadapter/1020/5 "2022-05-04T15:22:24Z")

</div>

1. I’m not too sure what you’re referring to here. Which measurements in your screenshots are broken, and how can you tell?

2. The CSV adapter is reading the timestamps and measurement values from the same row of data in the CSV file. It really shouldn’t be possible for there to be a mismatch between them. I can’t tell from your screenshot how you determined that there was mismatch there so I can’t really come up with any theories as to how it might have happened.

3. It looks like the CSV adapter is using `double.Parse()` on the input data, which will throw an error if you pass it an empty string. As a result, it is skipping those lines of input where you have blank cells in the `PPA:194` column. If you don’t have any valid data to enter into those blank cells, you can enter `NaN` instead to prevent the errors.

4. On the Graph Measurements page, the openPDC Manager is probably using the local clock to filter out stale measurements from the display. There should be a `Display Settings` link somewhere on that screen that enables you to modify that behavior so you can see the values of those measurements.  
.  
The device statistics are computed by the protocol parsers to determine network performance. The CSV adapter doesn’t produce statistics so that output on the Stream Statistics page is to be expected. I’m not sure whether any of those stats are even applicable to a CSV input or, if they were, what value they would provide.

---

<div class="post-metadata">

**Author:** ![ali94](http://discussions.gridprotectionalliance.org/letter_avatar_proxy/v4/letter/a/e274bd/32.png) [@ali94](http://discussions.gridprotectionalliance.org/u/ali94)\
**Post date:** [May 7, 2022, 9:02am UTC](http://discussions.gridprotectionalliance.org/t/unable-to-make-output-with-csvinputadapter/1020/6 "2022-05-07T09:02:49Z")

</div>

Thanks for the reply, `double.Parse()` error is fixed but still the PMU connection tester does not receive analog, digital, or calculated values and the last row of input data in the CSV file(row 32).

 ![image](https://gridbits.gridprotectionalliance.org/uploads/default/original/2X/a/a94be668970166d1d154c8e872d057a5028366ec.png)  
Input data in CSV Input Adapter(highlighted columns are analog, digital and calculated values that are not available in PMU connection tester exported data)

 ![image](https://gridbits.gridprotectionalliance.org/uploads/default/original/2X/e/e7c5d81e57eebbb266db6a9738c1868e3e19a5f0.png)  
export data in PMU connection tester

---

<div class="post-metadata">

**Author:** ![StephenCWills](http://discussions.gridprotectionalliance.org/user_avatar/discussions.gridprotectionalliance.org/stephencwills/32/20_2.png) [@StephenCWills](http://discussions.gridprotectionalliance.org/u/StephenCWills)\
**Post date:** [May 12, 2022, 5:15pm UTC](http://discussions.gridprotectionalliance.org/t/unable-to-make-output-with-csvinputadapter/1020/7 "2022-05-12T17:15:35Z")

</div>

> … still the PMU connection tester does not receive analog, digital, or calculated values and the last row of input data in the CSV file(row 32).

Probably, this has to do with your output stream configuration in openPDC. Note that when you are adding a device to an output stream using the Output Device Wizard, it will not automatically include digitals or analogs. You must explicitly mark the checkboxes to include them.

 ![image](http://discussions.gridprotectionalliance.org/uploads/default/original/2X/d/dcd1a2388deebd5b23801081c24977f93958df5c.png)

Also note that this wizard will not include calculated values regardless because there is no corresponding signal type in the IEEE C37.118 standard. If you want to include calculated values, you will have to change the signal type to one that is supported by the standard, such as an analog value. This can be done explicitly for the output stream by mapping the input measurement to an output measurement with an appropriate signal reference, but you may find it easier to simply change the signal type of the input measurement before adding the device to the output stream using the wizard.

---

<div class="post-metadata">

**Author:** ![ali94](http://discussions.gridprotectionalliance.org/letter_avatar_proxy/v4/letter/a/e274bd/32.png) [@ali94](http://discussions.gridprotectionalliance.org/u/ali94)\
**Post date:** [May 15, 2022, 9:56am UTC](http://discussions.gridprotectionalliance.org/t/unable-to-make-output-with-csvinputadapter/1020/8 "2022-05-15T09:56:33Z")

</div>

Thanks for the reply, Analog and Digital Values are successfully added but once I click on Initialize, these measurements disappear from `Measurements` in `Manage Output Streams`.

 ![image](https://gridbits.gridprotectionalliance.org/uploads/default/original/2X/a/ab67bd9d3e164c05109fadc0f69b8cce5cbcb3af.png)  
Input data in CSV Input Adapter(highlighted columns are analog, digital )

 ![image](https://gridbits.gridprotectionalliance.org/uploads/default/original/2X/c/cfe5ef56e97f7aa8c5347d67ecd9890208ca80a8.png)  
export data in PMU connection tester

---

<div class="post-metadata">

**Author:** ![StephenCWills](http://discussions.gridprotectionalliance.org/user_avatar/discussions.gridprotectionalliance.org/stephencwills/32/20_2.png) [@StephenCWills](http://discussions.gridprotectionalliance.org/u/StephenCWills)\
**Post date:** [May 16, 2022, 1:57pm UTC](http://discussions.gridprotectionalliance.org/t/unable-to-make-output-with-csvinputadapter/1020/9 "2022-05-16T13:57:39Z")

</div>

We do have some code that deletes invalid output stream measurements when reloading configuration from the database.

```auto
// Validate measurements associated with this output stream
foreach (DataRow outputStreamMeasurement in database.Connection.RetrieveData(database.AdapterType, $"SELECT * FROM OutputStreamMeasurement WHERE AdapterID = {adapterID} AND NodeID = {nodeIDQueryString}").Rows)
{
    // Parse output stream measurement signal reference
    deviceSignalReference = new SignalReference(outputStreamMeasurement.Field<string>("SignalReference"));

    // Validate that the signal reference is associated with one of the output stream's devices
    if (deviceAcronyms.BinarySearch(deviceSignalReference.Acronym, StringComparer.OrdinalIgnoreCase) < 0)
    {
        // This measurement has a signal reference for a device that is not part of the associated output stream, so we mark it for deletion
        measurementIDsToDelete.Add(outputStreamMeasurement.ConvertField<int>("ID"));
    }

    // Validate that the signal reference type is valid for an output stream
    if (validOutputSignalKinds.All(validSignalKind => deviceSignalReference.Kind != validSignalKind))
    {
        // This measurement has a signal reference type that is not valid for an output stream, so we mark it for deletion
        measurementIDsToDelete.Add(outputStreamMeasurement.ConvertField<int>("ID"));
    }
}

```

Both of these conditions have to do with the `SignalReference` field of the output stream measurements. After adding the analogs and digitals, can you check the signal references of your output stream measurements before initializing the output stream?

---

<div class="post-metadata">

**Author:** ![ali94](http://discussions.gridprotectionalliance.org/letter_avatar_proxy/v4/letter/a/e274bd/32.png) [@ali94](http://discussions.gridprotectionalliance.org/u/ali94)\
**Post date:** [May 16, 2022, 3:08pm UTC](http://discussions.gridprotectionalliance.org/t/unable-to-make-output-with-csvinputadapter/1020/10 "2022-05-16T15:08:04Z")

</div>

yes, output stream signal references are visible before initializing, but after initializing they disappear.

 ![image](https://gridbits.gridprotectionalliance.org/uploads/default/original/2X/f/fcce4d76e5756f0e29eb371348e3926215a96fa2.png)  
( before initializing)

 ![image](https://gridbits.gridprotectionalliance.org/uploads/default/original/2X/9/9cff1b3abc14adf19e391a30f2071b6d96e4c132.png)  
( after initializing)

 ![image](https://gridbits.gridprotectionalliance.org/uploads/default/original/2X/a/ae89eeadc770fed8c016c576642be8a0b707291a.png)  
( Analogs that are generated automatically after adding Device Wizard)

 ![image](https://gridbits.gridprotectionalliance.org/uploads/default/original/2X/7/750a49f350f09b59f19f5726f11a7f48da2f511e.png)  
( Digitals that are generated automatically after adding Device Wizard)

---

<div class="post-metadata">

**Author:** ![StephenCWills](http://discussions.gridprotectionalliance.org/user_avatar/discussions.gridprotectionalliance.org/stephencwills/32/20_2.png) [@StephenCWills](http://discussions.gridprotectionalliance.org/u/StephenCWills)\
**Post date:** [May 16, 2022, 3:22pm UTC](http://discussions.gridprotectionalliance.org/t/unable-to-make-output-with-csvinputadapter/1020/11 "2022-05-16T15:22:11Z")

</div>

Those measurements are being removed because their `Signal Reference` field isn’t right. It should be `SHELBY2-DV1` for the digital value and `SHELBY2-AV1` for the analog channel.

---

<div class="post-metadata">

**Author:** ![ali94](http://discussions.gridprotectionalliance.org/letter_avatar_proxy/v4/letter/a/e274bd/32.png) [@ali94](http://discussions.gridprotectionalliance.org/u/ali94)\
**Post date:** [May 16, 2022, 4:09pm UTC](http://discussions.gridprotectionalliance.org/t/unable-to-make-output-with-csvinputadapter/1020/12 "2022-05-16T16:09:22Z")

</div>

Thank you so much, I edited `Signal Reference` and it worked.