Merging data into a single row

[altenbach]

---

@Jousepa wrote:

 one row with (date, time, pH, temp, CD, temp).

---

Can you instead give the index of the desired fields? What's the difference between "temp" and "temp"?

[altenbach · Accepted solution]

Here's how you could use a map with the ID as key and the output row as value.

The rows in the original file can be in any order and the output is guaranteed to be sorted by ID.

In the full code, you should retain the map between iterations and just add new rows when they arrive.

Depending how "sane" the input file is, you might need additional checks and error handling)

For better debugging, I recommend to generate a test file where the pH and CD values actually differ between rows!

[Jousepa]

---

@altenbach  ha escrito:

---

@Jousepa wrote:

 one row with (date, time, pH, temp, CD, temp).

---

Can you instead give the index of the desired fields? What's the difference between "temp" and "temp"?

---

The index are (2 (date), 3 (time), 8 (ID), 9 (ph/CD), 11 (temp1/temp2) ).

The temperatures are provided by the pH and conductivity sensors themselves.

I guess its to late for this reply, I'll go check the code you just provided me. Thank you very much. 

Jousepa

[Jousepa]

---

@altenbach  ha escrito:

Here's how you could use a map with the ID as key and the output row as value.

The rows in the original file can be in any order and the output is guaranteed to be sorted by ID.

In the full code, you should retain the map between iterations and just add new rows when they arrive.

Depending how "sane" the input file is, you might need additional checks and error handling)

For better debugging, I recommend to generate a test file where the pH and CD values actually differ between rows!

---

I tried the code exactly as it was, adding a while loop, and I’m pretty sure it works just fine because im not doing a real streaming.
Thank you all, you are truly a blessing.

[altenbach]

---

@Jousepa wrote:
I tried the code exactly as it was, adding a while loop, and I’m pretty sure it works just fine because im not doing a real streaming.

Here's how I would add a while loop, keeping the map intact (no need to scan for duplicates, automatically sorted by ID, etc.). I also left some diagram comments for further development. For example, you could cache the map to file and reload on next program start.

Note that the file is only scanned on demand. It if should occur at regular intervals, you can add a reasonable timeout to the main event case..