AWS IOT on cRIO - dependency version clash with OpenSSL?

[rtollert]

---

@User002 wrote:
It is a new requirement in this library, yes, but it looks like (if I remember correctly) that OpenSSL 1.1 was released in something like 2018.

Yeah. It's not ideal. We'll try to get it updated soon.

---

Interesting. I assumed that because it and a few other key libraries like glibc were pinned, there were compatibility reasons for holding them steady.  From that, I figured that such a static linking was doomed to fail because of those other compatibility issues.  I'm still a bit reluctant to try it unless you feel pretty sure it's probably ok.  But if it is likely to succeed, that would be great news!

I mean... let me be clear. I'm an R&D engineer, not NI support. If it doesn't work, I don't have the bandwidth to help you try to fix it, and NI support will, and probably should, tell you to pound sand. At the end of the day, all I can provide to you right now is possibly true ideas, and hope that they make you more productive.

What I am saying is that, for Unix libraries in general, it is possible to build your own copies of stuff in a way that does not override whatever exists on the system. Ad it is also possible to force other libraries to use your builds instead of what the OS provides through suitable reconfiguration.

OTOH, it is also possible for your own OpenSSL build to conflict with what LabVIEW uses, for reasons I am presently unaware of.

---

---

Fascinating.  Looking at it again, it does make sense.  I might give this a try - hmm but to use it, I'd need to write such a module into whatever LabVIEW-native MQTT library used. 

Hmm, sounds like there is some traffic elsewhere on the subject — will this do you? Or did I google for the wrong project? https://github.com/LabVIEW-Open-Source/LV-MQTT-Broker/issues/154

As to why my frustration may have leaked out in my post - the cRIO's software compatibility is really close to having this be an easy project, but the ecosystem falls short in a bunch of little ways.  None of them is a single major flaw, but they seem to line up perfectly with each other and combine to nearly block me completely.  All the project really needs is an MQTT library that has good send queuing (because the link out there at the IoT edge is sometimes spotty), but none of the native LabVIEW libraries I could find supported that.  Well, and of course the mutual TLS authentication.  After spending years in systems engineering at NI and recommending cRIOs to our top customers and seeing the amazing applications they excel at (and being a huge fan of cRIO in general), this felt a bit like a letdown.

That frustration seems pretty well-placed. I'll make sure that this feedback reaches the appropriate parties.

Regarding client certs specifically: this whole situation is admittedly something I was not expecting. I looked hard for LabVIEW users who had strong requirements for client certificates. I honestly couldn't find them. X.509 certificate management is not something that I had believed LabVIEW developers ought to develop a competence in. 

---

---

Might you have any insight into more recent python libraries on RT?  (I'm fine moving to the newest (say, LV RT 2021) if I need to do that to get it).

I'm gonna have to fob you off to one of the NI Linux RT devs about the python situation. My recollection is that we are merely rebuilding what we're getting upstream from the Yocto Project. 

[rtollert]

---

@User002 wrote:

---

"TLS is barely available on some platforms in LabVIEW." Insofar as you're talking about NI Linux RT, I don't think I understand this comment.

---

Apologies if this was unclear - I meant barely in time rather than barely in functionality.  As if to say that since it took however many years to get TLS in LabVIEW, that it might take many more to get support for whatever comes next.

Forgive my indulgence in this next statement because it is not entirely justified because this case alone, but rather as a conclusion from a much broader set of very similar experiences.  So often NI and LabVIEW appear to have a need to re-implement things themselves rather than pull in compatibility to a 3rd party library.  From where I sit, it seems that if NI took a fraction of the cost they spend on rebuilding tools within LabVIEW and instead spent it on improving certain kinds of compatibility, the ecosystem would be very much better off.

---

Ah. Yeah, I think I get what you mean.

I think I agree, but part of the root cause is that the customer deliverables tend to get too concrete too quickly, and that problem lacks simple solutions. I do what I can.

Another big part of the problem is FFI impedance mismatches, which is a big part of why I think people want to bounce to python in the first place (when in many cases they might just be bouncing to C anyway).

Hopefully there will be continuous improvement towards these sorts of foundational interop technologies going forward. (That is: it is in general on our minds more, and it is actively happening right now.)

[User002]

---

I mean... let me be clear. I'm an R&D engineer, not NI support. If it doesn't work, I don't have the bandwidth to help you try to fix it, and NI support will, and probably should, tell you to pound sand. At the end of the day, all I can provide to you right now is possibly true ideas, and hope that they make you more productive.

What I am saying is that, for Unix libraries in general, it is possible to build your own copies of stuff in a way that does not override whatever exists on the system. Ad it is also possible to force other libraries to use your builds instead of what the OS provides through suitable reconfiguration.

OTOH, it is also possible for your own OpenSSL build to conflict with what LabVIEW uses, for reasons I am presently unaware of.

---

---

Oh that's fine - I don't expect any more than that from support.  Thank you - you've given me hope.  That's one possible path to success right there, even though it's not certain.

Hmm, sounds like there is some traffic elsewhere on the subject — will this do you? Or did I google for the wrong project? https://github.com/LabVIEW-Open-Source/LV-MQTT-Broker/issues/154

---

I had checked out that project in the past.  If mutual auth is indeed possible, then it would even be worth me adding it in there.

The main reason I didn't love that particular implementation was that it had no depth to the outbound queue of messages.  If you wanted acknowledgement, you had to wait after each message to make sure it went through - or so it appeared at the time.  It looks like AWS MQTT message size limits are 128 kB these days (https://stackoverflow.com/questions/57414725/what-is-the-maximum-message-payload-size-for-a-aws-iot-...) and I'm needing to send that much data every couple of seconds to keep up with the rate the stream is generating.  If there's lots of latency in the pipe that gets to be a problem real quick.  But there might be enough headroom for it... so that's another possibility.

It seems that some non-LabVIEW MQTT libraries I found offer deeper outbound queues are capable of having multiple messages at a time outstanding and awaiting acknowledgement, which lets you have greater throughput.  I didn't dig fully into the source, but did see things such as "outbound queue size" as settable parameters, which the LabVIEW-native implementations did not.

This (mutual auth and MQTT native to LabVIEW) is possible path number two.  Thanks for that tip.

That frustration seems pretty well-placed. I'll make sure that this feedback reaches the appropriate parties.

Regarding client certs specifically: this whole situation is admittedly something I was not expecting. I looked hard for LabVIEW users who had strong requirements for client certificates. I honestly couldn't find them. X.509 certificate management is not

It's nice to have a shoulder to cry on.  And thank you for passing it on.

As for certificates, it did catch me by surprise.  But following security news has me thinking that encryption (client cert or not) should be default for just about all communications these days... and yet that is definitely not the reality we're seeing on the ground floor in lots of places.

---

---

I'm gonna have to fob you off to one of the NI Linux RT devs about the python situation. My recollection is that we are merely rebuilding what we're getting upstream from the Yocto Project. 

---

Oh I don't expect anything newer than what's upstream in Yocto, I suppose, either.  I mostly got alarmed when my cRIO told me that its python 3.5.5 was becoming deprecated, without seeing a replacement when I queried the repos.  It's mostly knowing that the new version will get there in time for me to not be out of supported python.  I'm ok moving to the latest RT version if I need to.  An all- or mostly- python application is my third path, but it dries up if the base python packages are left without updates.

Thank you,

[User002]

---

---

Ah. Yeah, I think I get what you mean.

I think I agree, but part of the root cause is that the customer deliverables tend to get too concrete too quickly, and that problem lacks simple solutions. I do what I can.

Another big part of the problem is FFI impedance mismatches, which is a big part of why I think people want to bounce to python in the first place (when in many cases they might just be bouncing to C anyway).

Hopefully there will be continuous improvement towards these sorts of foundational interop technologies going forward. (That is: it is in general on our minds more, and it is actively happening right now.)

---

My preference is always LabVIEW.  If I have to go to another language, it's generally because it's the only possible way to get it done.  But better a less-than-optimal way than no way.  I've always appreciated that LabVIEW's "walled garden" has in general very short walls and many gates.  So much so that when I can't find an escape hatch I need, it's frustrating.

Your words are a reassurance to me.  Thank you again for taking the time to reply.

[captain_only]

We are working on clarifying the intent and roadmap around NI Linux RT. Partly because we aren't there yet, we made a decision to be open about our development. For now, the best place to get a sense of where we are going is to look at the github repo directly. For example: https://github.com/ni/openembedded-core/blob/nilrt/master/dunfell/meta/recipes-devtools/python/pytho...

[User002]

That's fantastic information.  If I'm reading this right, we're looking at Python 3.8.2, which looks great.

I can't make out the branch structure because it's full of code names but if I had to guess, this is aiming for the LabVIEW RT 2021 release.  Is that right, or will these new packages be available in the package repos for earlier versions too?  (In other words, do I need to plan to update my LV 2020 cRIO to LV 2021 in order to get this new python release, or will an opkg update and opkg upgrade be enough for an existing 2020 install?  This makes a big difference for planning this project, which is why I ask.  I'm ok either way, but it would be super helpful to know how the model works.)

Thanks,

[SAL9000]

We can't commit to specific dates or versions, but I can talk about the process a bit.

To get major version bumps for packages like Python, we do a full distribution update.  That allows us to insure that compatibility between packages is maintained (like some other distributions). We do offer compatibility of the distribution with older LabVIEW versions however, and thus all you should need is an updated System Image (how we ship the distribution) and drivers once this is available.

To be clear and answer your question directly based on the above:
...will an opkg update and opkg upgrade be enough for an existing 2020 install?  => No, you'll have to update the whole system image when it starts to support "Dunfell" to get that support.  It should work with LabVIEW 2020.

I hope that helps!