Weird Call Library Node problem...

[Southern_Cross]

All,

I'm stumped on an issue. I have created both a DLL and a linux shared object from the same source. I wrap these libraries with LabVIEW for execution. One of the functions exported by the library is not being loaded by LabVIEW when running on NI-Linux (cRIO-9068, LabVIEW 2015sp1). All functions execute just fine on windows.

This function is created and exported just like all the others. The other functions load and execute just fine on the cRIO. However, the function "void CPG(void)" fails to load giving the following error

 

It seems that the function is being exported properly. Here is the output of Dependency Walker on the windows DLL compilation:

I can equally analyze the exports from the shared library using:

nm -D -C --defined-only -g ./librobotwrapper.so

the output here seems to indicate the function is exported just fine:

Yet, the error from LabVIEW is not referencing the function "CPG" but what appears to be a mangled function name "CPG:C". Neither Dependency Walker (windows) nor NM (linux) indicate any name mangling. And, as I mentioned all other functions exported successfully load and execute except this one. In fact, these graphics where made with a modified source where the guts of the CPG() function have been ALL commented out. This appears to be entirely a LabVIEW linker issue. I created a second 'deleteme.vi' with a fresh Call Library Node and got the same results. See below,

 I'm completely stumped here. Is there something simple that I'm just not seeing?

Thanks in advance.

[rolfk]

Basically the syntax of the library name is maybe a bit obscure but it is not wrong. When you deploy to a realtime target the deploy dialog will show you the location of the problematic functions as follows:

<library path>:<function name>:<calling convention>

So looking at that name in the error it seems to be ok! 

Try to remove the -C option to nm to not demangle the name. While LabVIEW does some retry with and without prepended underscore to work around some more simple name mangling issues, it won't be able to work around other name mangling issues and you may have to use some compiler attribute in the source code for the function to make the function not get mangled in a specific way by the compiler.

Playing with other nm options might also show more info.

[Southern_Cross]

Thanks Rolfk.

I followed your suggestion and executed, 

nm -D --defined-only -g librobotwrapper.so

The symbols still appear as expected so there appears to be no name mangling going on. 

To provide a little more background... here is a subset of my header file that illustrates how I'm exporting these functions:

   #if defined(_MSC_VER)
        //  Microsoft 
        #define ROBOT_WRAPPER_DLL __declspec(dllexport)
    #elif defined(__GNUC__)
        //  GCC
        #define ROBOT_WRAPPER_DLL __attribute__((visibility("default")))
    #else
        //  do nothing and hope for the best?
        #define ROBOT_WRAPPER_DLL
        #pragma warning Unknown dynamic link import/export semantics.
    #endif

...

extern "C" {

...

ROBOT_WRAPPER_DLL void forwardKinematics();
ROBOT_WRAPPER_DLL void CPG();

...

} // extern "C"

The corresponding function definitions in the .cpp look like:

...
ROBOT_WRAPPER_DLL void forwardKinematics()
{
...
}
ROBOT_WRAPPER_DLL void CPG()
{
...
}
...

So, this is pretty straight-forward stuff. I'm not doing anything fancy.

The other functions, (like forwardKinematics) deploy and execute as expected. However, the CPG function does not. 

Last tidbit of background. I built a simple C++ application that loads my .so both on windows and cross-compiled for NI-Linux. Both executables run without error. This tells me that the library configuration on the NI-Linux system is in a good state since the linux executable ran without error. 

I'm sure there is some little detail that I'm overlooking, but, given these symptoms it points to the LabVIEW linker... 

[rolfk]

LabVIEW on these targets mostly just does a dlopen() and then a dlsym() for the library name and function you specify. If the dlsym() call fails it does a little name mangling such as retrying with the name prepended with an underscore and after that it simply considers the function to be not present.

So it most likely is something with the shared library which makes the Linux ELF loader not recognize the function name properly. The LabVIEW linker while present, doesn't really come into play much here.

One thing that ELF does very different than the Windows shared library loader is that when you link an executable to an ELF library function, the compiler will only add a weak reference to the executable and the actual function will be resolved when the function is first called. The ELF import section in your executable also doesn't normally specify the actual so name but just the function name and then the ELF loader searches through all presently loaded shared libraries if the searched symbol is present and only after that fails will it go and search for a shared library that may or may not resolve the symbol. In Windows when the executable links to a dynamic function that the DLL loader can not resolve, the entire loading of the executable fails immediately.

So even if you create an ELF executable that links to that symbol, if the function is never called it does not indicate that that symbol can really get resolved!

Other ideas to try: Make the name contain at least one lower case letter, probably best to start with one. Under Linux Uppercase or especially UPPERCASE function names are frowned at by many programmers, and CamelCase is generally cursed at. It's very much possible that the GCC compiler/linker assigns some special meaning to such functions that play up with you on this.

[Southern_Cross]

Thanks @ Rolfk.

You are correct. It's not the "LabVIEW linker". That was just my poor description of what I was speculating was going on. 

I had created a C++ test application to test without LabVIEW. It loaded the .so and called the functions in question. It ran just fine on the NI-Linux cRIO-9068. However, I linked directly to the .so when I built the application. I neither used dlopen() nor dlsym(). So, given your description, I made a new application that is based on dlopen() and dlsym(). It also works like a charm...

So, now that I've used the same mechanism that LabVIEW uses to dynamically load the .so and called the two functions that LabVIEW fails to deploy (i.e. "StepAll" and "CPG")... I'm stuck again. The fact that dlsym() successfully loaded and ran the "StepAll" and "CPG" functions in the new C++ application confirms the results from my previous NM tests. 

Any other ideas?

Here is the code that I wrote (NOTE: earlier I had changed the exported function name from 'CPG' to 'oscillator' as I was experimenting).

#include <iostream>
#include <dlfcn.h>
#include <stdint.h>

#define JDOF 12
#define NOL 4

int main() {
    using std::cout;
    using std::cerr;

    cout << "C++ dlopen demo\n\n";

    // open the library
    cout << "Opening robotwrapper.so...\n";
    void* handle = dlopen("./librobotwrapper.so", RTLD_LAZY);
    
    if (!handle) {
        cerr << "Cannot open library: " << dlerror() << '\n';
        return 1;
    }
    
    // load the INIT symbol
    cout << "Loading symbol 'init'...\n";
    typedef uint32_t (*init_t)(const char*, const char*);

    // reset errors
    dlerror();
    init_t init = (init_t) dlsym(handle, "init");
    const char *dlsym_error = dlerror();
    if (dlsym_error) {
        cerr << "Cannot load symbol 'init': " << dlsym_error <<
            '\n';
        dlclose(handle);
        return 1;
    }
    
    // load the StepAll symbol
    cout << "Loading symbol 'StepAll'...\n";
    typedef void (*stepall_t)(double*, double*, double*, uint32_t*, uint32_t, double, double*, double*, double* );

    // reset errors
    dlerror();
    stepall_t StepAll = (stepall_t) dlsym(handle, "StepAll");
    //const char *dlsym_error = dlerror();
    if (dlsym_error) {
        cerr << "Cannot load symbol 'StepAll': " << dlsym_error <<
            '\n';
        dlclose(handle);
        return 1;
    }
    
    // load the 'oscillator' symbol (formerly known as CPG)
    cout << "Loading symbol 'oscillator'...\n";
    typedef void (*oscillator_t)(void);

    // reset errors
    dlerror();
    oscillator_t oscillator = (oscillator_t) dlsym(handle, "oscillator");
    //const char *dlsym_error = dlerror();
    if (dlsym_error) {
        cerr << "Cannot load symbol 'oscillator': " << dlsym_error <<
            '\n';
        dlclose(handle);
        return 1;
    }



    // readDevices()
    double sensor_q[JDOF] = {-0.006, -0.061, 0.145, 0.006, -0.061, 0.145, -0.006, 0.061, -0.145, 0.006, 0.061, -0.145};
    double sensor_gaq[10] = {0,0,0,0,0,0,1,0,0,0};
    uint32_t sensor_contact[NOL] = {0,0,0,0};
    double estimated_qdot[JDOF] = {0,0,0,0,0,0,0,0,0,0,0,0};
    uint32_t status = 0;
    double time_stamp = 0.0;
    double tau_des_idyn[JDOF] = {0,0,0,0,0,0,0,0,0,0,0,0};
    double tau_trunkControl[JDOF] = {0,0,0,0,0,0,0,0,0,0,0,0};
    double tau_des[JDOF] = {0,0,0,0,0,0,0,0,0,0,0,0};
	
     // use it to do the calculation
    cout << "Calling 'init'...\n";
    init("lvproperties_allegro.xml", "./");
   
    cout << "Calling 'StepAll'...\n";
    StepAll(sensor_q, \
            estimated_qdot, \
            sensor_gaq, \
            sensor_contact, \
            status, \
            time_stamp, \
            tau_des_idyn, \
            tau_trunkControl, \
            tau_des);
    
    std::cout << "tau_des = " << std::endl;
    for(int i=0; i<JDOF; i++)  {
	std::cout << tau_des[i] << std::endl;
    }

    // use it to do the calculation
    cout << "Calling 'oscillator'...\n";
    oscillator();
    
    // close the library
    cout << "Closing library...\n";
    dlclose(handle);
}

[rolfk]

---

@Southern_Cross wrote:

So, now that I've used the same mechanism that LabVIEW uses to dynamically load the .so and called the two functions that LabVIEW fails to deploy (i.e. "StepAll" and "CPG")... I'm stuck again. The fact that dlsym() successfully loaded and ran the "StepAll" and "CPG" functions in the new C++ application confirms the results from my previous NM tests.  

---

I was under the impression that other functions worked and only CPG() didn't!

Is that not anymore correct? Because if that is the case then you probably have a more basic problem that the shared library can't be found by the ELF loader. The fact that it works in your executable doesn't automatically mean that LabVIEW should be able to reference it too. Shared libraries are normally installed in one of the /lib directories (and to make things even more fun you have often 32 bit and 64 bit library directories too on x86/x64 systems). 

When you copy a so to the target system (you have done that, haven't you, because LabVIEW will NOT deploy the so to the RIO target) you usually also have to run ldshared to make the so known to the shared library cache.

Maybe this will give you a bit more insight in how the shared library manager works on Linux. It is a pretty complex system and copying so's to the target system alone is usually not enough (although it sometimes works).

[Southern_Cross · Accepted solution]

@Rolfk

Thanks much for the insights. I solved the problem today. As always... user error. 

Your comments and shared link (which is excellent by the way!) helped me realize that I was not being sufficiently careful with my dependent libs. After discovering some conflicting libs, libs in the wrong location and the like I got the cross-compile .so to run with the LabVIEW wrappers.

For future users who may run in to this... In a nutshell, I needed to do the following:

1) make a file in /etc/ld.so.conf.d/mylibs.conf with a path(s) to the location of your .so files

2) make all needed symbolic links (e.g. libmydeplib.so.0 --> libmydeplib.so.0.0.0)

3) run LDCONFIG after everything is setup from #1 & #2

4) use LDD to analyze the dependencies of your top level .so (or executable). Pay close attention to the paths reported by LDD. Make sure that none of your own .so files conflict with system .so files. (One way to do this is to run LDD on your top level .so before steps 1-4 and take a screen shot. All system .so files will already be resolved with a system path. Your custom .so files will be labeled as unknown. After completing steps 1-4 compare the output of the first LDD with that of step 4. make sure that none of the system .so files' path has changed to an undersireable location). 

5) If this is unclear then go read the info in the link that Rolfk provided. It's very helpful.

@Rolfk, thanks again for all your help!