To show the front panel of your DLL you will need to use VI properties>>Window Appearance>>Customize. Set the Front Panel when called option. This will open the front panel to your DLL when the DLL is called.
If you call the DLL from a LabVIEW VI running in the user interface thread (a.k.a. execution system) you will get a deadlock if the VI called inside the DLL needs to do some work (such as showing its front panel).
To work around the deadlock, you must make sure that the DLL is called from a thread other than the user interface thread. To do so, first configure the shared library node to be "Reentrant" rather than "Run in UI Thread". Then make sure that the VI is not running in the UI thread by looking in VI Properties>>Execution>>Preferred Execution S
ystem
I think the same would apply if called from a C++ application.