This document presents an example of application using the Haro3D library version 2 that includes an API for the Microsoft Hololens.
In a previous document, it was shown how to use the OBJ file format and the Haro3D library to build and operate a virtual 3D robot from CAD files. The present document, and its associated code, is using the same approach to load the virtual robots in both LabVIEW and the Microsoft Hololens space. The virtual robot is then operated from the Hololens (or augmented reality) space.
The goal of the current document is twofold. First, the code is provided for anyone interested to use it to upload and control any type of robot, as long as the CAD files are converted and organized as explained in the document referenced above. Second, the provided code illustrates the general concepts of programming behind the Haro3D library that can be used for any other type of application.
A video showing the use of the VI described by the current document along with the Hololens operations is shown below.
The Haro3D library uses classes to create the 3D objects that will be uploaded either in a LabVIEW 3D picture control or in a Microsoft Hololens (or in any other future 3D display interface). The class structure of the Haro3D library along with the classes created for the current project are shown in the UML diagram below. The UML diagram was created from within LabVIEW using the NI GOOP Development Suite (the Haro3D library box and the legend were subsequently added).
UML diagram of the classes of the Haro3D library and of classes extended from it for the current project.
In the Haro3D library, all objects to be uploaded in a 3D space are derived from the abstract class 3D_Object. Two classes inherit from the 3D_Object class, the abstract class 3D_Shape and the OBJ_Mesh class. Generic 3D objects inherit from the abstract 3D_Shape class: Cylinder, Panel, Box, Sphere, and Mesh. The OBJ_Mesh class contains the attributes and methods to handle meshes described in a file of the Wavefront .obj format (Wikipedia link).
While inheriting from the 3D_Object class, the OBJ_Mesh class is actually an aggregation of objects of the Mesh class. The 3D virtual robots in the current project are described by .obj files. It was therefore decided that the robots would need to be described by OBJ_Mesh objects, or objects that inherited from that class. An actual robot can be defined as an assembly of axes, each of which is an assembly of different objects (robot parts, cabling, base, etc.). Each of those objects can be defined in the virtual world as one or more meshes, defined within an .obj file. A virtual robot for the current project is therefore an object of the class Holo_Robot that is a composite aggregation (or containment) of axis objects of the Holo_Robot_Axis class, that is itself a composite aggregation of objects of the Holo_Object class.
Within the VI, each Holo_Robot_Axis object is a collection of Holo_Object objects that can be handled in the 3D picture control and Hololens spaces using the methods defined for the OBJ_Mesh and the 3D_Object classes methods. The spatial relationships between the different Holo_Robot_Axis defining the robot axis movements, positions, and orientations, must be defined within the Holo_Robot and Holo_Robot_Axis classes.
The front panel and block diagram of the Hololens Robot Demo VI are shown below.
Front Panel of the Hololens Robot Demo VI. The 3D picture control, indicators, and controls are not essential but are provided for convenience. Robot in 3D picture control, position, angles, and axis values are updated as the axes are moved from the Hololens.
Block diagram of the Hololens Robot Demo VI. The architecture is a single-loop event-based Queued Message Handler.
Just unzip the attached file in a given directory.
This document presents an example of code uploading different robots into a Microsoft Hololens. The robot can be moved, scaled, and its axes independently moved by the Hololens user.
In a first time, the code presented here can be used as-is to upload any other type of robot as long as the CAD files are being converted and configured as described in a previous document.
In a second time, the code presented illustrates how to use the Haro3D library and how to extend the classes for any other type of application.
Please post question, feedback, and other comments by replying to the current post.