From Friday, January 17th 11 PM CDT (January 18th 5 AM UTC) through Saturday, January 18th 11:30 AM CDT (January 18th 5:30 PM UTC), ni.com will undergo system upgrades that may result in temporary service interruption.
We appreciate your patience as we improve our online experience.
Human Operator Modeling and Controller Design by Using a Human-Machine Interaction Experiment
by
Hakan Ertugrul, M.Sc., Ozkan Celik, M.Sc., and Seniz Ertugrul, Ph.D.
Istanbul Technical University, Mechanical Engineering,
34437, Gumussuyu, Istanbul, Turkey
Products Used:
National InstrumentsTM NI PCIe-6259, LabVIEWTM, NI Compact Vision System, IEEE 1394 Basler A601f Camera.
The Challenge:
This study involves building an experimental setup for collecting human operator control action data during completion of a specific control task and modeling of the human operator. Both linear parametric and intelligent modeling methods have been used. Obtained models have been embedded into the system as stand-alone controllers to replace the human operator in the closed-loop.
The Solution:
Human operators were asked to control a flexible robot arm using a joystick to follow trajectories appeared on the computer screen. During manual control, data were collected and mathematical models of human operator were obtained using system identification theory. DC motor driving the flexible robot arm and a potentiometer were connected to NI PCIe-6259 card for a part of the data acquisition system. The most important part of the solution is actually the vision system which provided the human-machine interaction as explained below. The mathematical models that could also be identified on-line successfully replaced human operators as controllers. Such a challenging job was achieved using Labview programming.
Abstract:
In this project, an experimental setup has been established to collect data for the purpose of human operator modeling. The experimental setup consists of data acquisition hardware, NI Compact Vision system and Labview software. Both linear and intelligent models have been obtained using the data acquired from different human operators. By the use of this novel modeling approach, successful models in context of both generalizing capability and prediction characteristics have been achieved. Models have been used to replace the human operator as controller in the experimental setup.
Introduction:
Human-machine interaction is one of the most attractive areas for researchers to achieve different goals. The control actions of human operators have been intensively studied since 1940s to compare pilot ratings, develop new control methods based on human response, improve vehicle handling quantities, etc. In order to obtain data from human operators for identification process, an experimental setup has been designed and constructed. There are two critical points in designing the experimental setup; (i) the limitations of the human operator have to be considered and (ii) since the aim is modeling the control actions of the human operator during a control task, a nonlinear system has been chosen which is relatively difficult to control. Therefore, the control task is established as the tip position control of a single flexible arm.
The hardware consists of a flexible arm, a DC motor with gearbox, a DC motor driver circuit, a data acquisition card, a joystick, a camera and a PC. The flexible arm has been made of plexiglass having the dimensions of 3 cm x 50 cm x 2 mm. After some trial and error, adding a small amount of tip weight have resulted oscillations with satisfactory amplitude and damping. A dry friction element has been placed at the base to stabilize the tip position. An electric motor has been mounted on the base of the flexible arm. The driver circuit consists of an H bridge having 4 MOSFETs and isolation elements to protect the PC. The data acquisition card, PCIe-6259, has been used to generate pulse width modulation (PWM) signals according to the data read from the joystick using the same DAQ card. The duty cycle of the generated PWM signal is directly proportional to the displacement of the joystick on horizontal axis and therefore is approximately directly proportional to the base rotational speed of the flexible arm. The human operator uses the joystick to rotate the base end of the flexible arm. A camera has been placed about two meters above the flexible arm to capture the video of the flexible arm for image processing. A white circle has been placed on the tip of the flexible arm to accelerate the extraction of the tip position by image processing. Operator does not observe the flexible arm directly; instead, he observes the tip position of the arm from the video captured by the camera on the computer screen and manipulates the joystick to follow the reference trajectory to be followed superimposed on the image.
Data acquisition and image-processing cause high computational load. Therefore, National Instruments© LabVIEW® software has been used.
The schematic diagram of the whole system is shown in Figure 1.
Figure 1. Schematic diagram of the whole system.
The Labview software enabled the development of visual interface between the operator and the system to be controlled
The operator has been told to track the tip reference trajectory as fast as possible without oscillating the tip too much within 180 of workspace. In order to emphasize the importance of the control task, a scoring system has been developed. The scores have not been used as data during the modeling process. The operator interface was kept as simple as possible not to distract the operator and
it is shown in Figure 2.
Figure 2. Operator Interface
The background, the flexible arm and the tip were covered with black, orange and white respectively to ease the image processing. The white semicircle and the “+” representing the rotation axis of the arm are imposed on the gathered video by image processing. The reference interval that the operator should place the tip of the arm in is demonstrated by two small yellow lines on the semicircle. The interface has a score counter (top left) that counts backwards from a value until the user successfully positions the tip of the arm in the interval. There is an LED indicator that turns on to indicate a successful completion of the positioning task (top middle). There is also a total score indicator that shows the score accumulated up to that time. There is a status indicator that has explanatory information on the status of the program. A data acquisition session can be started by the experimenter by hitting the start button. Exit button exits the program.
On-line identification has also been achieved by using Labview
Different mathematical models have been obtained by off-line calculations using collected data. Promising models have been used as controllers to replace human operator. Both linear parametric models and intelligent models have been successful replacing the operator. Using the mathematical models, 5-step ahead prediction of human operator actions have also been obtained during real-time manual operation. Parameters in selected models have been tried to be identified on-line during manual operation in the most recent stage of this reserach.
The Labview made seamless integration of hardware, software and vision systems possible.
This research yielded very good theoretical results including 2 master thesis and several publications.
