Technical Field

The invention belongs to the technical field of medical device, involving intelligent nursing technology, in particular to a robot nursing system based on cooperative control of visual and thought signals.

Background Technology

Just as industry and manufacturing are rushing to automate, robots have been seen as a promise to "save" the disabled. In recent years, artificial intelligence has ushered in a new wave of product application with the breakthrough of deep learning algorithm. In the context of insufficient supply and unbalanced distribution of medical resources, artificial intelligence has landed in various subdivisions of medical and health care, covering all application scenarios of the whole industrial chain. As the market demand is becoming urgent, nursing robot is bound to become the pioneer of service robot comprehensive marketization.

At the same time, with the development of the country, the society gradually aging. The demand for medical treatment, nursing and rehabilitation increases, due to the serious imbalance of the doctor-patient ratio in the domestic medical system. In the ward, many patients with mobility difficulties and no autonomous ability need special nursing staff to take care of: feeding water, medicine and so on. As a result, there is not enough nursing staff to meet the basic needs of every patient.

At present, the intelligent nursing system cannot realize the autonomous nursing of nursing equipment, and special nursing staff is needed to carry out nursing work, which cannot effectively liberate nursing staff and solve the tense situation of nursing staff.

Contents of the Invention

Aiming at the problems existing in the prior art, the invention provides a robot nursing system based on the cooperative control of vision and idea signal. By extracting the brain's idea signal and combining machine learning, the intelligent nursing function with weak supervised learning effect can be achieved. It solves the common needs of both doctors and patients, liberates the human resources of the hospital and improves the working efficiency of hospital nursing.

The robot nursing system based on the collaborative control of vision and idea signals is characterized by: head-mounted brainwave acquisition device, Kinect camera, display, robot arm, idea signal processing module, image processing module, face mouse control module and robot arm control module;The idea signal processing module is used to collect the brainwave signal and wink emg signal of the nursing object in real time, extract the concentration value in the brainwave signal, and judge whether the nursing staff is in the state of concentration.

The image processing module is used to collect and process RGB images and depth images in real time, extract key points of the face, identify the tension-closing state of the mouth, and determine the two-dimensional coordinates of the mouth and the depth of the key points of the mouth;

The face mouse control module, based on the nose tip movement the object control the mouse pointer on the liquid crystal display screen, to achieve the selection of required items, and use the wink emg which is detected by the idea signal processing module to determine whether to blink to select items;

The manipulator control module activates the action of the robot arm based on the attention state of the caregiver determined by the idea signal processing module and implements the grasping action according to the selected items of the face mouse control module. The object is sent to the mouth of the caregiver according to the two-dimensional coordinates of the mouth processed by the image processing module and the three-dimensional coordinates determined by the key point depth of the mouth.

Advantages of the Invention

1. The invention uses the idea signal as the starting signal and the visual feedback to ensure the operating accuracy of the system. The RGB image is combined with the depth image to capture the face in real time and output the mouth data, which has a higher overall accuracy than the existing nursing robot system.

2. The face mouse technology used in the invention provides great convenience for patients. Patients only need to move the nose tip to control the mouse to select the items they need, which reflects the characteristics of humanization and intelligence.

3. The invention can realize the autonomous fetching and feeding of edible goods according to the independent consciousness and needs of the caretakers, reducing the work intensity of the caretakers to a certain extent, liberating the caretakers, and relieving caretakers stress.

4. At present, most service robots operated by thought are based on brain-computer interface technology. However, in order to obtain more accurate "idea", we have high hardware requirements. The cost is expensive and it is difficult to popularize in a short time. The invention only needs a low-cost brainwave headset to train concentration, is with simple operation and conducive to market popularization.