European manufacturing represents approximately 22 percent of the GNP of the EU and provides jobs for 34 million in 230 000 enterprises. This part of our economy is under competitive pressure and increased skills and flexibility of the workers is an important countermeasure, says SINTEF researcher and Project Manager Geir K. Hanssen.
In partnership with industry the EU-funded research project HuMan has spent EUR 4.5 million in developing new technology that enable manufacturing workers to “put on” extra muscle power and adopt new cognitive skills. The combination of machine learning, super vision, known among researchers as “augmented reality”and robot technology will equip European companies to compete with low-cost countries such as China and India.
All innovations resulting from the HuMan project have been achieved with close involvement from those who intend to apply the technology.
User instructions before your very eyes
In a meeting room at SINTEF in Trondheim, Norway, we find researcher Manuel Oliveira and Project Manager Geir K. Hanssen ready with a demonstrator. Oliveira brings out a big case containing a hololens: This is a computer built into a pair of glasses that provides you with a hologram vision of a virtual reality, mixed with what your eyes can actually see. The hololens-system itself is off-the-shelf technology, but the software named KIT (Knowledge In Time) which is the software it contains, has been developed at SINTEF and a recently launched spin-off company.
KIT is equipped with in-built sensors: a camera and a “brain” that contains a wealth of knowledge about industrial processes, materials, design and production methods. KIT can also receive commands from the user.
So now it’s time for the journalist to have a go. I am putting on what appears to be a helmet-inspired set of 3D glasses. At once I can see a three-dimensional model of a big pillar that has previously been placed in front of me in the room. The pillar is used in the manufacture of industrial robots. As well as the 3D model of the pillar, I can see some graphics, arrows and text that describe how the cylinder can be mounted on a frame as part of the next stage of the assembly process. The whole experience seems quite unreal, but the process seems easy.
This is almost what it looks like to use KIT. The system will take the order “next” to apply grapics and instructions just before your eyes. Illustration by KIT-AR
“You can now give the voice- command “KIT next”, says Oliveira.
I do what he says and KIT obeys immediately, showing me text and arrows that explain the next stage in the process. So, although I am untrained and entirely without skills or instructions, I am guided through the process of assembling the high-tech component entirely at my own pace.
It is easy to imagine how this technology can be of great assistance in the manufacture of everything from aircraft engines to furniture and, not least, in the training of skilled workers and to teaching in other settings.
Solutions, innovations and products resulting from the project:
- Exo-skeleton for arms
- Exo-skeleton for legs
- Workplace Optimization Service (for the planning of optimal workplaces)
- Operator Awareness and Support Tool (guidelines for production workers)
- Shopfloor Insight Intelligence (for production data analysis)
- KIT (an augmented reality system with worker instructions and guidelines)
- HuMan intervention system (the monitoring of biometric data and guidelines based on machine learning)
“The aim of the HuMan project has been to enhance the cognitive and physical capability of the worker. We looked at applying “robot technology” to provide support for workers carrying out a variety of processes”, says Hanssen.
“One aspect of this is the so called “exo-skeleton”. Exo-skeletons are wearable high-tech suits that enable users to perform better and safer: The results of this project include an upper body exoskeleton that is currently being piloted at one of Airbus’ factories. A lower body reinforcement device is tested by the Spanish furniture manufacturer Royo”, he says.
To date, the project has also delivered smart reality-systems that can assist workers to find correct components or tools in large and complex manufacturing facilities. They have also developed a tool called SII (Shopfloor Insight Intelligence) that can be used to analyse manufacturing processes as a basis for improvements.
This is how the SII analysis tool looks like. It provides insights into the work processes that are supported by KIT. The work process is visualised in form of an automatically discovered process map depicting the actual sequence of steps workers performed. In the performance insights module, for each step detailed information is display about, for example, the variations in the time within which the step was performed, how often the step had to be repeated, or how the processing time varied over time. Several further modules give insight into training or workers, quality control steps and much more.
“KIT and the integrated SII tool represent our Norwegian contribution”, says Manuel Oliveira, who is a researcher at SINTEF and CEO at the recently launched company KIT-AR.
Millions from investment funds
Recently, the start up received proof of concept funding from SINTEF Venture V AS, University College London Tech Fund Ltd. and from the London Co-Investment Fund.
Facts:
- Funding from Horizon2020: EUR 4 million
- Start/end: Oct. 2016 – Sept. 2019
- 13 partners from six countries
- Three users/case studies: Airbus (Spain), Royo (Spain) and Comau (Italy)
- Coordinated by SINTEF
- Two patents
- One new company
“The technology is called Augmented Reality because the user’s true vision of an object is mixed with graphics and information about it”, says Oliveira, who currently commutes between University College London and SINTEF in Trondheim in order to assist with the launch of the technology into the global market.
“We believe that we have strong interest in the Norwegian market, which must be innovative to remain competitive in the global market, thus solutions such as KIT enable Norwegian companies to be successful and strive”, says Oliveira.
University College London is also contributing to the start-up KIT-AR, and is playing an important role in software development.
Robot production assisted by KIT in Torino, Italy. Photo: Thor Nielsen
Quality assurer and HSE Manager all rolled into one
KIT can assist industrial workers in completing complex tasks. However, the advanced technology that mesures heart rate, breathing rate, eye gaze, head orientation and video-recording, is also used to gather useful data from the work processes taking place in a manufacturing facility”, says researcher and co-founder Felix Mannhardt.
Mannhardt is responsible for the SII analysis tool that is also linked to the system. The technology reveals where problems may exist in the process and thus provides clear input to workflow and potential improvements in the facility’s work procedures. This is useful information in a world where the aim is to carry out all production as seamlessly as possible, and in which we are potentially competing with cut-price manufacturers based in low-cost countries.
SII is also undergoing further development at the spin-off company KIT-AR. SII’s function is based on data gathered both from workers’ experiences using KIT and from facilities’ process centres.
But the HuMan project also offers solutions for the HSE sector. By attaching a small sensor to factory workers’ bodies, the system is able to record if they are being subjected to tasks that are too difficult or strenuous for them or, for example, if the temperature in their working environment is too high or too low.
If anything happens, the system will send a signal to the worker via a smart watch and provide useful advice, such as to put on an exo-skeleton or take a break.
All innovations resulting from the HuMan project have been achieved with close involvement from those who intend to apply the technology.
“In technical jargon, this is called co-creation”, explains Project Manager Geir K. Hanssen. “To date the project has generated 13 scientific articles and two patents, as well as the spin-off company KIT AR, which will be taking KIT and SII out into the global market”, he says.
