Human Machine Interface Augmented


Progetto di ricerca aziendale




Realizzazione di un'app mobile per l'assistenza agli operatori durante le ordinarie operazioni di manutenzione

Augmented reality is an interactive graphics system that allows to intervene on a multimedia stream representing reality by adding, at the same time, virtual contents and animations.

Maintenance and repair activities represent one of the major fields of application for augmented reality: the opportunities they offer are innumerable and the possibility of taking full advantage of this technology is of increasing interest.

On the other hand, the benefits that derive from its application are several. Among all, the reduction in the execution times of maintenance activities and the reduction of human errors.

Requirement to satisfy

Both technological systems and industrial products require global, rapid and appropriate maintenance services. However, these operations can be subject to several difficulties, some listed below. 

  • INAPPROPRIATE LEVEL OF OPERATORS COMPETENCE: often, the level of competence of operators in the field is not up to par. Due to the continuous technological change, the staff required an equally continuous updating. Due to a service manual not always available or easy to understand, the personal upgrade becomes increasingly difficult;
  • INTERACTION WITH UNKNOWN OBJECTS: maintenance activities in complex systems include a wide range of operations which, not infrequently, also includes interaction with unknown objects, randomly distributed in the operating space;
  • UNCOMFORTABLE ENVIRONMENTS: the maintenance worker may find himself operating in uncomfortable environments or in which mobility is reduced.

The prototype created has the purpose of mitigating, if not solving, some of the critical issues described, providing the maintenance technician with a control tool that can be used through smart devices (whether they are mobile, such as smartphones and tablets, or wearables, such as smart-glasses and smartwatches) and easily integrable into existing industrial systems.

Our proposal

To pursue the final purpose, we have created a system that allows interfacing with the field without any additional hardware request, and minimizing changes to the existing  PLC software.

To allow this, the use of the SNAP 7 communication libraries, which have made it possible to flexibly create a complete remote access solution to the system control PLC.

System functioning 

The designed system allows to uniquely identify a device, monitor its operation and control it: all directly in the field through the use of a smart device.

By connecting to the same sub-network of the PLC via Wi-Fi, I.U.M.A collects and re-elaborates the data, then to create a data interface on the  devices accessible to the operator.

The recognition of the devices is carried out by reading specific QR-Codes. After identification, the operator has access to the supervision page where he can monitor or control the device. If necessary, it can also access the documentation associated with it.

All this takes place in total safety: operations are only allowed if the device or system to be supervised is put into manual control mode. Otherwise, the device returns an alert and does not allow to continue the supervision operations.


With I.U.M.A, the operator has at his disposal a tool that allows him to easily and immediately access information regarding the devices that make up the system, wherever he is and without the need to reach dedicated workstations.

Thanks to the command window, in case of need, the operator can perform elementary operations that allow him to immediately test the operation of the device.

Finally, with the Docs section, he always has at hand all the documentation necessary to complete the maintenance operation in the best way.