Web Application for Monitoring and Anomalies Detection in Orchestration-based Integration Applications

Companies which provide digital services for implementation of business cases based on data extraction, transformation and processing are needed in real-time monitoring tools to immediately react on every strange case of system behavior. This work is focused on full cycle activities for creating and support of complex integration routes which are chain of blocks which manipulate pieces of data, using modern Web UI. To assist user in making a correct choice of components for integration, module Decision Wizard is used, that is based on expert assessments by various criteria. After routes are launched (Apache Camel is used as an integration engine), user wants to be sure that there are no environmental or other problems at the current moment and in future. The key point is if everything works right now it doesn’t mean that there are no symptoms (as a rule they are hidden) of problems. For this purpose, system constantly collects metrics from various sources (CPU, RAM usage, number of failed messages, and so on) and checks their values fit into limits defined by user – alert rules, which are conditions like “If CPU usage was more than 90% for more than 8 seconds, then it is anomaly”. Due to deep integration with Apache Camel, information about anomaly contains supposed source of anomaly (part of integration chain) and data which was being processed at that time. Speaking about technical details, on backend Influx stack was used for event aggregation and anomaly detection, Kafka is for event queue, and on frontend Vue.js is used to render Web UI.