Architecture of a Multimodal Data Processing Constructor: Graph Models and Their Application in Information Security Tasks
( Pp. 55-67)
More about authors
Charugin Valery V.
lecturer, Department of Computer and Information Security, Institute of Artificial Intelligence, @mirea
MIREA – Russian Technological University
Moscow, Russian Federation Charugin Valentin V. lecturer, Department of Computer and Information Security, Institute of Artificial Intelligence; MIREA – Russian Technological University; Moscow, Russian Federation Stavtsev Alexey V. Cand. Sci. (Phys.-Math.); associate professor, Department of Computer and Information Security, Institute of Artificial Intelligence; MIREA – Russian Technological University; Moscow, Russian Federation@mirea Chesalin Alexander N. Cand. Sci. (Eng.), Associate Professor; Head, Department of Computer and Information Security, Institute of Artificial Intelligence; MIREA – Russian Technological University; Moscow, Russian Federation
MIREA – Russian Technological University
Moscow, Russian Federation Charugin Valentin V. lecturer, Department of Computer and Information Security, Institute of Artificial Intelligence; MIREA – Russian Technological University; Moscow, Russian Federation Stavtsev Alexey V. Cand. Sci. (Phys.-Math.); associate professor, Department of Computer and Information Security, Institute of Artificial Intelligence; MIREA – Russian Technological University; Moscow, Russian Federation@mirea Chesalin Alexander N. Cand. Sci. (Eng.), Associate Professor; Head, Department of Computer and Information Security, Institute of Artificial Intelligence; MIREA – Russian Technological University; Moscow, Russian Federation
Abstract:
The increasing volume and heterogeneity of data in the modern digital environment introduce additional challenges to the development of systems for information analysis and security threat detection. Conventional processing approaches lack the flexibility required to adapt to evolving conditions, growing system requirements, and the need for integrated analysis across heterogeneous data sources. This creates a demand for a multimodal data processing constructor capable of dynamically assembling analytical workflows and orchestrating the operation of diverse processing modules. To ensure the correctness and predictability of such a system, it is necessary to formalize its internal processes using graph-based models, which provide a structured representation of computational pipelines, data-flow dependencies and operation-execution rules. The purpose of the study: to develop an architecture for a multimodal data-processing constructor based on the application of graph models. The proposed architecture is intended to ensure consistent processing of multiple data modalities and to manage the execution sequence of operations within complex analytical workflows. The research methodology is a combined formalization based on integrating a directed acyclic graph, a colored Petri net, and a state transition graph into a unified formal model that describes the structure, data flows, and operational logic of the multimodal processing constructor. Using the constructed model, an analytical workflow is simulated to evaluate the consistency of processes and the correctness of modality interactions. Research results: formation of a formal architecture for a multimodal data-processing constructor based on graph models, representing the structure of analytical workflows, data flows, and component behavior. The conducted simulation demonstrated that the system ensures stable modality interaction and correct execution of operations.
How to Cite:
Charugin V.V., Charugin V.V., Chesalin A.N. and Stavtsev A.V. Architecture of a multimodal data processing constructor: Graph models and their application in information security tasks. Computational Nanotechnology. 13, 1 (2026), 55–67. DOI: 10.33693/2313-223X-2026-13-1-55-67. EDN: MMLKSN
Reference list:
Babkin A.N., Akchurina L.V., Alekseenko S.P. Modeling of threat of information attacks in the Internet network based on Petri nets. Bulletin of Voronezh Institute of the Ministry of Internal Affairs of Russia. 2023. No. 2. Pp. 101–106. (In Rus.)
Fedoseev A.I., Ponomareva L.A., Zabolotnikova V.S. Modeling the educational process in the notation of Petri nets for making effective management decisions. Applied Mathematics and Management Issues. 2024. No. 2. Pp. 76–87. (In Rus.)
Charugin V.V., Charugin V.V., Chesalin A.N. Multimodal data processing constructor. In: Innovative, information and communication technologies (INFO-2024). 2024. Pp. 146–150.
Charugin V.V., Charugin V.V., Chesalin A.N., Ushkova N.N. Natural language processing block constructor and its application in the problem of log structuring in information security. International Journal of Open Information Technologies. 2024. Vol. 12. No. 9. Pp. 111–118. (In Rus.)
Charugin V.V., Chesalin A.N. Application of generative algorithms in information security problems. In: Fundamental, exploratory, applied research and innovative projects. Proc. of the National Scientific and Practical Conf. Moscow: Association of Graduates and Employees of the Zhukovsky Air Force Engineering Academy, 2023. Pp. 146–150.
Bland J.A., Petty M.D., Whitaker T.S. Machine learning cyberattack and defense strategies. Computers & Security. 2020. Vol. 92. Art. 101738. DOI: 10.1016/j.cose.2020.101738.
Cardoso J., Valette R. Petri nets. Florianopolis: EdUFSC, 2024. DOI: 10.34849/t30e-ax86.
Diallo O., Rodrigues J.J.P.C., Sene M. Performances evaluation and Petri nets. In: Modeling and simulation of computer networks and systems. Methodologies and applications. Burlington: Morgan Kaufmann, 2015. Pp. 313–355. DOI: 10.1016/B978-0-12-800887-4.00011-0.
Healy P., Nikolov N.S. How to layer a directed acyclic graph. Springer Nature, 2001. Vol. 2265. Pp. 16–30. DOI: 10.1007/3-540-45848-42. (Lecture Notes in Computer Science)
Herodotou H., Chen Y., Lu J. A Survey on automatic parameter tuning for big data processing systems. ACM Computing Surveys (CSUR). 2020. Vol. 53. No. 2. Art. 43. Pp. 1–37. DOI: 10.1145/3381027.
Jiao T., Guo C., Feng X. et al. A comprehensive survey on deep learning multi-modal fusion: Methods, technologies and applications. Computers, Materials & Continua. 2024. Vol. 80. No. 1. Pp. 1–35. DOI: 10.32604/cmc.2024.053204.
Ozkaya M.Y., Catalyurek U.V. A simple and elegant mathematical formulation for the acyclic DAG partitioning problem. ArXiv Preprint. 2022. DOI: 10.48550/arXiv.2207.13638.
Taser P.Y., Taser M.M. Multimodal machine learning in cybersecurity. Innovative Artificial Intelligence. 2025. Vol. 1. Pp. 47–55.
Thomas C., Cosme M., Gaucherel C. et al. Model-checking ecological state-transition graphs. PLoS Comput Biol. 2022. Vol. 18. No. 6. Art. e1009657. DOI: 10.1371/journal.pcbi.1009657.
Wang J. Petri nets. Handbook of finite state based models and applications. Boca Raton: CRC Press, 2019. DOI: 10.1201/9780429184185-8.
Wang J., Zhu T., Xiong C. MultiKG: Multi-source threat intelligence aggregation for high-quality knowledge graph representation of attack techniques. ArXiv Preprint. 2024. DOI: 10.48550/arXiv.2411.08359.
Yao Z., Zhang Y., Deng N. et al. Multimodal embeddings for representation learning. SSRN Electronic Journal. 2025. DOI: 10.2139/ssrn.5189627.
Zhang Y., Zhao X., Ma Y. et al. MM-AttacKG: A multimodal approach to attack graph construction with large language models. ArXiv Preprint. 2025. DOI: 10.48550/arXiv.2506.16968.
Zhao F., Zhang C., Geng B. Deep multimodal data fusion. ACM Computing Surveys. 2024. Vol. 56. No. 9. Pp. 1–36. DOI: 10.1145/3649447.
Fedoseev A.I., Ponomareva L.A., Zabolotnikova V.S. Modeling the educational process in the notation of Petri nets for making effective management decisions. Applied Mathematics and Management Issues. 2024. No. 2. Pp. 76–87. (In Rus.)
Charugin V.V., Charugin V.V., Chesalin A.N. Multimodal data processing constructor. In: Innovative, information and communication technologies (INFO-2024). 2024. Pp. 146–150.
Charugin V.V., Charugin V.V., Chesalin A.N., Ushkova N.N. Natural language processing block constructor and its application in the problem of log structuring in information security. International Journal of Open Information Technologies. 2024. Vol. 12. No. 9. Pp. 111–118. (In Rus.)
Charugin V.V., Chesalin A.N. Application of generative algorithms in information security problems. In: Fundamental, exploratory, applied research and innovative projects. Proc. of the National Scientific and Practical Conf. Moscow: Association of Graduates and Employees of the Zhukovsky Air Force Engineering Academy, 2023. Pp. 146–150.
Bland J.A., Petty M.D., Whitaker T.S. Machine learning cyberattack and defense strategies. Computers & Security. 2020. Vol. 92. Art. 101738. DOI: 10.1016/j.cose.2020.101738.
Cardoso J., Valette R. Petri nets. Florianopolis: EdUFSC, 2024. DOI: 10.34849/t30e-ax86.
Diallo O., Rodrigues J.J.P.C., Sene M. Performances evaluation and Petri nets. In: Modeling and simulation of computer networks and systems. Methodologies and applications. Burlington: Morgan Kaufmann, 2015. Pp. 313–355. DOI: 10.1016/B978-0-12-800887-4.00011-0.
Healy P., Nikolov N.S. How to layer a directed acyclic graph. Springer Nature, 2001. Vol. 2265. Pp. 16–30. DOI: 10.1007/3-540-45848-42. (Lecture Notes in Computer Science)
Herodotou H., Chen Y., Lu J. A Survey on automatic parameter tuning for big data processing systems. ACM Computing Surveys (CSUR). 2020. Vol. 53. No. 2. Art. 43. Pp. 1–37. DOI: 10.1145/3381027.
Jiao T., Guo C., Feng X. et al. A comprehensive survey on deep learning multi-modal fusion: Methods, technologies and applications. Computers, Materials & Continua. 2024. Vol. 80. No. 1. Pp. 1–35. DOI: 10.32604/cmc.2024.053204.
Ozkaya M.Y., Catalyurek U.V. A simple and elegant mathematical formulation for the acyclic DAG partitioning problem. ArXiv Preprint. 2022. DOI: 10.48550/arXiv.2207.13638.
Taser P.Y., Taser M.M. Multimodal machine learning in cybersecurity. Innovative Artificial Intelligence. 2025. Vol. 1. Pp. 47–55.
Thomas C., Cosme M., Gaucherel C. et al. Model-checking ecological state-transition graphs. PLoS Comput Biol. 2022. Vol. 18. No. 6. Art. e1009657. DOI: 10.1371/journal.pcbi.1009657.
Wang J. Petri nets. Handbook of finite state based models and applications. Boca Raton: CRC Press, 2019. DOI: 10.1201/9780429184185-8.
Wang J., Zhu T., Xiong C. MultiKG: Multi-source threat intelligence aggregation for high-quality knowledge graph representation of attack techniques. ArXiv Preprint. 2024. DOI: 10.48550/arXiv.2411.08359.
Yao Z., Zhang Y., Deng N. et al. Multimodal embeddings for representation learning. SSRN Electronic Journal. 2025. DOI: 10.2139/ssrn.5189627.
Zhang Y., Zhao X., Ma Y. et al. MM-AttacKG: A multimodal approach to attack graph construction with large language models. ArXiv Preprint. 2025. DOI: 10.48550/arXiv.2506.16968.
Zhao F., Zhang C., Geng B. Deep multimodal data fusion. ACM Computing Surveys. 2024. Vol. 56. No. 9. Pp. 1–36. DOI: 10.1145/3649447.
Keywords:
multimodal data processing, graph models, directed acyclic graph, colored Petri net, state transition graph, data-flow management, information security, process modeling.
