The Possibilities of Using Big Data Technologies in Solving Problems of Processing Data on Atmospheric Air Pollution
( Pp. 162-170)

More about authors
Bogomolov Dmitry N. graduate student, Department of Instrumental and Application Software
MIREA – Russian Technological University
Moscow, Russian Federation Plotnikov Sergey B. Cand. Sci. (Eng.); associate professor, Department of Instrumental and Application Software; MIREA – Russian Technological University; Moscow, Russian Federation
Abstract:
The main objective of the article is to substantiate the possibility of using Big Data technologies in the field of atmospheric air monitoring. In the form of a diagram, a model for processing big data obtained from measuring meteorological gas analysis stations using the PySpark library for further experimental studies is presented. The factors accompanying the use of Big Data in the field of atmospheric air monitoring are derived, and the performance of the Pandas and PySpark libraries is compared. The obtained results will allow us to further rely on the derived factors and use the most optimal data processing technologies to build predictive machine learning models in the field of analyzing the level of atmospheric air pollution. Consistent use of big data and machine learning methods will ensure clean and healthy air for future generations through more effective predictive analytics. This article is valuable for students and specialists in the field of information technology, in particular, in the field of data processing and machine learning.
How to Cite:
Bogomolov D.N., Plotnikov S.B. The Possibilities of Using Big Data Technologies in Solving Problems of Processing Data on Atmospheric Air Pollution. Computational Nanotechnology. 2024. Vol. 11. No. 1. Pp. 162–170. (In Rus.) DOI: 10.33693/2313-223X-2024-11-1-162-170. EDN: ELXNHB
Reference list:
Azemov D.T. Assessment of the quality of atmospheric air in St. Petersburg based on the results of the operation of the automated air monitoring system in 2019. In: Modern problems of hydrometeorology and environmental monitoring in the CIS: A collection of abstracts of the International Scientific and Practical Conference dedicated to the 90th anniversary of the Russian State Hydrometeorological University (St. Petersburg, October 22–24, 2020). St. Petersburg: Russian State Hydrometeorological University, 2020. Pp. 103–104. EDN: BTHHCS.
Borisov I.D., Semenov V.A., Bychkova Ya.A., Zhao M.N. Apache Spark и Pyspark. In: Young Russia: Collection of materials of the XIV All-Russian scientific and practical conference of young scientists with international participation (Kemerovo, April 18–21, 2023). Kemerovo: Kuzbass State Technical University named after T.F. Gorbachev, 2023. Pp. 31603.1–31603.3. EDN: UMSNKI.
Bosubabu S. Air pollution monitoring and prediction system using the Internet of things. International Journal of Research and Development in Engineering Sciences. 2020. Vol. 2. Issue 3. Pp. 144–150.
Vinogradova E.A., Dmitriev M.M., Kudryavets A.S. Air eco-monitoring using technological solutions to process data from automatic online air quality monitoring systems using machine learning, artificial intelligence and big data analytics approaches. In: Modern technologies: Problems and development trends: Monograph. Petrozavodsk: International Center for Scientific Partnership “New Science” (Individual Entrepreneur Ivanovskaya I.I.), 2021. Pp. 174–189. EDN: FKJEWT.
Gusak D.V. Device concept for organization of monitoring network. Bulletin of the Peoples’ Friendship University of Russia. Series: Ecology and Life Safety. 2023. Vol. 31. Issue 2. Pp. 241–250. (In Rus.) DOI: 10.22363/2313-2310-2023-31-2-241-250. EDN: HCKTOL.
Egorov G.G. The use of big data technology for environmental protection. In: Ecophilosophy in the design of a noospheric city: A collection of articles based on the results of the Third Russian Round Table with international participation (Moscow, May 18, 2023. E.V. Barkova, O.M. Buzskaya (eds.). Moscow: Ruscience Limited Liability Company, 2023. Pp. 42–48.
Igonina E.I. Application of machine learning for clustering of Russian regions on population health and ecology. In: I. Lipanov’s scientific readings: Materials of the regional scientific conference (Izhevsk, June 15–16, 2021). Izhevsk: Izhevsk State Technical University named after M.T. Kalashnikov, 2021. Pp. 169–175. EDN: HURYBU.
Johansson C., Zhang Z., Engardt M. et al. Improving 3-day deterministic air pollution forecasts using machine learning algorithms. Atmos. Chem. Phys. Discus. 2023. DOI: 10.5194/acp-2023-38.
Kaplenkova P.A., Sivova A.N. Prediction of atmospheric air pollution using machine learning and PySpark. Science and Business: Ways of Development. 2020. No. 10 (112). Pp. 54–56. (In Rus.) EDN: LTZEYE.
Kostromin N.S., Sivova A.N. Application of machine learning methods for solving environmental problems. Modern Science. 2019. No. 5-3. Pp. 144–148. (In Rus.) EDN: YLPWAT.
Nandi B.P., Singh G., Jain Tayal D.K. Evolution of neural network to deep learning in prediction of air, water pollution and its Indian context. Int. J. Environ. Sci. Technol. 2023. DOI: 10.1007/s13762-023-04911-y.
Panarin V.M., Maslova A.A., Savinkova S.A. Automated monitoring of atmospheric air pollution in industrially developed territories. Tula: Tula State University, 2021. 219 p. ISBN: 978-5-7679-4817-8. EDN: ZDKTXH.
Parkavi P., Rathi S. Deep learning model for air quality prediction based on big data. International Journal of Scientific Research in Computer Science, Engineering and Information Technology (IJSRCSEIT). 2021. Vol. 7. Issue 3. Pp. 170–175. ISSN: 2456-3307. DOI: 10.32628/CSEIT217332.
Samad A., Garuda S., Vogt U., Yang B. Air pollution prediction using machine learning techniques – an approach to replace existing monitoring stations with virtual monitoring stations, atmospheric environment. International Journal of Computational Intelligence Studies. 2023. Issue 310. P. 119987. ISSN: 1352-2310. DOI: 10.1016/j.atmosenv.2023.119987.
Sossi Alaoui S., Aksasse B., Farhaoui Y. Air pollution prediction through internet of things technology and big data analytics. International Journal of Computational Intelligence Studies. 2020. No. 8. P. 177. DOI: 10.1504/IJCISTUDIES.2019.102525.
Shih D.-H., To T.H., Nguyen L.S.P. et al. Design of a spark big data framework for PM2.5 air pollution forecasting. Int. J. Environ. Res. Public Health. 2021. Vol. 18. No. 7087. 15 p. DOI: 10.3390/ijerph18137087.
Keywords:
big data, data processing, atmospheric air monitoring, pollution forecasting.


Related Articles

System analysis, management and information processing Pages: 9-15 Issue №12384
DEVELOPMENT OF A MATRIX REPRESENTATION OF GENERALIZED GRAPH STRUCTURES IN PROBLEMS OF BIG DATA DESCRIPTION AND ANALYSIS
gradostroitelnaya modeling matrix representation hypergraph metagraf big data
Show more
I. Political Science. @@1. Theory and philosophy of politics, history and methodology Pages: 32-37 Issue №15447
SOCIAL-POLITICAL BIG DATA CHALLENGES
digital society digitalization Big Data social-political studies political technologies
Show more
Mathematical and Software of Computеrs, Complexes and Computer Networks Pages: 83-91 DOI: 10.33693/2313-223X-2023-10-3-83-91 Issue №23683
Determination of Parameters of Hidden Threats of Early Detection in Information Systems for Machine Learning Tasks
Anylogic machine learning corporate information systems (CIS) simulation modeling data analysis
Show more
3. ACCOUNTING, STATISTICS 08.00.12 Pages: 72-77 DOI: 10.33693/2541-8025-2021-17-5-72-77 Issue №19821
Problems of Application of Information and Communication Technologies in Tax Control and their Improvement
tax control information and communication technologies digitalization blockchain big data
Show more
4. MATHEMATICAL AND INSTRUMENTAL METHODS OF ECONOMICS 08.00.13 Pages: 76-80 Issue №16787
Innovative development of banking infrastructure
banks information technology outsourcing mobile banking automated banking systems
Show more
4. ACCOUNTING, STATISTICS 08.00.12 Pages: 98-103 Issue №17214
Audit in the context of digitalization of the Russian economy: risks, opportunities, and limitations
outsourcing big data integrated reporting subject area risk-based concept
Show more
3. CIVIL LAW; ENTREPRENEURIAL LAW; FAMILY LAW; INTERNATIONAL PRIVATE LAW Pages: 82-85 Issue №13507
PROSPECTS FOR THE DEVELOPMENT OF STATE REGULATION OF FINANCIAL AND CREDIT RELATIONS IN THE RUSSIAN FEDERATION
Big Data
Show more
8. CIVIL LAW, BUSINESS LAW, FAMILY LAW; PRIVATE INTERNATIONAL LAW 12.00.03 Pages: 132-138 Issue №19821
The Legal Regime of Protection of Big Data as Part of Artificial Intelligence Systems
big data artificial intelligence database object of related rights personal data
Show more
1. INTERNATIONAL LEGAL FORUM "VECTOR OF LEGAL THOUGHT DEVELOPMENT: BUSINESS, ECONOMY, FINANCE IN THE LANGUAGE OF LEGISLATION" DEDICATED TO THE 100 ANNIVERSARY OF THE FINANCIAL UNIVERSITY, DECEMBER 4, 2018 Pages: 93-96 Issue №14165
ACTUAL ISSUES OF BIG DATA APPLICATION: LEGAL ASPECT
Big data
Show more
5.2.2. MATHEMATICAL, STATISTICAL AND INSTRUMENTAL METHODS OF ECONOMICS Pages: 150-155 Issue №20468
About Some Specific Limitations of Data Mining Application
Big data information technology data mining classification clustering
Show more