Оbfuscated Мalware Detection Model

Authors

  • Timur V. Jamgharyan National Polytechnic University of Armenia
  • Vaghashak S. Iskandaryan National Polytechnic University of Armenia
  • Artak A. Khemchyan National Polytechnic University of Armenia

Keywords:

Obfuscation, Reverse engineering, Data flow, Convolutional neural network, Machine learning, Clustering, IDA Pro, Mean shift

Abstract

The paper presents the research results on the detection of obfuscated malware using a method based on mean shift. The research aimed to train neural networks included in the intrusion detection system to detect obfuscated malware. Detection of obfuscated malware using deterministic obfuscators is also discussed. Software solutions Dotfuscator CE, Net Reactor, and Pro Guard were used as deterministic obfuscators. Athena, abc, cheeba, dyre, december_3, engrat, surtr, stasi, otario, dm, v-sign, tequila, flip, grum, mimikatz were used as test malware. The results were verified using the IDA Pro tool and various intrusion detection systems. Process modeling was carried out in the Hyper-V virtual environment.

References

Microsoft official website https://learn.microsoft.com/ru-ru/azure/machine-learning/concept-deep-learning-vs-machine-learning?view=azureml-api-2

Yu. Livshits, laboratory of mathematical logic at PDMI, “Obfuscation of programs", 2004. https://logic.pdmi.ras.ru/~yura/of/survey1.pdf

B. Barak, O. Goldreich, R. Impagliazzo, S. Rudich, A. Sahai, S. Vadhan and K. Yang. On the (im) possibility of obfuscating programs», Advances in Criptology Crypto 2001, LNCS 2139, pp. 1-18, Springer-Verlag, 2001.

C. Collberg. JasvirNagra Surreptitious Software: Obfuscation, Watermarking, and Tamperproofing for Software Protection. Addison-WesleyProfessional. Pub. Date: July 24, 2009. Print ISBN-10: 0-321-54925-2

C. Wang, J. Hill, J. Knight and J. Davidson, “Software Tamper Resistance: Obstructing Static Analysis of Programs”. Technical Report. University of Virginia, Charlottes ville, VA, USA., 18 p., 2000.

K. Monappa, “Learning Malware Analysis”, Packt, Birmingem-Mumbai, 453 p., 2019.

Official website of the Java programming language obfuscator, Zelix. https://www.zelix.com/

M. S. Karvandi et al., “The Reversing Machine: Reconstructing Memory Assumptions”, https://doi.org/10.48550/arXiv.2405.00298

C. Patsakis, F. Casino, N. Lykousas, “Assessing LLMs in Malicious Code Deobfus-cation of Real-world Malware Campaigns”, https://doi.org/10.48550/arXiv.2404.19715

S. Hasan, A. Dhakal, “Obfuscated Malware Detection: Investigating Real-world Scenarios through Memory Analysis”, https://doi.org/10.48550/arXiv.2404.02372

V. Eliseev, “ Artificial neural networks as a mechanism for obfuscation of calculations”, https://doi.org/10.17223/2226308X/12/46

J. Kornblum, “Identifying almost identical files using context triggered piecewise hashing”, Digital Investigation, Volume 3, Supplement, pp. 91-97, 2006, https://doi.org/10.1016/j.diin.2006.06.015

L. Chen and G. Wang, “An Efficient Piecewise Hashing Method for Computer Forensics”, First International Workshop on Knowledge Discovery and Data Mining (WKDD 2008), Adelaide, SA, Australia, pp. 635-638, 2008, https://doi.org/10.1109/WKDD.2008.80

V. Roussev, "Building a Better Similarity Trap with Statistically Improbable Features” 2009 42nd Hawaii International Conference on System Sciences, Waikoloa, HI, USA, pp. 1-10, 2009, https://doi.org/10.1109/HICSS.2009.97

M. Alyami, A. Alghamdi, M. Alkhowaiter, C. Zou, Y. Solihin, “Random Segmentation: New Traffic Obfuscation against Packet-Size-Based Side-Channel Attacks”, https://doi.org/10.48550/arXiv.2309.05941

I. Nunes, S. Hwang, S. Jakkamsetti, G. Tsudik, “Privacy-from-Birth: Protecting Sensed Data from Malicious Sensors with VERSA”, https://doi.org/10.48550/arXiv.2205.02963

M. Rosen, J. Parker, A. Malozemoff, “Balboa: Bobbing and Weaving around Network Censorship” https://doi.org/10.48550/arXiv.2104.05871

Liang Wang, Hyojoon Kim, Prateek Mittal, Jennifer Rexford, “Programmable In-Network Obfuscation of Traffic”, https://doi.org/10.48550/arXiv.2006.00097

Y. Cheng,”Mean shift, mode seeking, and clustering”, IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 17, no. 8, pp. 790-799, Aug. 1995, https://doi.org/10.1109/34.400568

T. V. Jamgharyan, “Research of Obfuscated Malware with a Capsule Neural Network”, Mathematical Problems of Computer Science, 58, 67–83, 2022. https://doi.org/10.51408/1963-0094

T. V. Jamgharyan, “Modernization of Intrusion Detection System using Generative Model”, Defense-Academic journal, National Defense Research University, Haykakan Banak (Armenian Army), 2(108), pp․ 69-79, 2021, https://razmavaraget.files.wordpress.com/2022/01/hb2-final.pdf

Malware Bazaar Database. [Online]. Available https://bazaar.abuse.ch/browse/

Malware database. [Online]. Available http://vxvault.net/ViriList.php

A free malware repository for researches. [Online]. Available https://malshare.com/

Malware repository. [Online]. Available https://avcaesar.malware.lu/

Malware repository. [Online]. Available https://www.virusign.com/

Viruses repository. [Online]. Available https://virusshare.com/

T. V. Jamgharyan, A.A.Khemchyan, “Malware Obfuscation Model Using Machine Learning”, Bulletin Of High Technology, N3 (31), pp. 77-83, 2024. https://doi.org/10.56243/18294898-2024.3-77

T. V. Jamgharyan, T. N. Shahnazaryan, “A Studu of a Model of Neural Network Application in the Decoy Infrastructure in the Defence Sphere”, Defence-Academic journal, National Defence Reseach University, Haykakan Banak (Armenian Army), 2(112), pp. 71-83, 2024. DOI: 10.61760/18290108-ehp24.2-71

All research results available on https://github.com/T-JN

Downloads

Published

2024-12-01

How to Cite

Jamgharyan, T. V., Iskandaryan, V. S., & Khemchyan, A. A. (2024). Оbfuscated Мalware Detection Model. Mathematical Problems of Computer Science, 62, 72–81. Retrieved from http://mpcs.sci.am/index.php/mpcs/article/view/862