XLCNN: pre-trained transformer model for malware detection

Abstract

The present thesis describes a Transformer-based neural network model that was developed in order to detect malicious software. We believe that the scientific community should take advantage of the contribution of Transformer models in the field of cybersecurity and go beyond the limits set by the classic natural language processing. For this purpose a new and more sophisticated algorithm was created based on the methodology used by the XLNet neural network which was proposed by the Google AI Brain Team. The proposed XLCNN model detects malicious code with a higher success rate than its predecessor. The method of detecting malware is based on the extraction and analysis of metadata contained in Windows executable files. From the experiments carried out, it was found that the size and architecture of the feed-forward neural network in combination with the size of its input is one of the most important factors of XLCNN for classification problems. To justify proving the concept of XLCNN as an effective approach to detecting malware, the success rate of the algorithm was measured for a finite number of epochs compared to XLNet using exactly the same parameters as the same inputs. Using this network has proven to be not only a reliable way for security researchers to detect malware, but also an effective and highly accurate method that offers high accuracy of 95.07%.