Distributed Intrusion Detection Systems in Big Data: A Survey

Hameed, Bashar; AlHabshy, AbdAllah A.; ElDahshan, Kamal A.

doi:10.21608/absb.2021.63810.1100

Distributed Intrusion Detection Systems in Big Data: A Survey

Document Type : Review Article

Authors

Mathematics Department, Faculty of Science, Al-Azhar University, Cairo, Egypt

10.21608/absb.2021.63810.1100

Abstract

We live in a time where data stream by the second, which makes intrusion detection a more difficult and tiresome task, and in turn intrusion detection systems require an efficient and improved detection mechanism to detect the intrusive activities. Moreover, handling the size, complexity, and availability of big data requires techniques that can create beneficial knowledge from huge streams of the information, which imposes the challenges on the process of both designing and management of both Intrusion Detection System (IDS) and Intrusion Prevention System (IPS) in terms of performance, sustainability, security, reliability, privacy, energy consumption, fault tolerance, scalability, and flexibility. IDSs and IPSs utilize various methodologies to guarantee security, accessibility and reliability of enterprise computer networks. This paper presents a comprehensive study of the Distributed Intrusion Detection Systems in Big Data, and presents intrusion detection and prevention techniques that utilize machine learning, big data analytics techniques in distributed systems of the intrusion detection.

Keywords

Main Subjects

Mathematics

References

Ahmed W, Wu YW. A survey on reliability in distributed systems. Journal of Computer and System Sciences. 2013;79(8):1243-55. DOI: https://doi.org/10.1016/j.jcss.2013.02.006

[2] Abraham A, Jain R, Thomas J, Han SY. D-SCIDS: Distributed soft computing intrusion detection system. Journal of Network and Computer Applications. 2007;30(1):81-98. DOI: https://doi.org/10.1016/j.jnca.2005.06.001

[3] Jones AK, Sielken RS. Computer system intrusion detection: A survey. Citeseer; 2000. DOI: http://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.24.7802&rep=rep1&type=pdf

[4] Ning P, Jajodia S. Intrusion detection techniques. The Internet Encyclopedia. 2004;2:355-67.DOI: https://doi.org/10.1002/047148296x.tie097

[5] Butun I, Morgera SD, Sankar R. A survey of intrusion detection systems in wireless sensor networks. IEEE communications surveys & tutorials. 2013;16(1):266-82. DOI: https://doi.org/10.1109/SURV.2013.050113.00191

[6] andala S, Ngadi MA, Abdullah AH. A survey on MANET intrusion detection. International Journal of Computer Science and Security. 2007;2(1):1-11. DOI: http://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.224.3523&rep=rep1&type=pdf

[7] Aldweesh A, Derhab A, Emam AZ. Deep learning approaches for anomaly-based intrusion detection systems: A survey, taxonomy, and open issues. Knowledge-Based Systems. 2020;189:105124.DOI: https://doi.org/10.1016/j.knosys.2019.105124

[8] Lunt TF. A survey of intrusion detection techniques. Computers & Security. 1993;12(4):405-18.DOI: https://doi.org/10.1016/0167-4048(93)90029-5

[9] Shah AA, Hayat MS, Awan MD. Analysis of machine learning techniques for intrusion detection system: a review. 2015. DOI: https://doi.org/10.5120/21047-3678

[10] Bagui S, Kalaimannan E, Bagui S, Nandi D, Pinto A. Using machine learning techniques to identify rare cyber‐attacks on the UNSW‐NB15 dataset. Security and Privacy. 2019;2(6):e91. DOI: https://doi.org/10.1002/spy2.91

[11] Moustafa N, Slay J, editors. UNSW-NB15: a comprehensive data set for network intrusion detection systems (UNSW-NB15 network data set). 2015 military communications and information systems conference (MilCIS); 2015: IEEE.DOI: https://doi.org/10.1109/MilCIS.2015.7348942

[12] 1. Samrin R, Vasumathi D, editors. Review on anomaly based network intrusion detection system. 2017 International Conference on Electrical, Electronics, Communication, Computer, and Optimization Techniques (ICEECCOT); 2017: IEEE. DOI: https://doi.org/10.1109/ICEECCOT.2017.8284655

[13] Khraisat A, Gondal I, Vamplew P, Kamruzzaman J. Survey of intrusion detection systems: techniques, datasets and challenges. Cybersecurity. 2019;2(1):1-22. DOI: https://doi.org/10.1186/s42400-019-0038-7

[14] Jose S, Malathi D, Reddy B, Jayaseeli D, editors. A survey on anomaly based host intrusion detection system. Journal of Physics: Conference Series; 2018: IOP Publishing. DOI: https://doi.org/10.1088/1742-6596/1000/1/012049

[15] Jyothsna V, Prasad R, Prasad KM. A review of anomaly based intrusion detection systems. International Journal of Computer Applications. 2011;28(7):26-35.DOI: http://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.259.1390&rep=rep1&type=pdf

[16] Mishra P, Varadharajan V, Tupakula U, Pilli ES. A detailed investigation and analysis of using machine learning techniques for intrusion detection. IEEE Communications Surveys & Tutorials. 2018;21(1):686-728. DOI: https://doi.org/10.1109/COMST.2018.2847722

[17] Mishra P, Varadharajan V, Tupakula U, Pilli ES. A detailed investigation and analysis of using machine learning techniques for intrusion detection. IEEE Communications Surveys & Tutorials. 2018;21(1):686-728. DOI: https://doi.org/10.1109/COMST.2018.2847722

[18] Mishra P, Varadharajan V, Tupakula U, Pilli ES. A detailed investigation and analysis of using machine learning techniques for intrusion detection. IEEE Communications Surveys & Tutorials. 2018;21(1):686-728. DOI: https://doi.org/10.1109/COMST.2018.2847722

[19]Liao H-J, Lin C-HR, Lin Y-C, Tung K-Y. Intrusion detection system: A comprehensive review. Journal of Network and Computer Applications. 2013;36(1):16-24. DOI: https://doi.org/10.1016/j.jnca.2012.09.004

[20] Debar H, Dacier M, Wespi A, editors. A revised taxonomy for intrusion-detection systems. Annales des télécommunications; 2000: Springer.DOI: https://doi.org/10.1007/bf02994844

[21] Khatkhate AM. Symbol time series analysis (STSA) for network event/intrusion detection.

[22] Eid HFAM. Computational Intelligence in Intrusion Detection System: MSc Thesis, Al-Azhar University; 2013. DOI: https://scholar.cu.edu.eg/sites/default/files/abo/files/phd_thesis_computational_intelligence_in_intrusion_detection_system_2013.pdf

[23] Deng L. A tutorial survey of architectures, algorithms, and applications for deep learning. APSIPA Transactions on Signal and Information Processing. 2014;3. DOI: https://doi.org/10.1017/atsip.2013.9

[24] Saravanan S, editor Performance evaluation of classification algorithms in the design of Apache Spark based intrusion detection system. 2020 5th International Conference on Communication and Electronics Systems (ICCES); 2020: IEEE. DOI: https://doi.org/10.1109/ICCES48766.2020.9138066

[25] Hassan MM, Gumaei A, Alsanad A, Alrubaian M, Fortino G. A hybrid deep learning model for efficient intrusion detection in big data environment. Information Sciences. 2020;513:386-96. DOI: https://doi.org/10.1016/j.ins.2019.10.069

[26] Alqahtani H, Sarker IH, Kalim A, Hossain SMM, Ikhlaq S, Hossain S, editors. Cyber intrusion detection using machine learning classification techniques. International Conference on Computing Science, Communication and Security; 2020: Springer. DOI: https://doi.org/10.1007/978-981-15-6648-6_10

[27] Vinayakumar R, Alazab M, Soman K, Poornachandran P, Al-Nemrat A, Venkatraman S. Deep learning approach for intelligent intrusion detection system. IEEE Access. 2019;7:41525-50. DOI: https://doi.org/10.1109/ACCESS.2019.2895334

[28] Faker O, Dogdu E, editors. Intrusion detection using big data and deep learning techniques. Proceedings of the 2019 ACM Southeast Conference; 2019. DOI: https://doi.org/10.1145/3299815.3314439

[29] Abdulhammed R, Musafer H, Alessa A, Faezipour M, Abuzneid A. Features dimensionality reduction approaches for machine learning based network intrusion detection. Electronics. 2019;8(3):322. DOI: https://doi.org/10.3390/electronics8030322

[30] Gao X, Shan C, Hu C, Niu Z, Liu Z. An adaptive ensemble machine learning model for intrusion detection. IEEE Access. 2019;7:82512-21. DOI: https://doi.org/10.1109/ACCESS.2019.2923640

[31] Belouch M, El Hadaj S, Idhammad M. Performance evaluation of intrusion detection based on machine learning using Apache Spark. Procedia Computer Science. 2018;127:1-6. DOI: https://doi.org/10.1016/j.procs.2018.01.091

[32] Shah SAR, Issac B. Performance comparison of intrusion detection systems and application of machine learning to Snort system. Future Generation Computer Systems. 2018;80:157-70. DOI: https://doi.org/10.1016/j.future.2017.10.016

[33] Othman SM, Ba-Alwi FM, Alsohybe NT, Al-Hashida AY. Intrusion detection model using machine learning algorithm on Big Data environment. Journal of Big Data. 2018;5(1):1-12. DOI: https://doi.org/10.1186/s40537-018-0145-4

[34] Sharafaldin I, Lashkari AH, Ghorbani AA. Toward generating a new intrusion detection dataset and intrusion traffic characterization. ICISSp. 2018;1:108-16. DOI: https://doi.org/10.5220/0006639801080116

[35] Almseidin M, Alzubi M, Kovacs S, Alkasassbeh M, editors. Evaluation of machine learning algorithms for intrusion detection system. 2017 IEEE 15th International Symposium on Intelligent Systems and Informatics (SISY); 2017: IEEE. DOI: https://doi.org/10.1109/SISY.2017.8080566

[36] Chowdhury MN, Ferens K, Ferens M, editors. Network intrusion detection using machine learning. Proceedings of the International Conference on Security and Management (SAM); 2016: The Steering Committee of The World Congress in Computer Science, Computer. DOI: https://www.proquest.com/conference-papers-proceedings/network-intrusion-detection-using-machine/docview/1807002945/se-2?accountid=201395

[37] Amoli PV, Hamalainen T, David G, Zolotukhin M, Mirzamohammad M. Unsupervised network intrusion detection systems for zero-day fast-spreading attacks and botnets. JDCTA (International Journal of Digital Content Technology and its Applications. 2016;10(2):1-13. DOI: http://users.jyu.fi/~pavahdan/Unsupervised%20Network%20Intrusion%20Detection%20Systems%20for%20Zero-Day%20Fast-Spreading%20Attacks%20and%20Botnets.pdf

[38] 1. Lin W-C, Ke S-W, Tsai C-F. CANN: An intrusion detection system based on combining cluster centers and nearest neighbors. Knowledge-based systems. 2015;78:13-21. DOI: https://doi.org/10.1016/j.knosys.2015.01.009

[39] Yassin W, Udzir NI, Muda Z, Sulaiman MN, editors. Anomaly-based intrusion detection through k-means clustering and naives bayes classification. Proc 4th Int Conf Comput Informatics, ICOCI; 2013. DOI: http://icoci.cms.net.my/PROCEEDINGS/2013/PDF/PID49.pdf

[40] Das S, Nene MJ, editors. A survey on types of machine learning techniques in intrusion prevention systems. 2017 International Conference on Wireless Communications, Signal Processing and Networking (WiSPNET); 2017: IEEE. DOI: https://doi.org/10.1109/WiSPNET.2017.8300169

[41] 1. Alves T, Das R, Morris T. Embedding encryption and machine learning intrusion prevention systems on programmable logic controllers. IEEE Embedded Systems Letters. 2018;10(3):99-102. DOI: https://doi.org/10.1109/LES.2018.2823906

[42] Patel A, Qassim Q, Wills C. A survey of intrusion detection and prevention systems. Information Management & Computer Security. 2010. DOI: https://doi.org/10.1108/09685221011079199

[43] Scarfone K, Mell P. Guide to intrusion detection and prevention systems (idps). NIST special publication. 2007;800(2007):94. DOI: https://doi.org/10.6028/nist.sp.800-94

[44] Wang L. Big Data in intrusion detection systems and intrusion prevention systems. Journal of Computer Networks. 2017;4(1):48-55. DOI: https://doi.org/10.12691/jcn-4-1-5

[45] ElDahshan KA, AlHabshy AA, Abutaleb GE. Data in the time of COVID-19: a general methodology to select and secure a NoSQL DBMS for medical data. PeerJ Computer Science. 2020;6:e297. DOI: https://doi.org/10.7717/peerj-cs.297

[46] Siddiqui S, Gupta D. Big data process analytics: a survey. Int J Emerg Res Manag Technol. 2014;3(7):117-23.

[47] Bendre MR, Thool VR. Analytics, challenges and applications in big data environment: a survey. Journal of Management Analytics. 2016;3(3):206-39. 8. DOI: https://doi.org/10.1080/23270012.2016.1186578

[48] ElDahshan K, Mancy H, editors. HPC based Modeling, Analyzing and Forecasting of a Century of Climate Big Data. The International Congress for global Science and Technology; 2015. DOI: https://www.researchgate.net/publication/284869677_Artificial_Intelligence_and_Machine_Learning_Journal_Volume_15_Issue_1_ICGST_Delaware_USA_Dec_2015

[49] Howlett RJ, Jain LC, Adelaide MLC. Smart Innovation, Systems and Technologies 022. DOI: https://doi.org/10.1007/978-3-642-27509-8

[50] Kumar KS, Mohanavalli S, editors. A performance comparison of document oriented NoSQL databases. 2017 International Conference on Computer, Communication and Signal Processing (ICCCSP); 2017: IEEE. DOI: https://doi.org/10.1109/ICCCSP.2017.7944071

[51] Abed AH. Recovery and concurrency challenging in big data and NoSQL database systems. International Journal of Advanced Networking and Applications. 2020;11(04):4321-9. DOI: https://doi.org/10.35444/ijana.2020.11041

[52] Moustafa N, Creech G, Slay J. Big data analytics for intrusion detection system: Statistical decision-making using finite dirichlet mixture models. Data analytics and decision support for cybersecurity: Springer; 2017. p. 127-56. DOI https://doi.org/10.1007/978-3-319-59439-2_5

[53] Wang L, Jones R. Big data analytics for network intrusion detection: A survey. International Journal of Networks and communications. 2017;7(1):24-31. DOI: https://doi.org/10.5923/j.ijnc.20170701.03

[54] Raja MC, Rabbani MA. Big data analytics security issues in data driven information system. Int J Innov Res Comput Commun Eng. 2014;2(10):6132-5. DOI: https://www.researchgate.net/publication/267753116_Big_Data_Analytics_Security_Issues_in_Data_Driven_Information_System

[55] 1. Cardenas AA, Manadhata PK, Rajan SP. Big data analytics for security. IEEE Security & Privacy. 2013;11(6):74-6. DOI: https://doi.org/10.1109/MSP.2013.138

[56] Shterenberg S, Poltavtseva MA. A distributed intrusion detection system with protection from an internal intruder. Automatic Control and Computer Sciences. 2018;52(8):945-53. DOI: https://doi.org/10.3103/S0146411618080230

[57] Zhong W, Yu N, Ai C. Applying big data based deep learning system to intrusion detection. Big Data Mining and Analytics. 2020;3(3):181-95. DOI: https://doi.org/10.26599/BDMA.2020.9020003

[58] Gao Y, Wu H, Song B, Jin Y, Luo X, Zeng X. A distributed network intrusion detection system for distributed denial of service attacks in vehicular ad hoc network. IEEE Access. 2019;7:154560-71. DOI: https://doi.org/10.1109/ACCESS.2019.2948382

[59] Idhammad M, Afdel K, Belouch M. Distributed intrusion detection system for cloud environments based on data mining techniques. Procedia Computer Science. 2018;127:35-41. DOI: https://doi.org/10.1016/j.procs.2018.01.095

[60] Dahiya P, Srivastava DK. Network intrusion detection in big dataset using spark. Procedia computer science. 2018;132:253-62. DOI: https://doi.org/10.1016/j.procs.2018.05.169

[61] Vimalkumar K, Radhika N, editors. A big data framework for intrusion detection in smart grids using apache spark. 2017 International Conference on Advances in Computing, Communications and Informatics (ICACCI); 2017: IEEE. DOI: https://doi.org/10.1109/ICACCI.2017.8125840

[62] Adhikari U, Morris TH, Pan S, editors. A causal event graph for cyber-power system events using synchrophasor. 2014 IEEE PES General Meeting| Conference & Exposition; 2014: IEEE.

DOI: https://doi.org/10.1109/PESGM.2014.6939285