F-MIM: Feature-based Masking Iterative Method to Generate the Adversarial Images against the Face Recognition Systems

Document Type : Research Paper

Authors

Department of Computer Engineering & Applications, GLA University, Mathura, 281406, (U.P.) India.

10.22059/jitm.2023.95247

Abstract

Numerous face recognition systems employ deep learning techniques to identify individuals in public areas such as shopping malls, airports, and other high-security zones. However, adversarial attacks are susceptible to deep learning-based systems. The adversarial attacks are intentionally generated by the attacker to mislead the systems. These attacks are imperceptible to the human eye. In this paper, we proposed a feature-based masking iterative method (F-MIM) to generate the adversarial images. In this method, we utilize the features of the face to misclassify the models. The proposed approach is based on a black-box attack technique where the attacker does not have the information related to target models. In this black box attack strategy, the face landmark points are modified using the binary masking technique. In the proposed method, we have used the momentum iterative method to increase the transferability of existing attacks. The proposed method is generated using the ArcFace face recognition model that is trained on the Labeled Face in the Wild (LFW) dataset and evaluated the performance of different face recognition models namely ArcFace, MobileFace, MobileNet, CosFace and SphereFace under the dodging and impersonate attack. The F-MIM attack is outperformed in comparison to the existing attacks based on Attack Success Rate evaluation metrics and further improves the transferability.

Keywords

References

  1.  

    Agrawal, K., & Bhatnagar, C. (2023). M-SAN: a patch-based transferable adversarial attack using the multi-stack adversarial network. Journal of Electronic Imaging, 32(2), 023033.

    Agrawal, K., & Bhatnagar, C. (2023, May). A Black-box based Attack Generation Approach to Create the Transferable Patch Attack. In 2023 7th International Conference on Intelligent Computing and Control Systems (ICICCS) (pp. 1376-1380). IEEE.

    Agrawal, K., Bhatnagar, C., 2021. Bmim: Generating adversarial attack on face recognition via binary mask, in: 2021 International Conference on Intelligent Technologies (CONIT), pp. 1–5. doi:10.1109/CONIT51480.2021.9498370.

    Biggio, B., Corona, I., Maiorca, D., Nelson, B., Srndi ˇ c, N., Laskov, P., Giacinto, G., Roli, F., 2013. Evasion attacks against machine learning at test time, in: Joint European conference on machine learning and knowledge discovery in databases, Springer. pp. 387–402.

    Bressan, G. M., de Azevedo, B. C. F., & de Souza, R. M. (2020). A fuzzy approach for diabetes mellitus type 2 classification. Brazilian Archives of Biology and Technology, 63. https://doi.org/10.1590/1678-4324-2020180742

    Yan, L. Meng, L. Li, et al., “Age-invariant face recognition by multi-feature fusion and
    decomposition with self-attention,” ACM Transactions on Multimedia Computing, Communications, and Applications (TOMM) 18(1s), 1–18 (2022).

    Chen, S., Liu, Y., GAO, X., & Han, Z. (2018, August). Mobilefacenets: Efficient cnns for accurate real-time face verification on mobile devices. In Chinese Conference on Biometric Recognition (pp. 428-438). Springer, Cham.

    Deb, D., Zhang, J., Jain, A.K., 2020. Advfaces: Adversarial face synthesis, in: 2020 IEEE International Joint Conference on Biometrics (IJCB), IEEE. pp. 1–10.

    Deng, J., Guo, J., Xue, N., Zafeiriou, S., 2019. Arcface: Additive angular margin loss for deep face recognition, in: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4690–4699.

    Devi, U. R., & Uma, K. (2019). A Study on Fuzzy Expert System for Diagnosis of Diabetes Mellitus. International Journal of Applied Engineering Research (IJAER), 14(4), 129–139. https://www.ripublication.com/ijaerspl2019/ijaerv14n4spl_16.pdf

    Dong, Y., Liao, F., Pang, T., Su, H., Zhu, J., Hu, X., Li, J., 2018. Boosting adversarial attacks with momentum, in: Proceedings of the IEEE conference on computer vision and pattern recognition, pp. 9185–9193.

    Geman, O., Chiuchisan, I., & Toderean, R. (2017). Application of Adaptive Neuro-Fuzzy Inference System for diabetes classification and prediction. 2017 E-Health and Bioengineering Conference, EHB 2017, Dm, 639–642. https://doi.org/10.1109/EHB.2017.7995505

    Goswami, G., Ratha, N., Agarwal, A., Singh, R., & Vatsa, M. (2018, April). Unravelling robustness of deep learning-based face recognition against adversarial attacks. In Proceedings of the AAAI Conference on Artificial Intelligence, 32(1).

    1. Qiu, D. Gong, Z. Li, et al., “End2end occluded face recognition by masking corrupted
      features,” IEEE Transactions on Pattern Analysis and Machine Intelligence (2021).

    Howard, A. G., Zhu, M., Chen, B., Kalenichenko, D., Wang, W., Weyand, T. & Adam, H. (2017). Mobilenets: Efficient convolutional neural networks for mobile vision applications. arXiv preprint arXiv:1704.04861.

    Howsalya Devi, R. D., Bai, A., & Nagarajan, N. (2020). A novel hybrid approach for diagnosing diabetes mellitus using farthest first and support vector machine algorithms. Obesity Medicine, 17, 100152. https://doi.org/10.1016/j.obmed.2019.100152

    Huang, G.B., Mattar, M., Berg, T., Learned-Miller, E., 2008. Labeled faces in the wild: A database forstudying face recognition in unconstrained environments, in: Workshop on faces in’Real-Life’Images: detection, alignment, and recognition.

    1. J. Goodfellow, J. Shlens, and C. Szegedy, “Explaining and harnessing adversarial exam-476
      ples,” arXiv preprint arXiv: 1412.6572 (2014).

    Khalil, R. M., & Al-Jumaily, A. (2017). Machine learning based prediction of depression among type 2 diabetic patients. Proceedings of the 2017 12th International Conference on Intelligent Systems and Knowledge Engineering, ISKE 2017, 2018-Janua, 1–5. https://doi.org/10.1109/ISKE.2017.8258766

    Khan, T., Singh, K., Manjul, M., Ahmad, M. N., Zain, A. M., & Ahmadian, A. (2022). A Temperature-Aware Trusted Routing Scheme for Sensor Networks: Security Approach. Computers & Electrical Engineering, 98, 107735.

    Kumar, A., Singh, K., & Khan, T. (2021). L-RTAM: Logarithm based reliable trust assessment model for WBSNs. Journal of Discrete Mathematical Sciences and Cryptography, 24(6), 1701-1716.

    Kumar, A., Singh, K., Khan, T., Ahmadian, A., Saad, M. H. M., & Manjul, M. (2021). ETAS: an efficient trust assessment scheme for BANs. IEEE Access, 9, 83214-83233.

    Kurakin, A., Goodfellow, I., Bengio, S., et al., 2016. Adversarial examples in the physical world.

    Liu, W., Wen, Y., Yu, Z., Li, M., Raj, B., Song, L., 2017. Sphereface: Deep hypersphere embedding for face recognition, in: Proceedings of the IEEE conference on computer vision and pattern recognition, pp. 212–220.

    Lukmanto, R. B., Suharjito, Nugroho, A., & Akbar, H. (2019). Early detection of diabetes mellitus using feature selection and fuzzy support vector machine. Procedia Computer Science, 157, 46–54. https://doi.org/10.1016/j.procs.2019.08.140

    Mahmood Sharif, Sruti Bhagavatula, Lujo Bauer, and Michael K Reiter. Accessorize to a crime: Real and stealthy attacks on state-of-the-art face recognition. In Proceedings of the 2016 acm sigsac conference on computer and communications security, pages 1528–1540, 2016.

    Massoli, F.V., Falchi, F., Amato, G., 2020. Cross-resolution face recognition
    adversarial attacks. Pattern Recognition Letters 140, 222–229.

    Niswati, Z., Mustika, F. A., & Paramita, A. (2018). Fuzzy logic implementation for diagnosis of Diabetes Mellitus disease at Puskesmas in East Jakarta. Journal of Physics: Conference Series, 1114(1). https://doi.org/10.1088/1742-6596/1114/1/012107

    Raj, R. S., Sanjay, D. S., Kusuma, M., & Sampath, S. (2019). Comparison of Support Vector Machine and Naïve Bayes Classifiers for Predicting Diabetes. 1st International Conference on Advanced Technologies in Intelligent Control, Environment, Computing and Communication Engineering, ICATIECE 2019, 41–45. https://doi.org/10.1109/ICATIECE45860.2019.9063792

    Rajeswari, A. M., Sidhika, M. S., Kalaivani, M., & Deisy, C. (2018). Prediction of Prediabetes using Fuzzy Logic based Association Classification. Proceedings of the International Conference on Inventive Communication and Computational Technologies, ICICCT 2018, Icicct, 782–787. https://doi.org/10.1109/ICICCT.2018.8473159

    Rozsa, A., Günther, M., & Boult, T. E. (2017, October). LOTS about attacking deep features. In 2017 IEEE International Joint Conference on Biometrics (IJCB) (pp. 168-176). IEEE.

    Sarwar, M. A., Kamal, N., Hamid, W., & Shah, M. A. (2018). Prediction of diabetes using machine learning algorithms in healthcare. ICAC 2018 - 2018 24th IEEE International Conference on Automation and Computing: Improving Productivity through Automation and Computing, September, 1–6. https://doi.org/10.23919/IConAC.2018.8748992

    Schroff, F., Kalenichenko, D., Philbin, J., 2015. Facenet: A unified embedding for face recognition and clustering, in: Proceedings of the IEEE conference on computer vision and pattern recognition, pp. 815–823.

    Sharif, M., Bauer, L., Reiter, M.K., 2018. On the suitability of lp-norms for creating and preventing adversarial examples, in: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition Workshops, pp. 1605–1613.

    Sharif, M., Bhagavatula, S., Bauer, L., Reiter, M.K., 2016. Accessorize to a crime: Real and stealthy attacks on state-of-the-art face recognition, in: Proceedings of the 2016 acm sigsac conference on computer and communications security, pp. 1528–1540.

    Swain, A., Mohanty, S., & Das, A. (2013). COMPARATIVE RISK ANALYSIS ON PREDICTION OF DIABETES MELLITUS USING MACHINE LEARNING APPROACH. 1, 3312–3317. International Conference on Electrical, Electronics, and Optimization Techniques (ICEEOT) - 2016

    Szegedy, W. Zaremba, I. Sutskever, et al., “Intriguing properties of neural networks,” arXiv474
    preprint arXiv: 1312.6199 (2013).

    Thakkar, H., Shah, V., Yagnik, H., & Shah, M. (2021). Comparative anatomization of data mining and fuzzy logic techniques used in diabetes prognosis. Clinical EHealth, 4, 12–23. https://doi.org/10.1016/j.ceh.2020.11.001

    Turk, M.A., Pentland, A.P., 1991. Face recognition using eigenfaces, in: Proceedings. 1991 IEEE computer society conference on computer vision and pattern recognition, IEEE Computer Society. pp. 586–587.

    Undre, P., Kaur, H., & Patil, P. (2015). Improvement in prediction rate and accuracy of diabetic diagnosis system using fuzzy logic hybrid combination. 2015 International Conference on Pervasive Computing: Advance Communication Technology and Application for Society, ICPC 2015, 00(c). https://doi.org/10.1109/PERVASIVE.2015.7087029

    Verma, D, M. N. (2017). using Data mining classification Techniques. 2017 International Conference on Intelligent Sustainable Systems (ICISS), Iciss, 533–538.

    Vijiyakumar, K., Lavanya, B., Nirmala, I., & Sofia Caroline, S. (2019). Random forest algorithm for the prediction of diabetes. 2019 IEEE International Conference on System, Computation, Automation and Networking, ICSCAN 2019, 1–5. https://doi.org/10.1109/ICSCAN.2019.8878802

    Wang, H., Wang, Y., Zhou, Z., Ji, X., Gong, D., Zhou, J., Li, Z., Liu, W., 2018. Cosface: Large margin cosine loss for deep face recognition, in: Proceedings of the IEEE conference on computer vision and pattern recognition, pp. 5265–5274.

    1. Wei, Y. Guo, and J. Yu, “Adversarial sticker: A stealthy attack method in the physical world,” IEEE Transactions on Pattern Analysis and Machine Intelligence, 1–1 (2022).
    2. Dong, H. Su, B. Wu, et al., “Efficient decision-based black-box adversarial attacks on face recognition,” in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern
      Recognition, 7714–7722 (2019).
    3. Dong, Q.-A. Fu, X. Yang, et al., “Benchmarking adversarial robustness on image classification,” in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 321–331 (2020).
    4. Li, X. Yang, B. Wu, et al., “Hiding faces in plain sight: Disrupting ai face synthesis with 484
      adversarial perturbations,” arXiv preprint arXiv: 1906.09288 (2019).

    Yang, L., Song, Q., Wu, Y., 2021. Attacks on state-of-the-art face recognition using attentional adversarial attack generative network. Multimedia Tools and Applications 80, 855–875.

    Zhang, K., Zhang, Z., Li, Z., Qiao, Y., 2016. Joint face detection and alignment using multitask cascaded convolutional networks. IEEE Signal Processing Letters 23, 1499–1503.

    Zhong, Y., Deng, W., 2020a. Towards transferable adversarial attack against deep face recognition. IEEE Transactions on Information Forensics and Security 16, 1452–1466.

    Zhou, Z., Tang, D., Wang, X., Han, W., Liu, X., Zhang, K., 2018. Invisible mask: Practical attacks on face recognition with infrared. arXiv preprint arXiv:1803.04683.

     

Files

Share

How to cite

Statistics