IA-Transformer: Prediction and Classification of β-Lactamase Proteins Using Transformer Model with Integrated Attention Mechanism

Yuankun  Du; Fengping  Liu; Yi  Hou

doi:10.6180/jase.202609_32.010

IA-Transformer: Prediction and Classification of β-Lactamase Proteins Using Transformer Model with Integrated Attention Mechanism

Computer Science and Information Engineering

Yuankun Du¹, Fengping Liu²This email address is being protected from spambots. You need JavaScript enabled to view it., and Yi Hou¹

¹College of Big Data and Artificial Intelligence, Zhengzhou University of Science and Technology, Zhengzhou, Henan 450064, China

²School of Information Engineering, Zhengzhou University of Science and Technology, Zhengzhou, Henan 450064, China

Received: September 29, 2025
Accepted: July 4, 2026
Publication Date: April 18, 2026

Copyright The Author(s). This is an open access article distributed under the terms of the Creative Commons Attribution License (CC BY 4.0), which permits unrestricted use, distribution, and reproduction in any medium, provided the original author and source are cited.

Download Citation: ||https://doi.org/10.6180/jase.202609_32.010

β-Lactamase proteins are the primary mediators of bacterial resistance to β-Lactam antibiotics, posing a severe threat to global public health. Accurate prediction and classification of β-Lactamase proteins are crucial for the development of novel antibiotics and the formulation of clinical treatment strategies. Traditional machine learning methods for β-Lactamase analysis often rely on manual feature engineering, which fails to fully capture the complex sequence patterns and contextual information of proteins. To address this limitation, this study proposes a Transformer model integrated with a multi-head attention mechanism (IA-Transformer) for the prediction and classification of β-Lactamase proteins. The IA-Transformer model innovatively integrates three attention modules: sequence-wise self-attention, residue-wise attention, and channel-wise attention. The sequence-wise self-attention captures long-range dependencies between amino acid residues in the protein sequence; the residue-wise attention emphasizes key functional residues related to β-Lactam hydrolysis; and the channel-wise attention optimizes the feature representation of different sequence motifs. Experimental results show that the IA-Transformer model achieves an accuracy of 98.2%, a sensitivity of 97.8%, a specificity of 98.5%, and an F1-score of 98.0% in β-Lactamase prediction, outperforming traditional methods such as SVM, Random Forest, and single-attention Transformer by 3.5%−7.2%.

Keywords: β-Lactamase; Transformer Model; Integrated Attention Mechanism; Protein Prediction; Protein Classification; Antibiotic Resistance

[1] P. Agarwal, R. P. Kumar, L.-M. Oleksiuk, V. Crall, A. A. Petrov, E. K. McCreary, J. Holder-Murray, Y.-F. Chang, N. Agarwal, D. K. Hamilton, et al., (2025) “Non–β-lactam antibiotic use, β-lactam allergy, and surgical site infections" JAMA surgery 160(11): 1260–1267. DOI: 10.1001/jamasurg.2025.3789.
[2] B. Bedeni´c, M. Pospišil, M. Na ¯d, and D. Bandi´c Pavlovi´c, (2025) “Evolution of β-Lactam antibiotic resistance in proteus species: from Extended-Spectrum and Plasmid-Mediated Amp C β-Lactamases to carbapen mases" Microorganisms 13(3): 508. DOI: 10.3390/ microorganisms13030508.
[3] V. T. Nguyen, B. T. Birhanu, V. Miguel-Ruano, C. Kim, M. Batuecas, J. Yang, A. M. El-Araby, E. Jimenez Faraco, V. A. Schroeder, A. Alba, et al., (2025) “Restoring susceptibility to β-lactam antibiotics in methicillin resistant Staphylococcus aureus" Nature chemical biology 21(4): 482–489. DOI: 10.1038/s41589-024-01688-0.
[4] J.Pacy´nska andP. Niedzielski, (2025) “Scoping Review of Extraction Methods for Detecting β-Lactam Antibiotics in Food Products of Animal Origin" Molecules 30(9): 1937. DOI: 10.3390/molecules30091937.
[5] W.-Y. Fan, X. Zhang, D.-H. Xie, K. M. Y. Leung, and G.-P. Sheng, (2025) “Cerium-based nanohydrolase for fast catalytic hydrolysis of β-lactam antibiotics in wastewater effluents" Journal of Hazardous Materials 484: 136800. DOI: 10.1016/j.jhazmat.2024.136800.
[6] N. Abdelmalek, S. W. Yousief, M. S. Bojer, M. S. A. Alobaidallah, J. E. Olsen, and B. Paglietti, (2025) “The secondary resistome of methicillin-resistant Staphylococcus aureus to β-lactam antibiotics" Antibiotics 14(2): 112. DOI: 10.3390/antibiotics14020112.
[7] Y. Cao, Y. Yang, W. Zhao, H. Liu, X. Zhang, H. Chen, M. Sui, and P. Ma, (2025) “SERS based determination of ceftriaxone, ampicillin, and vancomycin in serum using WS2/Au@ Ag nanocomposites and a 2D-CNN regres sion model" Spectrochimica Acta Part A: Molecular and Biomolecular Spectroscopy 333: 125850. DOI: 10.1016/j.saa.2025.125850.
[8] T. Li. “Time-Series Batch Predictive Control Based on MIC-LSTM-ATT”. In: 2025 5th International Conference on Artificial Intelligence and Industrial Technology Applications (AIITA). IEEE. 2025, 1202–1208. DOI: 10.1109/AIITA65135.2025.11047860.
[9] L. He, H. Li, R. Qi, Q. Zou, and Y. Wang,(2025)“MCT ARG: Identification and classification of antibiotic resistance genes based on a multi-channel Transformer model" Science of the Total Environment 1006: 180848. DOI: 10.1016/j.scitotenv.2025.180848.
[10] A. Zubair, M. Fazil, M. Jawad, and S. Wdidi, (2025) “The Role of Machine Learning in Addressing Antibiotic Resistance: A New Era in Infectious Disease Control" Mi crobiology Open 14(6): e70160. DOI: 10.1002/mbo3.70160.
[11] A. E. Nolasco-Rojas, E. Cruz-Del-Agua, C. Cruz Cruz, M. Á. Loyola-Cruz, B. A. Ayil-Gutiérrez, M. C. Tamayo-Ordóñez, Y. d. J. Tamayo-Ordoñez, A. Rojas-Bernabé, F. A. Tamayo-Ordoñez, E. M. Durán-Manuel, et al., (2025) “Microbiological risks to health associated with the release of antibiotic resistant Bacteria and β-lactam antibiotics through hospital wastewater" Pathogens 14(5): 402. DOI: 10.3390/ pathogens14050402.
[12] M.-J. Yang, M.-J. Li, L.-D. Huang, X.-W. Zhang, Y.-Y. Huang,X.-Y. Gou, S.-N. Chen, J. Yan, P. Du, and A.-H. Sun, (2025) “Response regulator protein CiaR regulates the transcription of ccn-microRNAs and β-lactam antibiotic resistance conversion of Streptococcus pneumoniae" International Journal of Antimicrobial Agents 65(1): 107387. DOI: 10.1016/j.ijantimicag.2024.107387.
[13] M. Labied, A. Belangour, and M. Banane, (2025) “P GELU: A Novel Activation Function to Optimize Whisper for Darija Speech Translation" IEEE Access 13: 100198 100218. DOI: 10.1109/ACCESS.2025.3574398.
[14] H.PerveenandJ. Weeds, (2025) “Protein sequence classification using natural language processing techniques" Discover Artificial Intelligence 5(1): 66. DOI: 10. 1007/s44163-025-00304-x.
[15] B. Wang, R. Meng, Z. Li, M. Hu, X. Wang, Y. Zhao, Z. Chai, Y. Jin, J. Yue, W. Chen, et al., (2025) “Predicting antibiotic resistance genes and bacterial phenotypes based on protein language models" Frontiers in Microbiology 16: 1628952. DOI: 10.3389/fmicb.2025.1628952.
[16] Y. Zhao, J. Zhang, Y. Gui, G. Ji, X. Huang, F. Xie, and H. Shen, (2025) “Probing the interaction mechanisms between three β-lactam antibiotics and penicillin-binding proteins of Escherichia coli by molecular dynamics simu lations" Comparative Biochemistry and Physiology Part C:Toxicology&Pharmacology287:110057. DOI: 10.1016/j.cbpc.2024.110057.
[17] H. S. Butman, M. A. Stefaniak, D. J. Walsh, V. S. Gondil, M. Young, A. H. Crow, A. M. Nemeth, R. J. Melander, P. M. Dunman, and C. Melander, (2025) “Phenyl urea based adjuvants for β-lactam an tibiotics against methicillin resistant Staphylococcus au reus" Bioorganic & medicinal chemistry letters 121: 130164. DOI: 10.1016/j.bmcl.2025.130164.
[18] A.Sharma,V.Diwakar,R.Kumar,andP.Garg,(2025) “Enzyme classification integrating LSTM and Prot-BERT sequence encoding" Applied Soft Computing: 113774. DOI: 10.1016/j.asoc.2025.113774.

Latest Articles