BIOGAN-BERT: BioGPT-2 Fine Tuned and GAN-BERT for Extracting Drug Interaction Based on Biomedical Texts

EasyChair Preprint 14738

Abstract

Drug-drug interactions (DDIs) occur when two or more drugs are used together, leading to unexpected and potentially harmful effects. Identifying DDIs requires manual annotations, but the increasing volume of research publications and the slow data annotation process make this challenging. Machine learning, especially deep learning, can efficiently extract and identify DDIs from biomedical literature. However, class imbalance in datasets reduces model performance. This study introduces BIOGAN-BERT, which combines data augmentation using the Pretrained Language Model (PLM) BioGPT-2 and Generative Adversarial Network (GAN) to address class imbalance in DDI extraction tasks. It identifies gaps in existing imbalance handling studies and proposes enhancements through PLM-based data augmentation and semi-supervised learning with GAN. BioGPT-2 generates additional data from labeled and unlabeled sources, enriching the training dataset. This data is then processed using GAN-BERT, allowing the model to learn from more complex data distributions, thereby improving data quality and model generalization. Traditional methods like sampling only increase the number of data instances, and loss functions merely assign greater representation to the loss values. While these methods expand the learning space for models, they do not enhance data representation. In contrast, this novel approach uses data augmentation to increase both the quantity and the diversity of data. Evaluation results show that BIOGANBERT outperforms several baselines, significantly increasing the micro F1-Score for minor classes to 0.85 compared to 0.83 for the best baseline model, demonstrating its effectiveness in handling class imbalance and contextual variations in biomedical data.

Keyphrases: Drug-drug interactions (DDI), Generative Adversarial Network (GAN), data augmentation, deep learning, machine learning

@booklet{EasyChair:14738,
  author    = {Made Arbi Parameswara and Rila Mandala},
  title     = {BIOGAN-BERT: BioGPT-2 Fine Tuned and GAN-BERT for Extracting Drug Interaction Based on Biomedical Texts},
  howpublished = {EasyChair Preprint 14738},
  year      = {EasyChair, 2024}}