nferruz

/

ProtGPT2

like 98

@nferruz Hello Noelia,
Thank you for sharing ProtGPT2 with the protein design community, great work. I'm new to deep learning and only have a few experiences in prediction/classification tasks using pre-trained protein language models, e.g., the ESM-2 model. For fine-tuning, I'm using Hugging Face Trainer.

May I ask how to define the compute_metrics function in fine-tuning ProtGPT2? For the classification task, the compute_metrics function is like:

def compute_metrics(eval_pred):
    metrics = load('accuracy')
    predictions, labels = eval_pred
    predictions = np.argmax(predictions, axis=1)
    return metrics.compute(predictions=predictions, references=labels)

I know we can use run_clm.py to fine-tune. I just want to try to use Hugging Face Trainer to make it run.

Don't know if it's appropriate to ask this question. I googled it, but what I got is all about validation for text generation tasks. Any hints would be appreciated.

Sincerely,
Beibei