need urgent help for ner deberta training

[u/Fragrant-Courage3548]

hi,
i am trying to train a deberta model for NER detection

this is my first time doing it so i would love any guidance on it.

my current pipeline looks like this,

dapt + lora for pretrianing, hpo with optuna (which consists both the stages of training data), and then a 2 stage finetuning which helps in generalization and then target data.

i am trying to reach a really good score for f1 on my use case (which i want to keep private for now)

i have few questions as well

1. do i need a two stage hpo as well cuase of the 2 stage finetuning
2. is it better if the hpo training set is a subset of the actual training set?

if you think anything can be improved and made better, or you think the pipeline is outright wrong, please mention your reasonings and thoughts :)

ps: lora was used cause of gpu budget constraints