Hyperparameters fine tuning for MARL comparative study [D]

[u/ham_bam0]

hello everyone. I'm training PPO variants on different multi-agent tasks from the VMAS library (Independent PPO / Graph PPO and such, see HetGPPO by Bettini et al.).

I noticed that for every architecture/scenario couple, the optimal hyperparameters sometimes tend to vary (learning rate, entropy coefficient, KL coefficient, SGD batch size, etc).

do I need - methodologically speaking - to unify the hyperparameters of all models in order to make a fair and correct comparison of architectures later on?

note: sometimes unifying these HP leads to some non converging models.

note 2 : my objective is to test these models' robustness under adversarial attack in test-time (frozen models).

thank you in advance.

[u/milesper]

Generally, I’d say you want to tune at least the LR separately per setting. Everything else can probably be held constant unless you have a good reason to believe it should vary.