LoRA, GPU & all hyperparameters
#1
by
despinapz
- opened
Hello, thank you for the provided information. Can you list all the hyperparameters and the GPU you used to achieve this accuracy, please? Also, did you use the LoRA method? I cannot replicate your results using your selected hyperparameters, as the loss is becoming 0.0 after just 0.1 of the first epoch. Thank you.
I used adamw_8bit as the optimizer.
despinapz
changed discussion status to
closed