Thanks for this tutorial.
I have tried the above training script with Idefics2. While the training and validation losses have reduced.
When I load the model and trained adapter for inference, the results are exactly same as the base model. It looks like an issue during saving or loading the adapter weights.
Is this example training script and inference tested on latest transformers and trl library versions ?
If not, is it possible to provide the library versions that’s worked for this script ?
