# M2M100 training does not improve model performance

**URL:** <https://discuss.huggingface.co/t/m2m100-training-does-not-improve-model-performance/23787>\
**Category:** 🤗Transformers\
**Created:** [September 29, 2022, 2:38pm UTC](https://discuss.huggingface.co/t/m2m100-training-does-not-improve-model-performance/23787 "2022-09-29T14:38:42Z")\
**Posts on this page:** 1\
**Page:** 1

<div class="post-metadata">

**Author:** ![joaomiguel26](https://avatars.discourse-cdn.com/v4/letter/j/b487fb/32.png) [@joaomiguel26](https://discuss.huggingface.co/u/joaomiguel26)\
**Post date:** [September 29, 2022, 2:38pm UTC](https://discuss.huggingface.co/t/m2m100-training-does-not-improve-model-performance/23787/1 "2022-09-29T14:38:42Z")

</div>

Hello,  
I’m trying to fine-tune M2M100 using the script run\_translation.py and it seems that the model is not improving.  
I am using the following command:

```auto
deepspeed examples/pytorch/translation/run_translation.py \
        --deepspeed tests/deepspeed/ds_config_zero3.json \
        --model_name_or_path facebook/m2m100_418M \
        --per_device_train_batch_size 8 \
        --per_device_eval_batch_size 8 \
        --output_dir output_dir --overwrite_output_dir \
        --fp16 \
        --do_train --do_eval --do_predict \
        --max_train_samples 500 --max_eval_samples 50 --max_predict_samples 50 \
        --num_train_epochs 0.001 \
        --dataset_name wmt16 --dataset_config "ro-en" \
        --source_lang en --target_lang ro \
        --predict_with_generate --forced_bos_token ro

```

Just to give you an example, if I train for 1 epoch I can get 20 BLEU points in the test set, but if I train for 3 epochs I get around 10 BLEU points.  
Am I doing anything wrong? Does M2M100 requires any specific hyperparameter/hyperparameter configuration?  
Thanks
