# Warm-starting encoder-decoder models using EncoderDecoderModel always giving an empty string after fine-tuning

**URL:** <https://discuss.huggingface.co/t/warm-starting-encoder-decoder-models-using-encoderdecodermodel-always-giving-an-empty-string-after-fine-tuning/78678>\
**Category:** 🤗Transformers\
**Created:** [March 25, 2024, 1:10pm UTC](https://discuss.huggingface.co/t/warm-starting-encoder-decoder-models-using-encoderdecodermodel-always-giving-an-empty-string-after-fine-tuning/78678 "2024-03-25T13:10:48Z")\
**Posts on this page:** 1\
**Page:** 1

<div class="post-metadata">

**Author:** ![ka05ar](https://avatars.discourse-cdn.com/v4/letter/k/e19adc/32.png) [@ka05ar](https://discuss.huggingface.co/u/ka05ar)\
**Post date:** [March 25, 2024, 1:10pm UTC](https://discuss.huggingface.co/t/warm-starting-encoder-decoder-models-using-encoderdecodermodel-always-giving-an-empty-string-after-fine-tuning/78678/1 "2024-03-25T13:10:48Z")

</div>

### System Info

I am trying to train a seq2seq model using **EncoderDecoderModel** class and found [this](https://huggingface.co/blog/warm-starting-encoder-decoder) blog very helpful. Thanks to @patrickvonplaten for his excellent explanation. Following this blog I fine-tuned a seq2seq model where I used a **BERT** ([BanglaBERT] an Electra) model as encoder and **[XGLM]** as decoder using [BanglaParaphrase] data. But after fine-tuning the model always generates an **empty string** or **garbage output**. Now I do not understand where the problem is. Can anyone please help me find the bug in the [code](https://www.kaggle.com/code/kawsar17/paraphrase-bert2xglm).

Thanks.

### Expected behavior

Input-output for my code:  
{‘target’: ‘সিপিও আহত থাকায় যুদ্ধ পরিচালনার দায়িত্ব এসে পড়েছিল সেম্প্রোনিয়াসের কাঁধে।’,  
‘pred\_target’: ‘’}

which should be something like this (should give the paraphrased sentence according to the input sentence in Bangla):  
{‘target’: ‘সিপিও আহত থাকায় যুদ্ধ পরিচালনার দায়িত্ব এসে পড়েছিল সেম্প্রোনিয়াসের কাঁধে।’,  
‘pred\_target’: ‘সিপিও কর্তৃক আহত হয়ে সেমপ্রোনিয়াসের কাঁধে যুদ্ধ পরিচালনার দায়িত্ব আসে।’}