# Ensemble Learning with various BERT models

**URL:** <https://discuss.huggingface.co/t/ensemble-learning-with-various-bert-models/32442>\
**Category:** Intermediate\
**Created:** [February 24, 2023, 1:21pm UTC](https://discuss.huggingface.co/t/ensemble-learning-with-various-bert-models/32442 "2023-02-24T13:21:12Z")\
**Posts on this page:** 2\
**Page:** 1

<div class="post-metadata">

**Author:** ![mox](https://avatars.discourse-cdn.com/v4/letter/m/f04885/32.png) [@mox](https://discuss.huggingface.co/u/mox)\
**Post date:** [February 24, 2023, 1:21pm UTC](https://discuss.huggingface.co/t/ensemble-learning-with-various-bert-models/32442/1 "2023-02-24T13:21:12Z")

</div>

Hi,

i have fine tuned different BERT models for a binary text classification task with the Trainer API. Now I want to try out, if ensemble learning (e.g. combining different BERT models) would help to improve the results.  
My question is, if there is a tutorial or script how this could be done with Trainer. I have never done enseble learning and don’t really know how to start. Also I am not sure how to predict new samples with ensemble models.

Thanks in advance,  
Max

---

<div class="post-metadata">

**Author:** ![Hatman](https://sea2.discourse-cdn.com/hellohellohello/user_avatar/discuss.huggingface.co/hatman/32/27670_2.png) [@Hatman](https://discuss.huggingface.co/u/Hatman)\
**Post date:** [February 25, 2023, 6:54am UTC](https://discuss.huggingface.co/t/ensemble-learning-with-various-bert-models/32442/2 "2023-02-25T06:54:12Z")

</div>

I’m not sure about using the Trainer API. You could just build your own [Decision Tree](https://careerfoundry.com/en/blog/data-analytics/what-is-random-forest/) with the trained models This might help [Multi-Model-Endpoint](https://www.youtube.com/watch?v=Vnkx_hOWLio&list=PLhr1KZpdzukcOr_6j_zmSrvYnLUtgqsZz&index=14) tutorial on Sagemaker.