# Is TFAlbert model pre-trainable?

**URL:** <https://discuss.huggingface.co/t/is-tfalbert-model-pre-trainable/293>\
**Category:** 🤗Transformers\
**Created:** [July 15, 2020, 5:46am UTC](https://discuss.huggingface.co/t/is-tfalbert-model-pre-trainable/293 "2020-07-15T05:46:13Z")\
**Posts on this page:** 2\
**Page:** 1

<div class="post-metadata">

**Author:** ![dangerzone](https://sea2.discourse-cdn.com/hellohellohello/user_avatar/discuss.huggingface.co/dangerzone/32/37445_2.png) [@dangerzone](https://discuss.huggingface.co/u/dangerzone)\
**Post date:** [July 15, 2020, 5:46am UTC](https://discuss.huggingface.co/t/is-tfalbert-model-pre-trainable/293/1 "2020-07-15T05:46:13Z")

</div>

Hello,

I’m interested in pre-training the Albert model on a large set of domain specific data. I see on GitHub that there is a `TFAlbertForPreTraining` class, but it is not showing up in the docs.

Is this class only there for the loading of pre-trained Albert models, or can I make use of this to do pre-training myself?

---

<div class="post-metadata">

**Author:** ![sgugger](https://sea2.discourse-cdn.com/hellohellohello/user_avatar/discuss.huggingface.co/sgugger/32/2291_2.png) [@sgugger](https://discuss.huggingface.co/u/sgugger)\
**Post date:** [July 15, 2020, 9:14pm UTC](https://discuss.huggingface.co/t/is-tfalbert-model-pre-trainable/293/2 "2020-07-15T21:14:42Z")

</div>

I think the team just forgot to put them in the docs. You can make a PR to add them or I’ll fix that next week when I’m back from vacation.