# ALBERT Pretraining example (Tensorflow)

**URL:** <https://discuss.huggingface.co/t/albert-pretraining-example-tensorflow/1427>\
**Category:** Beginners\
**Created:** [October 8, 2020, 3:43pm UTC](https://discuss.huggingface.co/t/albert-pretraining-example-tensorflow/1427 "2020-10-08T15:43:34Z")\
**Posts on this page:** 3\
**Page:** 1

<div class="post-metadata">

**Author:** ![UmarSpa](https://avatars.discourse-cdn.com/v4/letter/u/e47774/32.png) [@UmarSpa](https://discuss.huggingface.co/u/UmarSpa)\
**Post date:** [October 8, 2020, 3:43pm UTC](https://discuss.huggingface.co/t/albert-pretraining-example-tensorflow/1427/1 "2020-10-08T15:43:34Z")

</div>

Hello everyone,

Is there any guide / example of how to do pretraining of ALBERT (in Tensorflow) using code only from huggingface?

I would really appreciate any help.

Thanks

---

<div class="post-metadata">

**Author:** ![UmarSpa](https://avatars.discourse-cdn.com/v4/letter/u/e47774/32.png) [@UmarSpa](https://discuss.huggingface.co/u/UmarSpa)\
**Post date:** [October 23, 2020, 12:05pm UTC](https://discuss.huggingface.co/t/albert-pretraining-example-tensorflow/1427/2 "2020-10-23T12:05:15Z")

</div>

@sgugger any help here would be very appreciated

---

<div class="post-metadata">

**Author:** ![UmarSpa](https://avatars.discourse-cdn.com/v4/letter/u/e47774/32.png) [@UmarSpa](https://discuss.huggingface.co/u/UmarSpa)\
**Post date:** [November 23, 2020, 10:15am UTC](https://discuss.huggingface.co/t/albert-pretraining-example-tensorflow/1427/3 "2020-11-23T10:15:30Z")

</div>

Still waiting a reply on this