# Pre-training LayoutLMv2

**URL:** <https://discuss.huggingface.co/t/pre-training-layoutlmv2/11852>\
**Category:** Intermediate\
**Created:** [November 16, 2021, 6:57am UTC](https://discuss.huggingface.co/t/pre-training-layoutlmv2/11852 "2021-11-16T06:57:12Z")\
**Posts on this page:** 1\
**Page:** 1

<div class="post-metadata">

**Author:** ![pgladkevich](https://avatars.discourse-cdn.com/v4/letter/p/a587f6/32.png) [@pgladkevich](https://discuss.huggingface.co/u/pgladkevich)\
**Post date:** [November 16, 2021, 6:57am UTC](https://discuss.huggingface.co/t/pre-training-layoutlmv2/11852/1 "2021-11-16T06:57:12Z")

</div>

For the purpose of simulating few and multi-shot learning we would like to pre-train LayoutLMv2 in a fashion similar to using BertForPreTraining. Would anyone have any resources for someone who has potentially done something similar with LayoutLM or another HF transformer model?