# Multi-input classification (images + Texts)

**URL:** <https://discuss.huggingface.co/t/multi-input-classification-images-texts/22441>\
**Category:** Beginners\
**Created:** [September 2, 2022, 6:15am UTC](https://discuss.huggingface.co/t/multi-input-classification-images-texts/22441 "2022-09-02T06:15:03Z")\
**Posts on this page:** 7\
**Page:** 1

<div class="post-metadata">

**Author:** ![uniflow](https://avatars.discourse-cdn.com/v4/letter/u/a8b319/32.png) [@uniflow](https://discuss.huggingface.co/u/uniflow)\
**Post date:** [September 2, 2022, 6:15am UTC](https://discuss.huggingface.co/t/multi-input-classification-images-texts/22441/1 "2022-09-02T06:15:03Z")

</div>

Hi folks,

I am pretty new here. So here is my problem description: I am trying to classify a sequence of document images. These documents could be between 1 and 10 pages long. I noticed there are some models such as LayoutLM are designed explicitly for document images, however, it seems it can only intake one image at a time. In our setting, we need multiple images since two different documents could contain similar images somewhere in them respectively. I can also use OCR to convert the images to texts with corresponding coordinates.

I came from Tensorflow world. In the past, I have been using its functional API to train a single model that can intake multiple inputs. But I am not sure how to do that with HuggingFace. Does anyone have experience in this type of problem?

Thank you!

---

<div class="post-metadata">

**Author:** ![dinhanhx](https://sea2.discourse-cdn.com/hellohellohello/user_avatar/discuss.huggingface.co/dinhanhx/32/20115_2.png) [@dinhanhx](https://discuss.huggingface.co/u/dinhanhx)\
**Post date:** [September 2, 2022, 8:45am UTC](https://discuss.huggingface.co/t/multi-input-classification-images-texts/22441/2 "2022-09-02T08:45:02Z")

</div>

Uhm HuggingFace Transformers simply is a collection of models written with PyTorch, Tensorflow, and Flax. So yeah you just take a model source code you want then plug in whichever mode you want.

---

<div class="post-metadata">

**Author:** ![uniflow](https://avatars.discourse-cdn.com/v4/letter/u/a8b319/32.png) [@uniflow](https://discuss.huggingface.co/u/uniflow)\
**Post date:** [September 2, 2022, 6:42pm UTC](https://discuss.huggingface.co/t/multi-input-classification-images-texts/22441/3 "2022-09-02T18:42:53Z")

</div>

Thank you! Is it possible to load the pre-trained weight that way?

---

<div class="post-metadata">

**Author:** ![dinhanhx](https://sea2.discourse-cdn.com/hellohellohello/user_avatar/discuss.huggingface.co/dinhanhx/32/20115_2.png) [@dinhanhx](https://discuss.huggingface.co/u/dinhanhx)\
**Post date:** [September 3, 2022, 9:47am UTC](https://discuss.huggingface.co/t/multi-input-classification-images-texts/22441/4 "2022-09-03T09:47:44Z")

</div>

Yeah but you might have to mind about keys in the state dict (assuming you use PyTorch)

---

<div class="post-metadata">

**Author:** ![uniflow](https://avatars.discourse-cdn.com/v4/letter/u/a8b319/32.png) [@uniflow](https://discuss.huggingface.co/u/uniflow)\
**Post date:** [September 4, 2022, 6:25am UTC](https://discuss.huggingface.co/t/multi-input-classification-images-texts/22441/5 "2022-09-04T06:25:34Z")

</div>

Actually, I use Tensorflow. Do you have any tutorials I can read about? Sorry I am completely new to Huggingface. Appreciate your help and patience.

---

<div class="post-metadata">

**Author:** ![dinhanhx](https://sea2.discourse-cdn.com/hellohellohello/user_avatar/discuss.huggingface.co/dinhanhx/32/20115_2.png) [@dinhanhx](https://discuss.huggingface.co/u/dinhanhx)\
**Post date:** [September 4, 2022, 7:15am UTC](https://discuss.huggingface.co/t/multi-input-classification-images-texts/22441/6 "2022-09-04T07:15:45Z")

</div>

Sorry I am not familiar with Tensorflow enough try to read their doc more

---

<div class="post-metadata">

**Author:** ![thefaheem](https://sea2.discourse-cdn.com/hellohellohello/user_avatar/discuss.huggingface.co/thefaheem/32/23988_2.png) [@thefaheem](https://discuss.huggingface.co/u/thefaheem)\
**Post date:** [February 18, 2024, 4:20pm UTC](https://discuss.huggingface.co/t/multi-input-classification-images-texts/22441/7 "2024-02-18T16:20:21Z")

</div>

Hey @uniflow do you found a solution yet?
