# Gradient checkpointing without training

**URL:** <https://discuss.huggingface.co/t/gradient-checkpointing-without-training/47251>\
**Category:** Beginners\
**Created:** [July 18, 2023, 8:51pm UTC](https://discuss.huggingface.co/t/gradient-checkpointing-without-training/47251 "2023-07-18T20:51:46Z")\
**Posts on this page:** 1\
**Page:** 1

<div class="post-metadata">

**Author:** ![Mike5645](https://avatars.discourse-cdn.com/v4/letter/m/3e96dc/32.png) [@Mike5645](https://discuss.huggingface.co/u/Mike5645)\
**Post date:** [July 18, 2023, 8:51pm UTC](https://discuss.huggingface.co/t/gradient-checkpointing-without-training/47251/1 "2023-07-18T20:51:46Z")

</div>

I have a LlamaForCausalLM model. I want to do a single run of backprop on a single sample (one forward pass, one backward pass) and record all the gradients that are computed in the process. I do not want to actually update the model weights- I just want to record the gradients. The model is pretty big and I only have a single GPU, so to be able to do this I need to use gradient checkpointing. Is there a way to use a Trainer to accomplish this? Thanks!