What are the minimum requirements to run an inference?

[stargazing41]

Hi,

I am attempting to run the model on my machine however the code keeps dying due to lack of memory even though my machine has enough memory to load all of the files. Is there any way I can know the minimum requirements needed to use the model? Thanks!

[emahannon]

I have the same issue on RTX 2080 Ti. Would be curious to know the answer if anyone has run inferencing successfully.

[gangelidis]

I have the same issue with a 3080Ti (12 GB). I have halved the precision, but the memory issue persists. So I approached every modality in a different script, full precision, and it works—still figuring out how I can combine more of them.