Can enable_cache, enable_fusion, and enable_tf32 be used during training?

[tai-dang11]

Hi Protenix team,

thanks for the Protenix v0.7.0 release. It's great for inference.

Quick question: can any of the following inference acceleration flags also be used during training, or are they strictly inference-only?

--enable_cache
--enable_fusion
--enable_tf32

Thanks for the clarification!

[zhangyuxuann]

@tai-dang11 Hi, thank you for your interest! While these optimizations are theoretically applicable during training, their impact on model accuracy in that setting has not been rigorously validated. The open-sourced implementation guarantees near-identical performance metrics when enabled during inference.

[tai-dang11]

Hi @zhangyuxuann, thanks for the clarification.

As a side note, I’m planning to train Mini Protenix with the constraint model, and I’m considering experimenting with these optimizations. I was wondering if you’d mind sharing a bit more detail on how the base constraint model is trained. I couldn’t find specifics in the paper or elsewhere, such as: lr and scheduler, number of training steps, batch size, dataset used for constraint training

Also, I noticed it looks like you might be using a different lr and scheduler for the constraint embedder compared to the rest of the model — is that correct?

Thanks again