Avoid division by zero in adagrad

[mhucka]

The update subfunction in adagrad() will produce division-by-zero errors. These occur when running check/pytest in Python 3.12 with NumPy 2.x:

src/openfermion/resource_estimates/pbc/thc/factorizations/thc_jax_test.py::test_kpoint_thc_helper
src/openfermion/resource_estimates/pbc/thc/generate_costing_table_thc_test.py::test_generate_costing_table_thc
  /usr/local/google/home/mhucka/project-files/google/github/openfermion/src/openfermion/resource_estimates/thc/utils/adagrad.py:43:
  RuntimeWarning: divide by zero encountered in divide
    g_sq_inv_sqrt = np.where(g_sq > 0, 1.0 / np.sqrt(g_sq), 0.0)

A common method to avoid this situation is to add a small positive number to g_sq before taking the square root. This ensures that the value is never exactly zero, avoiding the division-by-zero problem. This approach is a common and robust method for avoiding zero denominators in gradient-based optimization. It also acts as a smoothing factor, preventing the update from becoming unstable when g_sq gets very small.

[dstrain115]

What do you think about doing this only if abs(g_sq<eps)?

[mhucka]

It may impact performance, but probably not too badly. I've rewritten it like this:

if np.abs(g_sq < eps):
    # Add a small number to avoid division by zero.
    g_sq_inv_sqrt = 1.0 / np.sqrt(g_sq + eps)
else:
    g_sq_inv_sqrt = 1.0 / np.sqrt(g_sq)

Does that seem okay?

I forgot that g_sq is an array, so a test like x < y means testing every value, and that's going to be more time-consuming.

My reading of to handle cases like this is that the original is not a bad approach in a case where gradients are being done, because ultimately the values are not used in the results. @dstrain115 do you think it's going to lead to errors if left in the original form?