Computing the mean and std of large frames (e.g. ~16 million points) can suffer from floating point error when they are performed using float32 values. Both mean and std can be passed dtype=np.float64 to perform these computations using float64 instead and thereby retain better accuracy. One place where this could be added is here:
|
self.noise_mean, self.noise_std = np.mean(clipped_data), np.std(clipped_data) |
but perhaps it would be good to add a dtype=np.float64 parameter to the enclosing function so that the user could specify a different type if desired.
Computing the mean and std of large frames (e.g. ~16 million points) can suffer from floating point error when they are performed using float32 values. Both
meanandstdcan be passeddtype=np.float64to perform these computations using float64 instead and thereby retain better accuracy. One place where this could be added is here:setigen/setigen/frame.py
Line 398 in 5ca049f
but perhaps it would be good to add a
dtype=np.float64parameter to the enclosing function so that the user could specify a different type if desired.