DIII-D unaccessible PTDATA pointnames

[yumouwei]

Description of the problem

The get_n1_bradial_parameters tries to access PTDATA DUSBRADIAL and (if it fails) ONSBRADIAL to fetch data for its computation. Currently both of these get data calls fail.

disruption-py/disruption_py/machine/d3d/physics.py

Lines 705 to 725 in 7284810

	# TODO: TREE NAME?
	# BUG: Failed here!
	dusbradial, t_n1 = params.mds_conn.get_data_with_dims(
	f"ptdata('dusbradial', {params.shot_id})",
	tree_name="d3d",
	)
	t_n1 /= 1e3 # [ms] -> [s]
	dusbradial = interp1(t_n1, dusbradial, params.times)
	dusbradial *= 1.0e-4 # [T]
	except mdsExceptions.MdsException as e:
	params.logger.debug(f"[Shot {params.shot_id}]:{traceback.format_exc()}")
	# If there's no data in dusbradial (DUD system)
	# check if ONFR was on and calculating the bradial
	try:
	dusbradial, t_n1 = params.mds_conn.get_data_with_dims(
	f"ptdata('onsbradial', {params.shot_id})",
	tree_name="d3d",
	)
	t_n1 /= 1e3 # [ms] -> [s]
	dusbradial = interp1(t_n1, dusbradial, params.times)
	dusbradial *= 1.0e-4 # [T]

Corresponding lines in the MATLAB script

disruption-py/DIII-D/get_n1_bradial_d3d.m

Lines 84 to 99 in 5a13906

	fprintf(1,['Reading bradial from MDS+ for shot%7i\n'], shot);
	[n1time,status] = mdsvalue(['dim_of(ptdata("dusbradial", ' num2str(shot) '))']);%[ms]
	if (mod(status,2) == 1 && length(n1time) >= 4);%1
	fprintf(1,'dusbradial\n')
	n1time = n1time/1.e3; % convert ms to s
	dusbradial = mdsvalue(['ptdata("dusbradial", ' num2str(shot) ')'])*1.e-4; %[T]
	mdsclose;
	else;
	% If there's no data in dusbradial (DUD system)
	% check if ONFR was on and calculating the bradial
	[n1time,status] = mdsvalue(['dim_of(ptdata("onsbradial", ' num2str(shot) '))']);
	if (mod(status,2) == 1 && length(n1time) >= 4);%2
	fprintf(1,'onsbradial\n')
	n1time = n1time/1.e3; % convert ms to s
	dusbradial = mdsvalue(['ptdata("onsbradial", ' num2str(shot) ')'])*1.e-4; %[T]
	mdsclose;

I also tried checking these pointnames in ReviewPlus but it gave ERROR %TREE-W-NNF, Node Not Found error as well.

Also, on Omega I tried to check jTraverser by using tree name = "d3d" or "efit01" but neither of them worked. What's the proper tree name I should use?

[gtrevisan]

watch out, PTDATA is not MDSplus, although I believe lots of the queries are made to be MDSplus-compatible.
have you tried treename="PTDATA"?

[yumouwei]

params.mds_conn.get_data_with_dims(f"ptdata('dusbradial', {params.shot_id})",tree_name="ptdata") failed as well.

[gtrevisan]

have you tried with normal python and mdsplus calls? outside of disrpy?

[yumouwei]

Doesn't seem to work either

import MDSplus as mds

pointname = 'onsbradial'   # MDSplus.mdsExceptions.TdiABORT: %TDI-E-ABORT, Program requested abort
shotno = 161228

conn = mds.Connection('atlas.gat.com')
# conn.openTree('PTDATA', shotno)   # TREE-E-FOPENR, Error opening file read-only
conn.openTree('d3d', shotno)

data = conn.get(f"ptdata('{pointname}', {shotno})").data()
time = conn.get("dim_of("+f"ptdata('{pointname}', {shotno})"+")").data()

conn.closeAllTrees()
conn.disconnect()

print(f"Time: {time}")
print(f"data: {data}")

Setting pointname = 'ip' succeeded so the MDSplus calls should be correct.

[gtrevisan]

could you also try with treename=None?

[yumouwei]

I tried None, 'efit01' & '_efit_tree' but neither worked.

[yumouwei]

Cannot find PTDATA point names dpsradcva and dpsradxdiv

• Fix get_peaking_parameters for DIII-D #265

[yumouwei]

Get_n1_bradial_parameters

There are 4 potential data sources for the bradial (locked mode) signal. These are:

1. DUSBRADIAL, which points to the DUD legacy warning algorithm ("category") in the PCS.
2. ONSBRADIAL, which points to the more recent ONFR system in the PCS. In general this point name should be checked first before checking DUSBRADIAL.
3. On IRIS, the /fusion/projects/disruption_warning/software/recalc_bradial/bradial/... folder (and the compressed bradial.tar.gz file). This folder stores data for shots 156199--172430 (3814 shots taken between 2014--2017) which were missing in the "standard" atlas tree path and were recomputed by B. Sammuli. The data are stored as MDS tree structures by shot.
4. On IRIS, /fusion/projects/disruption_warning/software/recalc_bradial/recalc.nc. This NetCDF file stores data for shots 176030--176912 (880 shots) that were recalculated also by B. Sammuli due to an error in hardware calibration.

Shot range	Data source	Tested shot
--156198	Check ONSBRADIAL -> DUSBRADIAL	156100: DUSBRADIAL available, ONSBRADIAL unavailable.
156199--172430	Get data from custom tree structures (3)	Note: all 4 testing shots are within this range.
172431--176029	Check ONSBRADIAL -> DUSBRADIAL	172435: DUSBRADIAL available, ONSBRADIAL unavailable.
176030--176912	Read data from recalc.nc (4)
176913--	Check ONSBRADIAL -> DUSBRADIAL	176913: ONSBRADIAL available.

TODO:

• Configurate disruption-py to read MDS tree structures stored on server.
• Decide where to store (3) & (4).

Bolometer peaking factors

The two point names DPSRRDCVA & DPSRRDXDIV (be aware of the typo in the signal names) point to the DPRF (Disruption Prediction via Random Forest) module in the PCS. These signals are archived as the data source of the DPRF algorithm. The DPRF module was implemented around 2019, therefore only data after that date would be available. We tested shot 180808 and it worked.

The DPRF module has not been actively maintained. For this reason it would be more preferable to compute these two signals natively in disruption-py (fixed in #265) instead of relying on the stored data.

[gtrevisan]

what's the relationship between Brian's two netcdf files, and this one already in the repo?

disruption-py/disruption_py/machine/d3d/physics.py

Lines 845 to 848 in 3c81c17

	if 176030 <= params.shot_id <= 176912:
	raise NotImplementedError
	# TODO: Move to a folder like "/fusion/projects/disruption_warning/data"
	filename = "/fusion/projects/disruption_warning/matlab_programs/recalc.nc"

EDIT: cannot read that path, but I do find /fusion/projects/disruption_warning/media/recalc.nc

[yumouwei]

I don't have permission to access /media so I can't check, but I assume it's the same as (4) because it covers the same shot numbers

(3) is not a netcdf file but a folder of MDS data structures. This range of shot numbers was mentioned in the MATLAB script but isn't included in the function's logic, so I assume it was done manually some time afterward. For us the problem is how to read these files because of that thick vs thin clients problem.

[gtrevisan]

hmm, okay. I don't see a quick way to implement this fallback in a nice, robust, and sustainable manner -- we'll have to think about it some more.

just FYI, it seems that these two files are different:

cd /fusion/projects/disruption_warning
ls -l {media,software/recalc_bradial}/recalc.nc
[snip] 3351859235 Aug 20  2018 media/recalc.nc
[snip] 3351848992 May  3  2018 software/recalc_bradial/recalc.nc