Skip to content

bpo-45711: [asyncio] Normalize exceptions immediately after Fetch, before they are stored as StackItem, which should be normalized #29890

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
iritkatriel merged 5 commits into from
Dec 3, 2021
Merged

bpo-45711: [asyncio] Normalize exceptions immediately after Fetch, before they are stored as StackItem, which should be normalized #29890

Show file tree
Hide file tree
Changes from 4 commits
Commits
Show all changes
5 commits
Select commit Hold shift + click to select a range
847fcdb
bpo-45711: Remove unnecessary normalization of exc_info
iritkatriel Dec 2, 2021
acc65e6
put the traceback update back in _PyErr_ChainStackItem
iritkatriel Dec 2, 2021
a995431
normalize exceptions after Fetch when they are about to be stored as …
iritkatriel Dec 2, 2021
6ce3848
📜🤖 Added by blurb_it.
blurb-it[bot] Dec 2, 2021
e1a719a
Revert "bpo-45711: Remove unnecessary normalization of exc_info"
iritkatriel Dec 2, 2021
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions Misc/NEWS.d/next/Library/2021-12-02-17-22-06.bpo-45711.D6jsdv.rst
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
Make :mod:`asyncio` normalize exceptions as soon as they are captured with :c:func:`PyErr_Fetch`, and before they are stored as an exc_info triplet. This brings :mod:`asyncio` in line with the rest of the codebase, where an exc_info triplet is always normalized.
19 changes: 12 additions & 7 deletions Modules/_asynciomodule.c
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -2702,6 +2702,11 @@ task_step_impl(TaskObj *task, PyObject *exc)
if (PyErr_ExceptionMatches(asyncio_CancelledError)) {
/* CancelledError */
PyErr_Fetch(&et, &ev, &tb);
assert(et);
PyErr_NormalizeException(&et, &ev, &tb);
if (tb != NULL) {
PyException_SetTraceback(ev, tb);
}

FutureObj *fut = (FutureObj*)task;
_PyErr_StackItem *exc_state = &fut->fut_cancelled_exc_state;
Expand All @@ -2714,14 +2719,12 @@ task_step_impl(TaskObj *task, PyObject *exc)

/* Some other exception; pop it and call Task.set_exception() */
PyErr_Fetch(&et, &ev, &tb);

assert(et);
if (!ev || !PyObject_TypeCheck(ev, (PyTypeObject *) et)) {
PyErr_NormalizeException(&et, &ev, &tb);
}
PyErr_NormalizeException(&et, &ev, &tb);
if (tb != NULL) {
PyException_SetTraceback(ev, tb);
}

o = future_set_exception((FutureObj*)task, ev);
if (!o) {
/* An exception in Task.set_exception() */
Expand Down Expand Up @@ -2965,7 +2968,7 @@ task_step(TaskObj *task, PyObject *exc)
PyObject *et, *ev, *tb;
PyErr_Fetch(&et, &ev, &tb);
leave_task(task->task_loop, (PyObject*)task);
_PyErr_ChainExceptions(et, ev, tb);
_PyErr_ChainExceptions(et, ev, tb); /* Normalizes (et, ev, tb) */
return NULL;
}
else {
Expand Down Expand Up @@ -3014,8 +3017,10 @@ task_wakeup(TaskObj *task, PyObject *o)
}

PyErr_Fetch(&et, &ev, &tb);
if (!ev || !PyObject_TypeCheck(ev, (PyTypeObject *) et)) {
PyErr_NormalizeException(&et, &ev, &tb);
assert(et);
PyErr_NormalizeException(&et, &ev, &tb);
if (tb != NULL) {
PyException_SetTraceback(ev, tb);
}

result = task_step(task, ev);
Expand Down
20 changes: 0 additions & 20 deletions Python/errors.c
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -647,26 +647,6 @@ _PyErr_ChainStackItem(_PyErr_StackItem *exc_info)
PyObject *typ, *val, *tb;
_PyErr_Fetch(tstate, &typ, &val, &tb);

PyObject *typ2, *val2, *tb2;
typ2 = exc_info->exc_type;
val2 = exc_info->exc_value;
tb2 = exc_info->exc_traceback;
#ifdef Py_DEBUG
PyObject *typ2_before = typ2;
PyObject *val2_before = val2;
PyObject *tb2_before = tb2;
#endif
_PyErr_NormalizeException(tstate, &typ2, &val2, &tb2);
#ifdef Py_DEBUG
/* exc_info should already be normalized */
assert(typ2 == typ2_before);
assert(val2 == val2_before);
assert(tb2 == tb2_before);
#endif
if (tb2 != NULL) {
PyException_SetTraceback(val2, tb2);
}

Copy link
Copy Markdown
Member

@gvanrossum gvanrossum Dec 2, 2021

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is this deletion supposed to be part of this PR? (The PR title didn't indicate a cleanup in errors.c.)

@1st1 should probably review this.

Copy link
Copy Markdown
Member Author

@iritkatriel iritkatriel Dec 2, 2021

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It is intended, I thought the error.c change doesn’t need to be in news because it’s a private function.

Yes, I’ll wait to hear from @1st1.

Once we have just exc_value in StackItem, I think we should add something like a PyErr_FetchNormalized(&exc) for these cases. Then you use PyErr_Fetch only for Fetch-Restore.

Copy link
Copy Markdown
Member Author

@iritkatriel iritkatriel Dec 2, 2021

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actually it's a good point, I'll remove this from this PR so it's just asyncio and @1st1 won't need to worry about this part.

/* _PyErr_SetObject sets the context from PyThreadState. */
_PyErr_SetObject(tstate, typ, val);
Py_DECREF(typ); // since _PyErr_Occurred was true
Expand Down