Issue 22519: integer overflow in computing byte's object representation

# PyBytes_Repr(PyObject *obj, int smartquotes)
# {
#     PyBytesObject* op = (PyBytesObject*) obj;
# 1   Py_ssize_t i, length = Py_SIZE(op);
#     size_t newsize, squotes, dquotes;
#     ...
# 
#     /* Compute size of output string */
#     newsize = 3; /* b'' */
#     s = (unsigned char*)op->ob_sval;
#     for (i = 0; i < length; i++) {
#       ...
#         default:
#             if (s[i] < ' ' || s[i] >= 0x7f)
# 2               newsize += 4; /* \xHH */
#             else
#                 newsize++;
#         }
#     }
#     ...
# 3   if (newsize > (PY_SSIZE_T_MAX - sizeof(PyUnicodeObject) - 1)) {
#       PyErr_SetString(PyExc_OverflowError,
#         "bytes object is too large to make repr");
#       return NULL;
#     }
# 4   v = PyUnicode_New(newsize, 127);
#     ...
#     *p++ = 'b', *p++ = quote;
#     for (i = 0; i < length; i++) {
#         ...
# 5         *p++ = c;
#     }
#     *p++ = quote;
# 6   assert(_PyUnicode_CheckConsistency(v, 1));
#     return v;
# }
# 
# 1. length=2^30+1=1073741825
# 2. newsize=length*4+3=7 (overflow)
# 3. check is inefficient, because newsize=7
# 4. allocated buffer is too small
# 5. buffer overwrite
# 6. this assert will likely fail, since there is a good chance the allocated
#    buffer is just before the huge one, so the huge one will overwrite itself.

New changeset d9cd11eda152 by Benjamin Peterson in branch '2.7':
fix overflow checking in PyString_Repr (closes #22519)
https://hg.python.org/cpython/rev/d9cd11eda152

New changeset f5c662a7f7e6 by Benjamin Peterson in branch '3.3':
fix overflow checking in PyBytes_Repr (closes #22519)
https://hg.python.org/cpython/rev/f5c662a7f7e6

New changeset ed31cdf11ac2 by Benjamin Peterson in branch '3.4':
merge 3.3 (closes #22519)
https://hg.python.org/cpython/rev/ed31cdf11ac2

New changeset 0ddc5fc5f395 by Benjamin Peterson in branch 'default':
merge 3.4 (#22519)
https://hg.python.org/cpython/rev/0ddc5fc5f395

It would be nice to have a "bigmem" test checking that repr(b'\x00'*(2**30+1)) doesn't crash anymore.

The code doesn't crash any more. It took me more than 5 GB of resident memory and about 90 CPU seconds to reproduce the circumstances of the overflow.

nosy: + christian.heimes
messages: + msg277326

type: security -> behavior
resolution: fixed

nosy: + python-dev
messages: + msg227856

resolution: fixed
stage: resolved