Issue 28752: datetime object fails to restore from reduction

On Python 3.5, the datetime would reduce and restore cleanly.

$ python3.5 -c "import datetime; func, params = datetime.datetime.now().__reduce__(); func(*params)"

With Python 3.6.0b3, it now fails with a TypeError.

$ python3.6 -c "import datetime; func, params = datetime.datetime.now().__reduce__(); func(*params)"
Traceback (most recent call last):
  File "<string>", line 1, in <module>
TypeError: an integer is required (got type bytes)

Pickle still works, so pickle must be relying on a different protocol for serialization.

$ python
Python 3.6.0b3 (v3.6.0b3:8345e066c0ed, Oct 31 2016, 18:05:23) 
[GCC 4.2.1 (Apple Inc. build 5666) (dot 3)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> import pickle
>>> import datetime
>>> pickle.loads(pickle.dumps(datetime.datetime.now()))
datetime.datetime(2016, 11, 20, 10, 56, 43, 264436)

Now pickling of the datetime.datetime objects is implemented with __reduce_ex__. __reduce__ is not defined in datetime.datetime and is inherited from datetime.date.

>>> datetime.datetime.__reduce_ex__
<method '__reduce_ex__' of 'datetime.datetime' objects>
>>> datetime.datetime.__reduce__
<method '__reduce__' of 'datetime.date' objects>

__reduce_ex__ has higher priority and is called instead of __reduce__. It is weird that __reduce_ex__ and __reduce__ are not consistent. __reduce__ should be defined as

    def __reduce__(self):
        return self.__reduce_ex__(2)

or just

    __reduce__ = object.__reduce__

Other way is to define __reduce_ex__ instead of __reduce__ in datetime.date.

Proposed patch restores the __reduce__() methods and makes Python and C implementations more consistent.

> On Nov 20, 2016, at 11:10 AM, Serhiy Storchaka <report@bugs.python.org> wrote:
> 
> Other way is to define __reduce_ex__ instead of __reduce__ in datetime.date.

I would prefer this solution.

> Other way is to define __reduce_ex__ instead of __reduce__ in datetime.date.

Sorry, I was wrong. This would wouldn't work with C implementation. And explicitly setting __reduce__ = object.__reduce__ doesn't work. The only way is to define both __reduce_ex__ and __reduce__ for time and datewtime.

> On Nov 20, 2016, at 12:34 PM, Serhiy Storchaka <report@bugs.python.org> wrote:
> 
> The only way is to define both __reduce_ex__ and __reduce__ for time and datewtime.

OK. I'll review your patch and get it committed shortly.

The patch LGTM.  I'll commit it tonight unless Serhiy beats me to it.

If you can push this in the next hour or two, it can still make b4.

> If you can push this in the next hour or two ...

I'll do it now.

New changeset 0a2a0061e425 by Serhiy Storchaka in branch '3.6':
Issue #28752: Restored the __reduce__() methods of datetime objects.
https://hg.python.org/cpython/rev/0a2a0061e425

New changeset 23140bd66d86 by Serhiy Storchaka in branch 'default':
Issue #28752: Restored the __reduce__() methods of datetime objects.
https://hg.python.org/cpython/rev/23140bd66d86

It looks like Serhiy has already committed it.  Thanks!

Thanks all! So pleased to see this fixed.

stage: patch review

components: + Extension Modules
type: behavior