Issue 23133: Pickling of ipaddress classes

Currently ipaddress classes support pickling, but the pickling is not efficient and is implementation depened. Proposed patch makes pickling more compact and implementation agnostic.

Patch looks good to me. For further efficiency, addresses could be pickled as ints (but beware of interfaces and networks).

Yes, pickling (and especially unpickling) ints is more efficient, but the code will more complex. Interfaces should be pickled as strings for backward compatibility, and interfaces are subclasses of addresses.

Here are microbenchmarks:

./python -m timeit -s "import ipaddress, pickle; ips = [ipaddress.ip_address('192.0.2.%s'%i) for i in range(1, 101)]" -- "pickle.dumps(ips)"
./python -m timeit -s "import ipaddress, pickle; ips = [ipaddress.ip_address('2001:db8::%x'%i) for i in range(1, 101)]" -- "pickle.dumps(ips)"
./python -m timeit -s "import ipaddress, pickle; ips = [ipaddress.ip_address('192.0.2.%s'%i) for i in range(1, 101)]; pickled = pickle.dumps(ips)" -- "pickle.loads(pickled)"
./python -m timeit -s "import ipaddress, pickle; ips = [ipaddress.ip_address('2001:db8::%x'%i) for i in range(1, 101)]; pickled = pickle.dumps(ips)" -- "pickle.loads(pickled)"

Results for unpatched module:
1000 loops, best of 3: 1.56 msec per loop
1000 loops, best of 3: 1.62 msec per loop
1000 loops, best of 3: 1.08 msec per loop
1000 loops, best of 3: 1.09 msec per loop

With ipaddress_pickle.patch:
100 loops, best of 3: 3.43 msec per loop
100 loops, best of 3: 10.6 msec per loop
100 loops, best of 3: 7.76 msec per loop
100 loops, best of 3: 8.58 msec per loop

With ipaddress_pickle_2.patch:
1000 loops, best of 3: 1.11 msec per loop
1000 loops, best of 3: 1.16 msec per loop
1000 loops, best of 3: 1.88 msec per loop
100 loops, best of 3: 2.05 msec per loop

With ipaddress_pickle_3.patch:
1000 loops, best of 3: 1.12 msec per loop
1000 loops, best of 3: 1.15 msec per loop
1000 loops, best of 3: 1.13 msec per loop
1000 loops, best of 3: 1.15 msec per loop

ipaddress_pickle_3.patch breaks one test (testMissingAddressVersion). Is this test needed?

I don't understand what the test is for. I think it's safe it's remove it.

Then I'll remove it. Could you please make a review of optimized patch?

The patch looks fine to me.

New changeset 781b54f7bccc by Serhiy Storchaka in branch 'default':
Issue #23133: Pickling of ipaddress objects now produces more compact and
https://hg.python.org/cpython/rev/781b54f7bccc

Thank you Antoine.

And here is comparison of pickle size.

Unpatched:
>>> len(pickle.dumps([ipaddress.ip_address('192.0.2.%s'%i) for i in range(1, 101)]))
2971
>>> len(pickle.dumps([ipaddress.ip_address('2001:db8::%x'%i) for i in range(1, 101)]))
4071
>>> len(pickle.dumps([ipaddress.ip_interface('192.0.2.%s/27'%i) for i in range(1, 101)]))
19341
>>> len(pickle.dumps([ipaddress.ip_interface('2001:db8::%x/124'%i) for i in range(1, 101)]))
22741
>>> len(pickle.dumps([ipaddress.ip_network('192.0.2.%s/27'%(i&-32)) for i in range(1, 101)]))
10614
>>> len(pickle.dumps([ipaddress.ip_interface('2001:db8::%x/124'%(i&-32)) for i in range(1, 101)]))
22741

Patched:
>>> len(pickle.dumps([ipaddress.ip_address('192.0.2.%s'%i) for i in range(1, 101)]))
1531
>>> len(pickle.dumps([ipaddress.ip_address('2001:db8::%x'%i) for i in range(1, 101)]))
2631
>>> len(pickle.dumps([ipaddress.ip_interface('192.0.2.%s/27'%i) for i in range(1, 101)]))
2963
>>> len(pickle.dumps([ipaddress.ip_interface('2001:db8::%x/124'%i) for i in range(1, 101)]))
3256
>>> len(pickle.dumps([ipaddress.ip_network('192.0.2.%s/27'%(i&-32)) for i in range(1, 101)]))
2938
>>> len(pickle.dumps([ipaddress.ip_interface('2001:db8::%x/124'%(i&-32)) for i in range(1, 101)]))
3209

New changeset 712ac77b772b by Serhiy Storchaka in branch 'default':
Fixed tests for issue #23133 (pickling of IPv4Network was not tested).
https://hg.python.org/cpython/rev/712ac77b772b

stage: patch review -> resolved

messages: + msg234121