Created on 2014-05-12 18:24 by pitrou, last changed 2014-05-15 19:06 by pitrou. This issue is now closed.
Here is a patch to optimize ipv4 netmask parsing by maintaining a cache (there are not many valid ipv4 netmask representations). This should be especially useful with #16531.
Updated patch, also optimizing v6 netmask parsing (same principle).
Before patch:
$ ./python -m timeit -s "import ipaddress" "net = ipaddress.IPv6Network(('2001:db8::', 96))"
10000 loops, best of 3: 26.1 usec per loop
$ ./python -m timeit -s "import ipaddress" "net = ipaddress.IPv4Network(('10.0.0.0', 23))"
100000 loops, best of 3: 17 usec per loop
After patch:
$ ./python -m timeit -s "import ipaddress" "net = ipaddress.IPv6Network(('2001:db8::', 96))"
100000 loops, best of 3: 13.8 usec per loop
$ ./python -m timeit -s "import ipaddress" "net = ipaddress.IPv4Network(('10.0.0.0', 23))"
100000 loops, best of 3: 14.3 usec per loop
Why not just use functools.lru_cache?
Because that would incur the cost of LRU logic and locking, which we don't need here.
With C implementation (issue14373) functools.lru_cache is so fast as manually written specialized code.
What I want to say, the patch LGTM, but after committing issue14373 we should simplify the code by using functools.lru_cache().
Actually, using lru_cache(maxsize=None) would enable a simple infinite cache like in the patch. But it's not like a lot of code would be saved.
New changeset 2158614e1607 by Antoine Pitrou in branch 'default': Issue #21486: Optimize parsing of netmasks in ipaddress.IPv4Network and ipaddress.IPv6Network. http://hg.python.org/cpython/rev/2158614e1607
messages:
+ msg218349
title: optimize v4 netmask parsing -> optimize v4 & v6 netmask parsing