[proxy] bugs.python.org← back | site home | direct (HTTPS) ↗ | proxy home | ◑ dark◐ light

Message 305993 - Python tracker

Message305993

Author Olivier.Grisel
Recipients Olivier.Grisel, pitrou, serhiy.storchaka
Date 2017-11-09.22:21:23
SpamBayes Score -1.0
Marked as misclassified Yes
Message-id <1510266084.02.0.213398074469.issue31993@psf.upfronthosting.co.za>
In-reply-to
Content
More benchmarks with the unix time command:

```

(py37) ogrisel@ici:~/code/cpython$ git checkout master
Switched to branch 'master'
Your branch is up-to-date with 'origin/master'.
(py37) ogrisel@ici:~/code/cpython$ time python ~/tmp/large_pickle_dump.py --use-pypickle
Allocating source data...
=> peak memory usage: 2.014 GB
Dumping to disk...
done in 10.677s
=> peak memory usage: 5.936 GB

real	0m11.068s
user	0m0.940s
sys	0m5.204s
(py37) ogrisel@ici:~/code/cpython$ time python ~/tmp/large_pickle_dump.py --use-pypickle
Allocating source data...
=> peak memory usage: 2.014 GB
Dumping to disk...
done in 5.089s
=> peak memory usage: 5.978 GB

real	0m5.367s
user	0m0.840s
sys	0m4.660s
(py37) ogrisel@ici:~/code/cpython$ git checkout issue-31993-pypickle-dump-mem-optim 
Switched to branch 'issue-31993-pypickle-dump-mem-optim'
(py37) ogrisel@ici:~/code/cpython$ time python ~/tmp/large_pickle_dump.py --use-pypickle
Allocating source data...
=> peak memory usage: 2.014 GB
Dumping to disk...
done in 6.974s
=> peak memory usage: 2.014 GB

real	0m7.300s
user	0m0.368s
sys	0m4.640s
(py37) ogrisel@ici:~/code/cpython$ time python ~/tmp/large_pickle_dump.py --use-pypickle
Allocating source data...
=> peak memory usage: 2.014 GB
Dumping to disk...
done in 10.873s
=> peak memory usage: 2.014 GB

real	0m11.178s
user	0m0.324s
sys	0m5.100s
(py37) ogrisel@ici:~/code/cpython$ time python ~/tmp/large_pickle_dump.py --use-pypickle
Allocating source data...
=> peak memory usage: 2.014 GB
Dumping to disk...
done in 4.233s
=> peak memory usage: 2.014 GB

real	0m4.574s
user	0m0.396s
sys	0m4.368s
```

User time is always better in the PR than on master but is also much slower than system time (disk access) in any case. System time is much less deterministic.
History
Date User Action Args
2017-11-09 22:21:24Olivier.Griselsetrecipients: + Olivier.Grisel, pitrou, serhiy.storchaka
2017-11-09 22:21:24Olivier.Griselsetmessageid: <1510266084.02.0.213398074469.issue31993@psf.upfronthosting.co.za>
2017-11-09 22:21:23Olivier.Grisellinkissue31993 messages
2017-11-09 22:21:23Olivier.Griselcreate