bpo-33873: Fix bug in `runtest.py` and add checks for invalid `-R` parameters #7735

pablogsal · 2018-06-15T22:59:37Z

https://bugs.python.org/issue33873

… an extra time than specified

pablogsal · 2018-06-15T23:43:52Z

I have added the checks for the --huntrleaks/-R in Regrtest._main to avoid entering Regrtest.run_test and do the sys.exit() there but I am not sure what is the best location. Also, I used a new errorcode (2) but again, not sure if we want a custom error code for that.

pablogsal · 2018-06-15T23:45:54Z

~/github/cpython bpo33873 ✔ 8s
❯ ./python -m test -R 0:0 test_os -m test_access
Invalid values for the --huntrleaks/-R parameters
                                                                                                                                               
~/github/cpython bpo33873 ✔
❯ ./python -m test -R 0:1 test_os -m test_access
Invalid values for the --huntrleaks/-R parameters
                                                                                                                                               
~/github/cpython bpo33873 ✔
❯ ./python -m test -R 1:0 test_os -m test_access
Invalid values for the --huntrleaks/-R parameters
                                                                                                                                               
~/github/cpython bpo33873 ✔
❯  ./python -m test -R 0:-2 test_list
Invalid values for the --huntrleaks/-R parameters

vstinner · 2018-06-16T00:22:45Z

Lib/test/libregrtest/main.py

+        if self.ns.huntrleaks:
+            warmup, repetitions, _ = self.ns.huntrleaks
+            if warmup < 1 or repetitions < 1:
+                print("Invalid values for the --huntrleaks/-R parameters",


You must mention the minimum number of warmups and minimum number of repetitions (1 and 1).

vstinner

IMHO a NEWS entry would be worth it here. You can use the name "regrtest" to name the CPython test runner libregrtest.

vstinner · 2018-06-20T10:37:23Z

Lib/test/libregrtest/main.py

+            if warmup < 1 or repetitions < 1:
+                msg = ("Invalid values for the --huntrleaks/-R parameters. The "
+                       "number of warmups and repetitions must be at least 1 "
+                       "each (1:1).")


Nitpick: maybe write "(-R 1:1)" rather than "(1:1)"?

vstinner

LGTM.

Since you are now a core dev, would you like to merge it yourself? :-)

vstinner · 2018-06-20T10:39:13Z

"VSTS: docs — docs_20180616.05 failed"

I don't understand the failure. It says "Initialize phase: Failed" but I don't see any failure in logs. I suggest to ignore this error.

vstinner · 2018-06-26T11:47:35Z

@pablogsal: The change LGTM, do you want to merge it? It would help me to work on my PR #7827.

pablogsal · 2018-06-26T14:17:51Z

Merged! ✨

miss-islington · 2018-06-26T21:05:43Z

Thanks @pablogsal for the PR 🌮🎉.. I'm working now to backport this PR to: 3.7.
🐍🍒⛏🤖

miss-islington · 2018-06-26T21:05:43Z

Thanks @pablogsal for the PR 🌮🎉.. I'm working now to backport this PR to: 3.6.
🐍🍒⛏🤖

bedevere-bot · 2018-06-26T21:07:03Z

GH-7933 is a backport of this pull request to the 3.7 branch.

…rameters (pythonGH-7735) Fix bug in `Lib/test/libregrtest/runtest.py` that makes running tests an extra time than the specified number of runs. Add check for invalid --huntrleaks/-R parameters. (cherry picked from commit 58ed730) Co-authored-by: Pablo Galindo <Pablogsal@gmail.com>

bedevere-bot · 2018-06-26T21:07:54Z

GH-7934 is a backport of this pull request to the 3.6 branch.

…rameters (pythonGH-7735) Fix bug in `Lib/test/libregrtest/runtest.py` that makes running tests an extra time than the specified number of runs. Add check for invalid --huntrleaks/-R parameters. (cherry picked from commit 58ed730) Co-authored-by: Pablo Galindo <Pablogsal@gmail.com>

…rameters (pythonGH-7735) Fix bug in `Lib/test/libregrtest/runtest.py` that makes running tests an extra time than the specified number of runs. Add check for invalid --huntrleaks/-R parameters. (cherry picked from commit 58ed730)

* bpo-33718: regrtest: use format_duration() to display failed tests (GH-7686) * Enhance also format_duration(): work on integers and rounds towards +infinity (math.ceil). * Write unit tests on format_duration() (cherry picked from commit 4ffe9c2) * bpo-33873: regrtest: Add warning on -R 1:3 (GH-7736) regrtest: Add warning when using less than 3 warmup runs like -R 1:3. (cherry picked from commit cac4fef) * bpo-33873: Fix bug in `runtest.py` and add checks for invalid `-R` parameters (GH-7735) Fix bug in `Lib/test/libregrtest/runtest.py` that makes running tests an extra time than the specified number of runs. Add check for invalid --huntrleaks/-R parameters. (cherry picked from commit 58ed730)

* bpo-33718: regrtest: use format_duration() to display failed tests (pythonGH-7686) * Enhance also format_duration(): work on integers and rounds towards +infinity (math.ceil). * Write unit tests on format_duration() (cherry picked from commit 4ffe9c2) * bpo-33873: regrtest: Add warning on -R 1:3 (pythonGH-7736) regrtest: Add warning when using less than 3 warmup runs like -R 1:3. (cherry picked from commit cac4fef) * bpo-33873: Fix bug in `runtest.py` and add checks for invalid `-R` parameters (pythonGH-7735) Fix bug in `Lib/test/libregrtest/runtest.py` that makes running tests an extra time than the specified number of runs. Add check for invalid --huntrleaks/-R parameters. (cherry picked from commit 58ed730) (cherry picked from commit d1f9481)

* bpo-33718: regrtest: use format_duration() to display failed tests (GH-7686) * Enhance also format_duration(): work on integers and rounds towards +infinity (math.ceil). * Write unit tests on format_duration() (cherry picked from commit 4ffe9c2) * bpo-33873: regrtest: Add warning on -R 1:3 (GH-7736) regrtest: Add warning when using less than 3 warmup runs like -R 1:3. (cherry picked from commit cac4fef) * bpo-33873: Fix bug in `runtest.py` and add checks for invalid `-R` parameters (GH-7735) Fix bug in `Lib/test/libregrtest/runtest.py` that makes running tests an extra time than the specified number of runs. Add check for invalid --huntrleaks/-R parameters. (cherry picked from commit 58ed730)

* bpo-33718: regrtest: use format_duration() to display failed tests (GH-7686) * Enhance also format_duration(): work on integers and rounds towards +infinity (math.ceil). * Write unit tests on format_duration() (cherry picked from commit 4ffe9c2) * bpo-33873: regrtest: Add warning on -R 1:3 (GH-7736) regrtest: Add warning when using less than 3 warmup runs like -R 1:3. (cherry picked from commit cac4fef) * bpo-33873: Fix bug in `runtest.py` and add checks for invalid `-R` parameters (GH-7735) Fix bug in `Lib/test/libregrtest/runtest.py` that makes running tests an extra time than the specified number of runs. Add check for invalid --huntrleaks/-R parameters. (cherry picked from commit 58ed730) (cherry picked from commit d1f9481)

…rameters (pythonGH-7735) Fix bug in `Lib/test/libregrtest/runtest.py` that makes running tests an extra time than the specified number of runs. Add check for invalid --huntrleaks/-R parameters.

the-knights-who-say-ni added the CLA signed label Jun 15, 2018

bedevere-bot added the awaiting review label Jun 15, 2018

Fix bug in Lib/test/libregrtest/runtest.py that makes running tests…

0f0ea7f

… an extra time than specified

pablogsal force-pushed the bpo33873 branch from c56c94a to 0f0ea7f Compare Jun 15, 2018

Add check for invalid --huntrleaks/-R parameters

bc1c874

pablogsal force-pushed the bpo33873 branch from aee5345 to bc1c874 Compare Jun 15, 2018

pablogsal changed the title ~~bpo-33873: Fix bug in runtest.py that makes running tests an extra time~~ bpo-33873: Fix bug in runtest.py and add checks for invalid -R parameters Jun 15, 2018

vstinner reviewed Jun 16, 2018

View changes

pablogsal added 2 commits Jun 16, 2018

fixup! Add check for invalid --huntrleaks/-R parameters

7510359

Add News entry

7a5ffc3

pablogsal force-pushed the bpo33873 branch from 8a0b6c0 to 7a5ffc3 Compare Jun 16, 2018

vstinner reviewed Jun 20, 2018

View changes

vstinner approved these changes Jun 20, 2018

View changes

bedevere-bot added awaiting merge and removed awaiting review labels Jun 20, 2018

pablogsal merged commit 58ed730 into python:master Jun 26, 2018
8 of 9 checks passed

bedevere-bot removed the awaiting merge label Jun 26, 2018

pablogsal deleted the bpo33873 branch Jun 26, 2018

vstinner added needs backport to 3.6 needs backport to 3.7 labels Jun 26, 2018

bedevere-bot removed the needs backport to 3.7 label Jun 26, 2018

bedevere-bot removed the needs backport to 3.6 label Jun 26, 2018

vstinner mentioned this pull request Jun 26, 2018

bpo-33873: Backport regrtest from master to 3.7 #7935

Merged

vstinner mentioned this pull request Jun 26, 2018

[2.7] bpo-33873: Backport regrtest from master #7936

Merged

vstinner mentioned this pull request Jun 26, 2018

[3.6] bpo-33873: Backport regrtest from master to 3.6 (GH-7935) #7937

Merged

python / cpython Public

bpo-33873: Fix bug in `runtest.py` and add checks for invalid `-R` parameters #7735

bpo-33873: Fix bug in `runtest.py` and add checks for invalid `-R` parameters #7735

pablogsal commented Jun 15, 2018 •

edited by bedevere-bot

pablogsal commented Jun 15, 2018

pablogsal commented Jun 15, 2018

vstinner Jun 16, 2018

vstinner left a comment

vstinner Jun 20, 2018

vstinner left a comment

vstinner commented Jun 20, 2018

vstinner commented Jun 26, 2018

pablogsal commented Jun 26, 2018

miss-islington commented Jun 26, 2018

miss-islington commented Jun 26, 2018

bedevere-bot commented Jun 26, 2018

bedevere-bot commented Jun 26, 2018

python / cpython Public

bpo-33873: Fix bug in runtest.py and add checks for invalid -R parameters #7735

bpo-33873: Fix bug in runtest.py and add checks for invalid -R parameters #7735

Conversation

pablogsal commented Jun 15, 2018 • edited by bedevere-bot

pablogsal commented Jun 15, 2018

pablogsal commented Jun 15, 2018

vstinner Jun 16, 2018

Choose a reason for hiding this comment

vstinner left a comment

vstinner Jun 20, 2018

Choose a reason for hiding this comment

vstinner left a comment

vstinner commented Jun 20, 2018

vstinner commented Jun 26, 2018

pablogsal commented Jun 26, 2018

miss-islington commented Jun 26, 2018

miss-islington commented Jun 26, 2018

bedevere-bot commented Jun 26, 2018

bedevere-bot commented Jun 26, 2018

bpo-33873: Fix bug in `runtest.py` and add checks for invalid `-R` parameters #7735

bpo-33873: Fix bug in `runtest.py` and add checks for invalid `-R` parameters #7735

pablogsal commented Jun 15, 2018 •

edited by bedevere-bot