Issue 10203: sqlite3.Row doesn't support sequence protocol

sqlite.Row class doesn't implement sequence protocol, which is rather unfortunate, because it is described and expected to work like a tuple, with extra mapping-like functionality.

Specific issue I hit:

Adding rows to PyGTK ListStore,

        model = gtk.ListStore(*db.getSchema())
        for r in listGen():
            model.append(r)

I get:

TypeError: expecting a sequence

Looking at PyGTK sources, append() method uses PySequence Check() on the argument. Looking at Python 2.6.5 abstract.c:

int
PySequence_Check(PyObject *s)
{
        if (s && PyInstance_Check(s))
                return PyObject_HasAttrString(s, "__getitem__");
        if (PyObject_IsInstance(s, (PyObject *)&PyDict_Type))
                return 0;
        return s != NULL && s->ob_type->tp_as_sequence &&
                s->ob_type->tp_as_sequence->sq_item != NULL;
}

And sqlite3.Row doesn't set ob_type->tp_as_sequence as of Py 2.6.5 or 2.7.

Hello Paul, thanks for the report.  The doc only describe Row as a tuple-like object, but tuple does implement the Sequence ABC, so I’m inclined to agree with you this is a bug and not a feature request (my first reaction).

Adding Georg, the maintainer of the module, to the nosy list (found his username in Misc/maintainers.rst).

(Gerhard, sorry, not well awake :)

Just as data point: the DB-API 2.0 requires that the row objects returned by the various .fetch*() methods are sequences, i.e. they need to implement the sequence protocol.

Hello! Here's a simple patch which makes sqlite.Row to act like a real sequence.

Patch looks good!  Are documentation changes needed?

I guess not, the documentation already states that Row tries to mimic a tuple in most of its features. Probably a MISC/News entry is required.

I got warning in compiling your patch:

gcc -pthread -fPIC -Wno-unused-result -Werror=declaration-after-statement -g -O0 -Wall -Wstrict-prototypes -DMODULE_NAME="sqlite3" -DSQLITE_OMIT_LOAD_EXTENSION=1 -IModules/_sqlite -I/usr/include -I./Include -I. -IInclude -I/usr/include/x86_64-linux-gnu -I/usr/local/include -I/home/ethan/Documents/code/python/cpython3.4/Include -I/home/ethan/Documents/code/python/cpython3.4 -c /home/ethan/Documents/code/python/cpython3.4/Modules/_sqlite/row.c -o build/temp.linux-x86_64-3.4-pydebug/home/ethan/Documents/code/python/cpython3.4/Modules/_sqlite/row.o
/home/ethan/Documents/code/python/cpython3.4/Modules/_sqlite/row.c:212:28: warning: initialization from incompatible pointer type [enabled by default]
/home/ethan/Documents/code/python/cpython3.4/Modules/_sqlite/row.c:212:28: warning: (near initialization for ‘pysqlite_row_as_sequence.sq_item’) [enabled by default]

$ gcc --version
gcc (Ubuntu/Linaro 4.7.2-2ubuntu1) 4.7.2

Thanks, Vajrasky! Here's an updated patch.

What can be done to move this forward?

I've changed the versions, hope I've got them correct.

Thanks Mark.  There will be a last 3.3 release with bugfixes before it switches to security fixes only.

I would like another core developer more intimate with C to review the patch.

Ah, 3.3 won’t follow that custom given that it had a bugfix release very recently.

Issue #13583 ("sqlite3.Row doesn't support slice indexes") is partly related.

LGTM. Perhaps it is worth to add a test for negative indices (valid (-1) and invalid (< -length)).

Thanks. Here's the updated patch. It supports negative indeces (my previous patch didn't do that).

Looks good, but there is one detail. Whith issue10203.patch when integer index overflows C long, sqlite3.Row.__getitem__() doesn't raise an exception. Instead overflow exception is raised later.

>>> import sqlite3
>>> con = sqlite3.connect(":memory:")
>>> con.row_factory = sqlite3.Row
>>> row = con.execute("select 1 as a, 2 as b").fetchone()
>>> row[2**1000]
2
>>> 
OverflowError: Python int too large to convert to C long

Thanks. Here's a fix.

Compare with tuple:

>>> (1, 2)[2**1000]
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
IndexError: cannot fit 'int' into an index-sized integer

Thanks. Patch modified.

New changeset 6e2833ae1718 by Serhiy Storchaka in branch '2.7':
Issue #10203: sqlite3.Row now truly supports sequence protocol.  In particular
http://hg.python.org/cpython/rev/6e2833ae1718

New changeset 6af865f1a59d by Serhiy Storchaka in branch '3.4':
Issue #10203: sqlite3.Row now truly supports sequence protocol.  In particulr
http://hg.python.org/cpython/rev/6af865f1a59d

New changeset 474c97a5f0c8 by Serhiy Storchaka in branch 'default':
Issue #10203: sqlite3.Row now truly supports sequence protocol.  In particulr
http://hg.python.org/cpython/rev/474c97a5f0c8

Thank you Claudiu for your contribution.

But please be more careful, your patches contained trailing whitespaces.

Thank you Paul for your report.

stage: commit review -> resolved

messages: + msg219208

messages: + msg219179

messages: + msg219103

messages: + msg213868
versions: + Python 2.7, Python 3.4

messages: + msg205201

messages: + msg205172
stage: patch review

messages: + msg204789

keywords: + patch

messages: + msg119698
versions: + Python 3.1, Python 3.2, - Python 2.6