Python, list comprehension for efficiency and readability

Added 9 Feb 2023, 9:17 a.m. edited 18 Jun 2023, 1:12 a.m.

List comprehension is a fantastic part of Python, a feature sometimes overlooked but is something that should be part of every Python coders toolkit, if you're not familiar with the advantages you really should learn how they can help.

Often I'm more likely to prefer more verbose code over cryptic ascii soup, simply because its just far easier to maintain, its likely far more readable (especially years later)

If I'm honest the first time I put down some code, it is often a quick rough version to get things working, frequently its only then it becomes obvious how you should have been done.

Let take a look at an initial version of a short piece of code, we want to present a comma separated list of items from a database (exception handling removed for the sake of clarity)

        srs = conn2.curs.fetchall()
        sks = []
        for s in srs:
        p.add_tag('span', text= ", ".join(sks))

Not particularly "pythonic" nor for that matter efficient, but it does work and it is fairly evident what it does. This is where list comprehension is a real boon, without loosing readability and as a bonus gaining efficiency we can improve the creation of the string.

        srs = conn2.curs.fetchall()
        sks = [s['stack'] for s in srs]
        p.add_tag('span', text=", ".join(sks))

When reading list comprehension code, I find it useful to start at the for then finish with the start of the line, so just as in the first example we're iterating the result set and pulling out a particular field. No loss of readability, less code and more efficiency. Genuinely a win, win, situation

But is there too much of a good thing? can we take this even further?

        srs = conn2.curs.fetchall()
        p.add_tag('span', text=", ".join(s['stack'] for s in srs))

By putting the list comprehension directly inside the join call, we're down to a single line of code, without the need of an extra variable, and to boot its still just as readable. Obviously I dare say if you try hard enough you could use list comprehension to obfuscate and confuse but used correctly, it is genuinely a very useful feature of the language.

Of course inside the Python VM a loop is still being executed, but its likely optimised, especially in the case where you're not creating an extra variable, and where appropriate it may be that map might be faster, but meeting the language half way will often result in more efficient code. Don't forget if in doubt and where speed is important there is nothing wrong with making multiple versions of your code and timing them, just don't assume one use case will always be the same in every situation.