Python code style

This note contains several remarks on coding style, programming principles, Python idioms and conventions.

Style

For PEP8, read https://peps.python.org/pep-0008/ .

Following the official PEP8 code style guidelines has major advantages. Because it's officially endorsed it's widely used and many tools, such as linters, exist that automatically check for PEP8 compliance.

So the easiest way to learn is to install a linter that gives you errors for code that does not comply to PEP8. I can recommend flake8. I run flake8 every time I save a file in Vim. You probably have some smart IDE that does something similar :-).

Language

PEP8 recommends using English for comments and documentation unless you are 100% sure your code will never be read by someone not native in your language of choice. For consistency, it makes sense to also use English for other naming conventions.

PEP8 highlights

Best is to just read PEP8 once, but here are some highlights:

  • Use 4 spaces per indentation level and not TABS
    • You probably use TAB but make sure your editor converts them to spaces
    • Spaces and tabs for indentation may not be mixed
  • The line length for blocks of text (such as docstrings or comments) should be limited to 72 characters
  • Line length of code should be limited to 79 characters
    • This makes things like code review, where you have code side by side, much easier
    • Many tools wrap after 80 characters
    • If everyone agrees, we may increase this to 99 characters.
  • Blank lines
    • Top-level functions (so not inside a class) should be surrounded by two blank lines
    • Method definitions are surrounded by a single blank line
    • Use blank lines in functions sparingly to indicate logical sections within a function
  • Imports
    • After module comments and docstrings, but before module globals and constants
    • Use absolute imports (more details) unless there is a very good reason not to do so
    • First standard library imports, then third party imports, then local imports
    • Avoid wildcard imports from module x import *.
  • Naming properties
    • _single_leading_underscore indicates this object is for "internal use" and should not be called from outside the module (non-public; but Python does not enforce this)
    • __double_leading_underscore applies name mangling so that the class name is prepended to the name: __bar in class Foo becomes _Foo__bar.
      • Best to avoid. You typically do not need this!
    • __double_leading_and_trailing__: reserved "magic" objects such as __init__. Only use the documented cases, do not make these yourself.
    • trailing_underscore_: this is a convention used to avoid name clashes with reserved keywords, e.g. class_
  • Naming modules: short, all-lowercase names; underscores are allowed but discouraged
  • Naming classes: use the CapWords convention
  • Naming functions and variables: lowercase words separated by underscores as necessary
  • Naming constants: capital letters with underscores, e.g. MAX_OVERFLOW
  • Spaces
    • Have spaces around almost all operators, e.g.: i = i + 1
    • Exception: x = x*2 - 1 (notice how * has a higher priority)
    • Exception for logical grouping: c = (a+b) * (a-b)
    • Do not have spaces around keyword args: def complex(real, imag=0.0): ...
    • But do use spaces when using type annotations: def munge(sep: AnyStr = None): ...
  • Do not have trailing spaces (good to have a tool check for it!)
    • E.g. they are not visible but may break line continuations with a backslash
  • Comments
    • In English
    • Inline comments should be separated by at least two spaces from the code. Use them sparingly
  • Arguments
    • Use self as first argument for instance methods
    • Use cls as first argument to class methods
  • Python files are always encoded as UTF-8 and do not require an encoding declaration

Type hinting

Python nowadays supports type hinting. At the moment of writing Python itself does not do type checking, but several external tools do.

The proposal is to aim to use type hints as much as possible during development, but be more strict about type hints in production code. Using type hints makes it very explicit which arguments a function expects and what it returns, that is, you sign a "contract" about what a function does. Even if you don't use type checking tools, this improves the readability of the code and helps collegues more quickly assess what a funtion is supposed to do.

This is a simple example, see PEP484 for more details:

def greeting(name: str) -> str:
    return 'Hello ' + name

Docstring

The goal is to settle on a consistent docstring format that can be extracted by autodocumenting tools. A well-known tool for autodocumentation is called Sphinx. By default Sphinx uses the reST (reStructuredText) docstring format. This format is also the format auto-generated in PyCharm. The Pyment tool for managing docstring also outputs this format by default. In other words, the reST format is a good candidate as our default docstring format.

The reST format was proposed in PEP287. It's very similar to the well-known javadoc style. An earlier and more general description of docstring can be found in PEP257.

This docstring format looks like this:

"""
This is a reST style docstring.

:param param1: this is a first param
:type param1: str
:param param2: this is a second param
:type param2: int
:returns: this is a description of what is returned
:rtype: bool
:raises keyError: raises an exception
"""

The Pyment documentation lists some examples of docstring formats.

Conventions

To read the "Zen of Python" open a Python REPL and run import this.

Do not have two disjointed statements on the same line. The exception are list comprehensions, that are essentially compounded statements

Only use *args and **kwargs when there are good reasons to, i.e. if positional and keyword arguments do not suffice.

Use the "truthy" value of Python constants. Not: if attr == True but if attr:.

Access dictionary elements by their key. With d = {'hello': 'world'}, do not use if d.has_key('hello'): but if 'hello' in d:.

Only use a list comprehension if you need to use the resulting list. If you just want to loop through a list once and compute something once, use a generator expression (same syntax but round instead of square brackets).

Do not use a list comprehension for "side-effects", e.g. no [print(x) for x in [1,2,3]].

If you want to keep a counter in a loop, use the enumerate() function instead of manually keeping a counter:

for i, item in enumerate(list(range(10))):
    print(i, item)

Never remove items from a list while you are iterating through it. Instead, filter using a list comprehension or generator expression: [value for value in sequence if value != x]

When modifying a list, be aware of possible side effects. Let's say we define a list and then let another variable reference it. Then these two variables refer to the same object in memory:

seq = list(range(10))  # [0 .. 9]
seq_dup = seq
assert id(seq) == id(seq_dup)  # They reference the same object

The question now is, if we modify list seq (short for sequence), does the change also propagate to seq_dup?

In this case it does not because we assign a new list to seq:

# Filtering the value 4 with a list comprehension
seq = [value for value in seq if value != 4]
assert id(seq) == id(seq_dup)  # Fails because `seq` is a new list!

But if we use the slice assignment syntax, we modify the original list in place so that both variables still refer to the same object:

seq = list(range(10))  # [0 .. 9]
seq_dup = seq
assert id(seq) == id(seq_dup)  # They reference the same object

# Filtering the value 4 with a list comprehension
seq[::] = [value for value in seq if value != 4]
assert id(seq) == id(seq_dup)  # Passes!

This is because assignment does not create a new object. So actually, be aware that assignment has side effects and that often it's safer to create a new list.

This example has side-effects in b:

# Add three to all list members.
a = [3, 4, 5]
b = a                     # a and b refer to the same list object

for i in range(len(a)):
    a[i] += 3             # b[i] also changes

print(a == b)             # True

But this example does not:

a = [3, 4, 5]
b = a

# assign the variable "a" to a new list without changing "b"
a = [i + 3 for i in a]
print(a == b)  # False

Be aware of circular imports, i.e. module X import module Y, and module Y importing module X. See here why.

When reading from a file, use the with syntax, because this handles the closing of the file even if an exception occurs in your code before the f.close() statement is reached:

with open('file.txt') as f:
    for line in f:
        print(line)

If you have a too long line, you can use a line continuation in two ways. (But first check if having such a long line can be avoided to begin with!) Either have a backslash at the end of the line:

long_string = """Let's say this string is \
    too long for one line."""

But note that this breaks if you have a white space behind your backslash. You can also often use brackets:

from some.module import (
    a, b, c, d
)

Also have a look at the programming recommendations from PEP8. They are a bit more specific but worth reading.

Python Idioms, or: how to write "Pythonic" code

Unpacking:

a, (b, c) = 1, (2, 3)
a, *rest = [1, 2, 3]  # a = 1, rest = [2, 3]
a, *middle, c = [1, 2, 3, 4]  # a = 1, middle = [2, 3], c = 4

Ignoring during unpacking:

a, _, c = 1, 2, 3

Unpacking iterables:

tup = ('h', 'o', 'i')
print(*tup)  # prints "h o i"

l = ['h', 'o', 'i']
print(*l)  # prints "h o i"

You can initialize a flat list of length 4 as such: [None] * 4 but this does not work for a list of lists. Then you need a list comprehension [[] for _ in range(4)], see here why.

Joining a list of strings by joining on an empty string:

word = ''.join(['h', 'o', 'i', '!'])

Use a set or dictionary instead of lists when:

  • The collection will contain a large number of items
  • You will be repeatedly searching for items in the collection
  • You do not have duplicate items.

This is because look ups in sets or dictionary are hash-based, whereas a look up in a list requires linear traversal of the list.

Programming Principles

  • Don't repeat yourself
    • If you find yourself writing something twice, replace it with an abstraction such as a function or a class
    • This way you avoid that you have to change the functionality in multiple places instead of one
    • Then "a modification of any single element of a system does not require a change in other logically unrelated elements"
  • YAGNI: You aren't gonna need it
    • Only add functionality when it's actually necessary and needed
    • For example, do not add keyword function arguments just in case you need them in the future. Removing these (and the coupled logic) is much more work than adding a keyword argument when you actually need it.

Sources

  • https://docs.python-guide.org/writing/style/
  • https://peps.python.org/pep-0008/#programming-recommendations
  • https://peps.python.org/pep-0484/
  • https://peps.python.org/pep-0257/