Stupid Python Ideas

The first time you try to write a network client or server directly on top of sockets, you do something like this:

for filename in filenames: with open(filename, 'rb') as f: sock.sendall(f.read())

And then, on the other side:

for i in count(0): msg = sock.recv(1<<32) if not msg: break with open('file{}'.format(i), 'wb') as f: f.write(msg)

At first, this seems to work, but it fails on larger files. Or as soon as you try to use it across the internet. Or 1% of the time. Or when the computer is busy.

In reality, it can't possibly work, except in special circumstances. A TCP socket is a stream of bytes. Every time you call send (or sendall), you put more bytes on that stream. Every time you call recv, you get some or all of the bytes on the stream.

(Someone pointed out to me that Ned Batchelder has a similar post called Keep data out of your variable names. As usual for him, he's got a very concise answer that covers everything that matters in only a few paragraphs, so you might want to just read that.)

One of the most common Python questions on StackOverflow is some variation of "How do I create a bunch of variables in a loop?"

The answer is: You don't.

You can…

The direct answer is simple.

Novices to Python often come up with code that tries to build and evaluate strings, like this:

for name in names: exec('{} = {}'.format(name, 0))

… or this:

exec('def func(x): return {} * x**2 + {} * x + {}'.format(a, b, c)) return func

99% of the time, this is a problem you shouldn't be solving in the first place. (In the first case, you should almost certainly be using a dictionary or namespace full of names, instead of a bunch of dynamically-created variables.

If you look at the major changes in Python 3, other than the Unicode stuff, most of them are about replacing list-based code with iterator-based code. When you run 2to3 on your code, every map(x, seq) or d.items() gets turned into list(map(x, seq)) or list(d.items()), and if you want to write dual-version code you have to use things like six.map(x, seq) or list(six.iteritems(d)).

So, why?

Often, you don't really need a list, you just need something you can iterate over.