If you want to write a forking server, one obvious way to do it is to use the multiprocessing module to create a pool of processes, then hand off requests as you get them.

The problem is that handling a request usually involves reading from and writing to a client socket. So, you need to send the socket object as part of the request. And you can't pickle sockets.

There are easy ways around this for some cases:

For protocols without much data per request and response, there's an easy solution: the main process does the reads, dispatches a buffer instead of a socket, receives back a buffer, and does the writes. But that won't work for, e.g., streaming large files.

I was recently trying to optimize some C code using cachegrind, and discovered that branch misprediction in an inner loop was the culprit. I began wondering how much anything similar could affect Python code.
3

A lot of people have questions like this:

I've got a 100MB CSV file. I read it in to a list, populated with a csv.DictReader, and my computer starts swapping. Why? Let's look at what it takes to store a 100MB file as a list of dicts of strings.

But first, let's look at how to solve this problem, instead of how to understand it.
1

There are hundreds of questions on StackOverflow from people who want to know something like, "How do I edit a file in-place, without creating a new file."

In general, the answer is, "You can't. You really do want to create a new file." (There is an exception, which I'll get to later.)

Why? Well, files are just streams of bytes—or, for text files, streams of characters.
Blog Archive
About Me
About Me
Loading
Dynamic Views theme. Powered by Blogger. Report Abuse.