Review of Twisted Network Programming Essentials

Abstract

A review of the book Twisted Network Programming Essentials by Abe Fettig.

Updated on 2005-12-07: added one contributed errata entry.

Disclosure

I have commit access to the Twisted source tree. I've been working with the project for quite some time. I would probably say good things about any book that mentioned Twisted.

Please also see my review of Foundations of Python Network Programming, as I will be comparing these two books.

And, in case you didn't read the review above and don't know me, let me restate this: I am not going to write a glowing fluffy review, give a link to amazon and increase sales -- even if I was bribed with a shiny book, which I do appreciate -- but to really dig into this book, see how good it is. Blame it on me being an average pessimistic Finn, or something. Take it this way: everything I don't complain about is just perfect.

First Impressions

The book looks just like all the other O'Reilly books. Of course, that is a good thing, as the O'Reilly standard for quality is quite high. Leafing through the book, I made two initial remarks. First, the table of contents is has a very light feel to it, and is actually very readable. And second, there is a lot of long source code listings in the book, which is sort of a pet peeve of mine. I hate page breaks in source code, especially when the pages are duplex and bound together, like they are in a book.

An Overview

A bit less than a dozen pages at the beginning are dedicated to installing Twisted and getting started. The next dozen pages walk us through some very basic examples showing how to write TCP clients and servers, including an echoserver.py example, which is practically identical to the one you can find in the tutorials of the Twisted project itself.

Writing web clients, both with getPage() and by subclassing HTTPClientFactory, is covered on 13 pages. Server-side web things are given 25 pages, but note that this does not contain twisted.web2, Nevow, or just about anything beyond basic twisted.web.resource.Resource. There is about a page worth of explanation on page 54 on how the web server things covered in the book are old and will be deprecated, and says that anything newer is going through too many changes to document in a book. That is, do not buy this book to help you do web things.

REST, XML-RPC, SOAP and pb are all crammed in about twenty pages, basically implementing the same simple dict manipulation calls with each mechanism.

Twisted's authentication mechanism, twisted.cred, gets 20 pages of its own, and the topic is discussed quite thoroughly.

Writing simple clients for POP3, SMTP, and IMAP are covered, with a few pages each. Server-side implementations for the same protocols are demonstrated in 5-14 pages each.

NNTP, including a client and server implementation and some background, is given ten pages.

Using Twisted's implementation of SSH, known as Conch, gets explained in 10 pages for server side and a few pages for client side.

The final 15 pages of the book briefly explains twistd, the concept of dropping root privileges, and talks about logging.

There are practically no tables or figures in the book, right now I can't find anything except the twisted.cred overview on page 92, which is basically identical with the one in the actual Cred HOWTO.

The Python in the Book

The book is written in clear and concise English, with surprisingly few typos. That is very good and makes reading the explanations quite easy. But, when it comes the technical books, I'm more concerned about the technical things. When the topic is programming, the examples are what fail or save the book.

All in all, I'm quite happy with the examples in this book. It is mostly readable and easy to follow. There are, however, a few things that left me wishing for more.

Representing source code listings in a book form is always challenging. What ruins parts of this book is the tendency to show big chunks of code, spanning multiple pages. I constantly lost my place when a class or a function was broken between pages, and found myself having to turn back two pages before being reminded of the name of the class I was looking at. For contrast, Foundations of Python Network Programming managed to introduce its concepts iteratively, with most of the source code listings being less than one page.

Some of the things I encountered in the listings I considered a bit unpythonic, or they seemed old-fashioned to me. Here are some examples, with my preferred forms below them. These repeated through the book.

# Part of example 5-1, `wiki.py`, on page 65.
def hasPage(self, path):
    return self.pages.has_key(path)
def hasPage(self, path):
    return path in self.pages

or even just

def __contains__(self, path):
    return path in self.pages
# Part of example 8-1, `smtpserver.py`, on page 124.
if not user.dest.domain in self.validDomains:
	...
if user.dest.domain not in self.validDomains:
	...
# Part of example 5-1, `wiki.py`, on page 66.
def deletePage(self, path):
	del(self.pages[path])
def deletePage(self, path):
	del self.pages[path]
# Part of example 7-3, `smtpsend.py`, on page 113.
attachment.add_header('Content-Disposition', 'attachment',
					  filename=os.path.split(filename)[1])
attachment.add_header('Content-Disposition', 'attachment',
					  filename=os.path.basename(filename))
# Part of example 7-5, `imapfolders.py`, on page 117.
def __gotMailboxList(self, list):
	return [boxInfo[2] for boxInfo in list]
def __gotMailboxList(self, mailboxes):
	return [name for flags, delimiter, name in mailboxes]
# Part of example 8-4, `imapserver.py`, on page 141.
def initMetadata(self):
	if not self.metadata.has_key('flags'):
		self.metadata['flags'] = {} # dict of message IDs to flags
	if not self.metadata.has_key('uidvalidity'):
		self.metadata['uidvalidity'] = random.randint(1000000, 9999999)
	if not self.metadata.has_key('uids'):
		self.metadata['uids'] = {}
def initMetadata(self):
	self.metadata.setdefault('flags', {})
	self.metadata.setdefault('uidvalidity', random.randint(1000000, 9999999))
	self.metadata.setdefault('uids', {))
# Part of example 10-1, `sshserver.py`, on page 174.
publicMethods = filter(
	lambda funcname: funcname.startswith('do_'), dir(self))
commands = [cmd.replace('do_', '', 1) for cmd in publicMethods]
PREFIX = 'do_'
commands = [cmd[len(PREFIX):] for cmd in dir(self) if cmd.startswith(PREFIX)]

Sometimes the data structures used are not really suitable for the task at hand.

# Part of example 8-4, `imapserver.py`, on page 143.
... gather message UIDs in list allUIDs ...
for uid in messageSet:
	...
	if uid in allUIDs:
		sequence = allUIDs.index(uid)+1
		...

which walks through the list a lot, once for if and once for index.

In at least one case, poor data structure choice actually produces visible defects:

# Part of example 5-7, `pb_wiki.py`, on page 84.
...
neededPages = []
for page in allPages:
	...
	for pageName in wikiWords:
		if not self.wikiData.hasPage(pageName):
			neededPages.append(pageName)

If multiple WikiNames point to a missing page, neededPages will have duplicates. The sample output on page 86 actually shows such a case. Just making neededPages a Set would fix this, make the example a bit faster, and most of all would be the right data type for the task. Missing pages have no inherent ordering, but the question "is this page missing?" is asked a lot.

Throughout the book, I kept wondering about the file modes. Why open a file for writing if you only read from it? But my point with this example was the "don't look before you jump, just fail gracefully" idiom. Just let the file open fail, handle the error, and if it was caused by file not existing, use the default content.

# Part of example 5-1, `wiki.py`, on page 65.
if os.path.exists(filename):
	self.pages = pickle.load(file(filename, 'r+b'))
else:
	self.pages = {'WikiHome': 'This is your Wiki home page.'}
# My version of `wiki.py`.
self.pages = {'WikiHome': 'This is your Wiki home page.'}
try:
	f = file(filename, 'rb')
except IOError, e:
	if e.errno == errno.ENOENT:
		pass
	else:
		raise
else:
	self.pages = pickle.load(f)
	f.close()

There are other similar things, like closing and reopening a file for no gain. Oh, and relying on Python GC to close the file early so the content written is actually flushed to the filesystem!

# Part of example 3-4, `validate.py`, on page 30.
def showPage(pageData):
	# write data to temp .html file, show file in browser
	tmpfd, tmp = tempfile.mkstemp('.html')
	os.close(tmpfd)
	file(tmp, 'w+b').write(pageData)
	webbrowser.open('file://' + tmp)
	...
# My version of `validate.py`.
def showPage(pageData):
	"""Write data to temp .html file, show file in browser."""
	tmpfd, tmp = tempfile.mkstemp('.html')
	f = os.fdopen(tmpfd, 'w')
	f.write(pageData)
	f.close()
	webbrowser.open('file://' + tmp)
	...

There's also a bit of clumsy regexp use in the wiki example.

# Part of example 5-1, `wiki.py`, from pages 65-66.
reWikiWord = re.compile(r'\b(([A-Z][a-z]+){2,})\b')
...
wikiWords = [match[0] for match in reWikiWord.findall(pageData)]
for word in wikiWords:
	pageData = pageData.replace(
		word, "<a href='%s'>%s</a>" % (word, word))
return pageData

Why define two matching groups if you don't want more than one?

# My version of `wiki.py`.
reWikiWord = re.compile(r'\b((?:[A-Z][a-z]+){2,})\b')
...
for word in reWikiWord.findall(pageData):
	pageData = pageData.replace(
		word, "<a href='%s'>%s</a>" % (word, word))
return pageData

Why do a clumsy manual loop if the substitution isn't any smarter than that?

# My version of `wiki.py`, refactored.
reWikiWord = re.compile(r'\b(([A-Z][a-z]+){2,})\b')
...
s, count = reWikiWord.subn(r'<a href="\1">\1</a>', pageData)
return s

Of course, the proper thing to do would be to actually output valid XML, regardless of input data. This is a trend throughout the book, everything involving HTML is just tag soup. Hopefully a later book on Nevow will clean things up.

Even outside HTML, the output formats of the examples are a mess. Most of the examples embedding user data in the output do nothing to escape potentially malicious strings. Adding quotes is generally done with "foo '%s'" % evil, when the even simpler "foo %r" % evil would work a lot better.

In example 5-1, wiki.py, class WikiData consists almost purely of application logic, but then gets a getRenderedPage method that renders it to HTML. Cleaner separation of responsibilities would have been, well, cleaner. The same example also manages to disappoint me by putting operations like "edit" and "save" in the same namespace as wiki pages -- that's a mistake I expect to see in a poorly-written PHP application, not in a book that sings praise to REST and Twisted.

Throughout the book, docstrings are rare, fully lowercase, and mostly appear to be left there by an absent-minded programmer. While I don't feel comments to be needed in a cleanly written program, the layout of comments almost made me see red:

# Part of example 6-2, `dbcred.py`, on page 96 (without the indentation bugs).
...
return (simplecred.INamedUserAvatar,
		simplecred.NamedUserAvatar(username, fullname),
		lambda: None) # null logout function

Oh, and example 8-4, imapserver.py, claims on page 144 that def _addedMessage(self, _, flags): ... is a Twisted idiom. I feel its scope is wider than that -- Deferreds just tend to make it quite common, with the mandatory first argument to callbacks.

The Twisted in the Book

This is a book on Twisted, so one expects the examples to be written with a deep understanding of Twisted fundamentals and best practices. And with this book, it really seems to be so. Most of the examples are delightful to read and show good grasp of the concepts behind the Twisted framework. This was a big relief for me, personally, as I was a bit disappointed in the Twisted-using examples in Foundations of Python Network Programming. See my review of it for more on that subject.

I need to get this said out loud:

<blink>
DO NOT BUY Twisted Network Programming Essentials TO LEARN TO WRITE WEB APPS
</blink>

The whole chapter 4, Web Servers, is just about horrible, and most things in it should not see the light of day again. Don't even think about subclassing Request to manage your web app presentation. Please ignore any mention of NOT_DONE_YET. Skip chapter 4 and read Nevow or twisted.web2 examples, instead. They may not be as nicely written, but at least they don't do active damage to your understanding of the topic.

The book mostly shows small and simple examples, and only explains things that are absolutely required to read the examples. The exception to this seems to be Cred, which is covered quite well. Deferreds are given a couple of pages, but I failed to find any motivation for their existence. Most of the examples seem to be totally sequential in nature, taking no real advantage of Twisted, apart from the protocol libraries. Even the server applications are never shown doing anything in parallel. A small example that runs multiple operations in parallel would have served nicely here. For example, see ldap_parallelsearch.py from my Ldaptor project, which opens 5 parallel TCP connections each doing 10 asynchronous requests.

The book never really bothers to explain interfaces. In the only real mention of them, outside of being sentinel values Checkers can use to ensure they are used with the right kind of credentials, is on page 193, and even that is confused: "You can iterate through the current services in your application by wrapping the application object in the service.IServiceCollection interface".

Things like unit testing Twisted Protocols, reactor.callLater and reactor.spawnProcess are not even mentioned in the book.

The explanation of pb on page 83 seems to be a bit confused. The book claims pb is not "language- and platform-neutral", does not need to "interoperate with anything else", and "uses serialized Python objects". While clearly aimed to be pythonic, and to work well with Python objects, none of these seems to be true. pb is cross-platform, cross-language, and completely capable of interoperating with just about anything where the protocol gets implemented. The serialization works for many kinds of things, not just Python objects. At least one Java implementation is known to exist.

The book also claims that pb is "conservative about what it exposes to the network", which as such is true, but the examples only manage to show the exact same self.remote_* -style access control as all the other RPC mechanism have shown. Naturally the book refers to the fact that with pb you can access complex Python objects over the network, and the code needs to pre-register the classes with pb to control what it knows is safe to receive; XML-RPC and SOAP don't have such need, because they are limited to transporting only most primitive types. But the examples do not show this at all.

Many examples, especially the ones dealing with twisted.cred, overly complicate things by using Deferreds for synchronous actions.

# Part of example 6-1, `simplecred.py`, on page 89.
def requestAvatarId(self, credentials):
	username = credentials.username
	if self.passwords.has_key(username):
		if credentials.password == self.passwords[username]:
			return defer.succeed(username)
		else:
			return defer.fail(
				credError.UnauthorizedLogin("Bad password"))
	else:
		return defer.fail(
			credError.UnauthorizedLogin("No such user"))

That's just overly complex. Portal uses maybeDeferred exactly to allow individual checkers to use Deferreds where they make sense, but not require them to (and to catch even accidentally triggered exceptions).

# My version of `simplecred.py`.
def requestAvatarId(self, credentials):
	passwd = self.passwords.get(credentials.username)
	if passwd is None:
		raise credError.UnauthorizedLogin("No such user")
	if passwd == credentials.password:
		return credentials.username
	else:
		raise credError.UnauthorizedLogin("Bad password")

Speaking of simplecred.py, the book defines a class which is practically identical to InMemoryUsernamePasswordDatabaseDontUse, apparently to be able to show a simplistic Checker in action. However, I would have hoped it to at least point to InMemoryUsernamePasswordDatabaseDontUse, with a short discussion on really avoiding it in real application. Or, even better, implementing a simple file-backed Checker. The difference in the complexity of source would not have been that big, I think. This continues in example 8-3, pop3server.py, on page 135, which actually uses a file storing usernames and passwords -- except it, too, reads the file fully into memory, and reimplements InMemoryUsernamePasswordDatabaseDontUse once more!

There are also style differences. I dislike the way Deferreds are handled, for example on page 72:

# Part of example 5-2, `rest_client.py`, on page 72.
def test(self):
	return self.createPage("RestTestTmp").addCallback(
		lambda _: self.deletePage("RestTestTmp")).addCallback(
		lambda _: self.createPage("RestTest")).addCallback(
		lambda _: self.getPage("RestTest")).addErrback(
		self.handleFailure)

Gah. There's just no way I am going to bother figuring out what that actually does, and whether the parens are actually balanced. And by the time I read the last line, I have already forgotten the return. Compare with this:

# My version of `rest_client.py`.
def test(self):
	d = self.createPage("RestTestTmp")
	d.addCallback(lambda _: self.deletePage("RestTestTmp"))
	d.addCallback(lambda _: self.createPage("RestTest"))
	d.addCallback(lambda _: self.getPage("RestTest"))
	d.addErrback(self.handleFailure)
	return d

Page 25 does note that this is a matter of personal opinion, but I stand firmly in the belief that form shown in the book is bad for readability of source code.

I also feel strongly that starting and stopping the reactor is a task for twistd or a top-level main function. Almost every example in the book does something like this:

# Part of example 3-4, `validate.py`, on page 30.
def showPage(pageData):
	...
	webbrowser.open('file://' + tmp)
	reactor.stop()

def handleError(failure):
	print "Error:", failure.getErrorMessage()
	reactor.stop()

if __name__ == "__main__":
	...
	postRequest.addCallback(showPage).addErrback(handleError)
	reactor.run()
# My version of `validate.py`.
def showPage(pageData):
	...
	webbrowser.open('file://' + tmp)

def showError(failure):
	print "Error:", failure.getErrorMessage()

if __name__ == "__main__":
	...
	postRequest.addCallback(showPage)

	# the book only talks about logging in last example, so
	# let this be.. real code would say d.addErrback(log.err)
	postRequest.addErrback(showError)
	postRequest.addBoth(lambda _: reactor.stop())
	reactor.run()

Sometimes this leads to even more confused results:

# Part of example 7-2, `pop3download.py`, on page 109.
def handleError(error):
	print error
	print >> sys.stderr, "Error:", error.getErrorMessage()
	reactor.stop()

if __name__ == "__main__":
	...
	f.deferred.addCallback(
		lambda _: reactor.stop()).addErrback(
		handleError)
	...

That is, if reactor.stop() raises an exception, print it out and try again?

Worse than style, I feel this book, as so many others, fails to introduce readers to the proper mindset of writing robust networked applications.

Some of the examples in the book are prone to data loss on race conditions and abnormal terminations, especially when dealing with files. There are no attempts to use temporary files or such. 1

# Part of example 5-1, `wiki.py`, on page 65.
def save(self):
	pickle.dump(self.pages, file(self.filename, 'w+b'))

Now, you may say that these are merely examples, and adding all that checking would needlessly overcomplicate them. Humbug, I say. Here's all that is needed to make that safe (in this case):

# My version of `wiki.py`.
def save(self):
	tmp = '%s.tmp' % self.filename
	f = file(tmp, 'wb')
	pickle.dump(self.pages, f)
	os.fsync(f)
	f.close()
	os.rename(tmp, self.filename)

Okay, so it changed one line to six. So make a class AtomicFile from it, and reuse that through the book.

# My version of `wiki.py`, refactored to use `AtomicFile`.
def save(self):
	f = AtomicFile(self.filename)
	pickle.dump(self.pages, f)
	f.close()

Besides, there's a 9-page IMAP server example, any readability worries should be concentrated right there. After 9 pages of source code, we get 4 pages of explanation, and then the topic switches to NNTP. The example is way too long to understand as a whole, and the book provides absolutely nothing to help you chunk it up to digestible pieces. The whole experience felt very tedious, and still we did not get to see an IMAP server one would actually want to run.

Similar non-atomic file operations are found also in example 8-4, imapserver.py, on page 142, which not only writes a pickle file non-atomically, but will also lose badly if a user opens more than one simultaneous connection -- and almost all IMAP clients do so.

Even worse is the remote manipulation of entries in a list based on indexes in example 6-4, pbcred.py, on page 103.

Unrelated to Twisted, but just horribly bad form, is how on pages 105, 118 and elsewhere through the book, you are expected to type your password on the command line. Please stop that, some people don't actually know better and will emulate the behaviour in their production applications.

On page 26, example 3-2, webcat2.py, downloads a web page to a temporary file, only to read it back line by line and print to standard output. Why not just stream the results directly to standard output? And why bother being line-based? And while I'm at it, why open a file with mode r+b, if you are only going to read the file?

Similarly but even worse, example 8-1, smtpserver.py, on page 124, delivers messages to maildirs with a custom class MaildirMessageWriter that buffers the message in memory, when all the maildir mechanisms in Twisted fully handle streaming the message directly to disk, and the whole class seems totally unnecessary.

Also in smtpserver.py, LocalDelivery.receivedHeader confuses the identity of the SMTP client host, as seen in the HELO command, with its own identity, and thus adds an invalid Received line to the message. This confusion is continued on pages 127 and 128, where the book says the first argument to receivedHeader is "... a tuple containing two strings: the server name by which the client addressed the server when it said HELO, ...". The HELO line identifies the client, not the server. Page 127 also correctly explains the format of the Received to be "FROM domain BY domain", but the example uses a different order. Also, the "with ESMTP" part is a lie, as the message could be sent with plain SMTP.

Example 8-2, smtpgoogle.py, on pages 129-131, is an email autoresponder with absolutely no loop prevention or rate limiting. Please oh please never run it for real. Having recently implemented a well-behaving autoresponder as part of the Scalemail project, I hope I can assure you that smtpgoogle.py is still a long way from one.

Example 7-5, imapfolders.py, on page 117 uses Deferred.called, which I feel is an abstraction violation, fiddling with the internals of Deferred, and down right ugly. Similarly example 7-6, imapdownload.py on page 120. The use case is detecting whether a connectionLost event was caused by an error or a clean logout (though I don't see imapfolders.py ever calling self.logout()). I feel that should be done with a state flag in the class object, not by fiddling with Deferred internals. Also, as example 7-2, pop3download.py contained no such logic, I feel compelled to ask why it was added here, and not earlier? The examples also don't bother to call IMAP4Client.connectionLost.

Page 63 says embedding small HTTP servers in random application is handy. I wish there would have been some thought given to things like what IP address and port to listen on, even how to integrate to be a part of a "bigger" HTTP server. Maybe that will have to wait until a book on Twisted web development comes out, some day. I would have appreciated a small reminder on security of administrative HTTP servers, too.

Then there's this unimaginably weird thing:

# Example 11-1, `reverse.py`, on page 188.
class ReverserService(internet.TCPServer):
	def __init__(self):
		internet.TCPServer.__init__(self, 2323, ReverserFactory())
# Example 11-2, `reverse_app.py`, on page 188.
...
reverserService = reverse.ReverserService()
...

why on earth would you do that, instead of just reverserService = TCPServer(2323, reverse.ReverserFactory())?

If you use Maildir++-style folder structure, beware that the maildir examples in the book use the Dovecot-style layout. For example, example 8-4, imapserver.py, on page 141 writes IMAP metadata to file named .imap-metadata.pickle, which in Maildir++-style layout would be a maildir.

Comparing the Books

Twisted Network Programming is about half the width of Foundations of Python Network Programming, with 213 pages compared to 512 pages.

Compared to Foundations of Python Network Programming, Twisted Network Programming Essentials never gives you any understanding of the wire protocols, and only explains how to use various Twisted implementations. For example, the the former has a table that lists data types known to XML-RPC, whereas the latter just tells you the name of the O'Reilly book to buy for more information. Neither of the books actually explains much about the underlying protocols.

As both Foundations of Python Network Programming and Twisted Network Programming Essentials talk about writing IMAP clients with twisted, lets compare them, deathmatch style.

Example tlist.py vs. example 7-5, imapfolders.py:

  • FoPNP scores one point by separating business logic from its IMAP4Client subclass.

  • TNPE scores one point from not calling reactor.stop from inside a ClientFactory. But gets a minus for calling it from the function that prints the folders to standard output.

  • TNPE scores one point from at least trying to handle connectionLost sanely.

  • TNPE scores one point for separating display of results from actual processing (FoPNP comes close, but still, the printing is inside a business logic object).

FoPNP 1 point, TNPE 3-.

Example tdownload.py vs. example 7-6, imapdownload.py:

  • things already mentioned above will not gain points again

  • one point to FoPNP for managing the overall structure of its Deferred chain in one place, thus making the program more readable.

  • one point to FoPNP for parallelizing the message downloads, just in case that might help, and to demonstrate the power of asynchronous operations

FoPNP 2 points, TNPE 0.

Final score: FoPNP 3 points, TNPE 3-.

Unfinished Things in the Book

Some parts of the book feel like they slipped through in the final rush to get it to print.

Example 7-2, pop3download.py, on page 109, seems like it was supposed to contain more, but was edited at the last moment, and is now both broken and in conflict with the surrounding text. The sample run on page 110 shows lines like "Downloading message x of y", when the actual source contains no such message. The DeferredList returned from _gotMessageSizes is never checked for errors. The error handling is referred to on page 119, which makes this seem all the more an error in the final edits. The source contains misspelled identifier retreivers. Errors catched by handleError are printed twice, once to standard output, once to standard error.

Example 7-2, pop3download.py, on page 108, contains a use of chainDeferred, with pretty much no explanation. Example 7-5, imapfolders.py, on page 117, again uses chainDeferred, and now the text on page 119 spends multiple paragraphs explaining the function, including the false claim "that [chainDeferred] hadn't been used in any of the previous examples".

Example 8-3, pop3server.py, on page 135, contains an interesting comment "null logout function (FIXME: explain why)".

Pages 117 and 139 both explain where the IMAP protocol is defined, in slightly different ways. They also can't agree on whether it is "more than 100 pages" or "more than 80 dense pages" long.

Conclusion

Twisted Network Programming Essentials is a good book. If you want to learn Twisted, and feel you need a book to do so, go ahead and buy it, you won't regret it.

However, a "Twisted Bible" it isn't, and will not become. It is a skinny book with some introductory examples of the most trivial bits of: twisted basics, web clients, different RPC methods, mail clients and servers, NTTP and SSH, with a good dose of twisted.cred added for spice. It is written in a very introductory style, except for the twisted.cred chapter, which actually gives some food for thought.

I honestly do not expect to refer back to it much. I do expect to introduce many people to Twisted with it, though. Don't expect it to teach you about networks or protocols, though; there's no space for that in a book this slim.

Overall, the book is of good quality, especially in the prose and layout. Some of the source code was hard to read, and I feel the presentation of the examples could have been a lot better, but all in all I believe the examples will do their job just fine.

Parts of the book seemed unfinished and sometimes actually conflicted with themselves, giving the image of a tired author trying to hurry things just before a deadline. The book also ended quite abruptly.

Perhaps the biggest thought the book left in my mind is how big Twisted and related projects really have become. After all this time with this book, I still want to see books that link Twisted to unit testing and test driven development, walk through twisted.web2 and a version of Nevow ported to it (including Athena, NuFox and Mantissa), talk about the VFS layer, Shtoom, Divmod, Vertex, Ldaptor after an API rework, and so much more.

Suggestions for Errata

As is my habit, I kept notes while reading the book, including typos etc. Here's a list of those, mostly to help the publisher should they choose to print a second edition of the book:

  • Throughout the book: The underscores in __main__ look different from other underscores.

  • Page 23, at least on line 2: typo LineReciever should be LineReceiver.

  • Page 66, example 5-1, wiki.py: literal string in the middle of a function, should probably be a comment or docstring.

  • Page 84, pb_wiki.py: method remote_getReporter should return WikiReporter, not RemoteWikiReporter. Spotted by Ed Suominen.

  • Page 89, example 6-1, simplecred.py: INamedUserAvatar would be more readable with explicit pass.

  • Page 96, example 6-2, dbcred.py: _gotQueryResults indentation is broken.

  • Page 99, example 6-3, multiavatar.py: INewUserAvatar.setName should not have argument self (see the z.i README.txt for more). Also consider using Attribute.

  • Page 99, example 6-3, multiavatar.py: requestAvatar indentation is broken.

  • Page 108, "and then downloads each message to an mbox file": for some reason, I read that as one message per mbox.

  • Page 112, "The current specification for SMTP is defined in RFC 2821": err, surely the RFC is the specification, while the protocol itself may be defined in it.

  • Page 117, example 7-5, imapfolders.py: suddenly switches to prefixing methods with two underscores, where everything seen before was single underscore. No explanation given.

  • Page 119: the explanation claims that chainDeferred is equal to separate addCallback and addErrback calls, while the actual implementation uses addCallbacks. In this case, this mostly affects backtraces, but confusing these methods has caused bugs and should be avoided.

  • Page 123, "..., decide what to do with the incoming message based on the email address, ...": as this means destination, it should be a plural, "based on the email addresses". SMTP is one sender to multiple recipients.

  • Page 123, "Example 8-1 accepts email for all addresses in a given domain ...": the actual example uses a list of domains, so this should be plural.

  • Pages 124-125, example 8-1, smtpserver.py: _getAddressDir is called with str(...) and does "%s" % .... Surely it will be string soon -- maybe once more for luck?

  • Page 126: the Thunderbird command line argument --ProfileManager is printed as not double-minus but em dash or such.

  • Page 128, "... a tuple with the hostname used in by the client ..."

  • Page 133, third paragraph: the real reason to really avoid delaying mail acceptance is because any connection loss, for any reason, that occurs between a client sending the end of message and server responding to it usually causes duplication of messages.

  • Page 134, example 8-3, pop3server.py: suddenly adds support for a self.debug flag, with no explanation why it was added here but never before.

  • Page 136, figure 8-4: the image is down right pathetic in how small it is compared to the box it sits in.

  • Page 139, "... tools to store, organize, and mail on a central server.": missing word after and?

  • Page 142, example 8-4, imapserver.py: _assignUIDs linewrap causes broken indentation, getFlags indentation is broken.

  • Page 178, "the server is given a copy of a user's private key": surely, but surely, you mean public key.

  • Page 179, example 10-2, pubkeyssh.py: requestAvatarId does raise failure.failure(error.ConchError(...)). I'm guessing you meant return failure.Failure(error.ConchError(...)), but actually raise error.ConchError(...) would be even better. See earlier comments about example 6-1.

  • Page 184, example 10-4, sshclient.py:


  1. Don't get me started on pickle.