Review of Twisted Network Programming Essentials
Abstract
A review of the book Twisted Network Programming Essentials by Abe Fettig (see also 2nd ed ).
Updated on 2005-12-07: added one contributed errata entry .
Disclosure
I have commit access to the Twisted source tree. I've been working with the project for quite some time. I would probably say good things about any book that mentioned Twisted.
Please also see my review of Foundations of Python Network Programming , as I will be comparing these two books.
And, in case you didn't read the review above and don't know me, let me restate this: I am not going to write a glowing fluffy review, give a link to amazon and increase sales -- even if I was bribed with a shiny book, which I do appreciate -- but to really dig into this book, see how good it is. Blame it on me being an average pessimistic Finn, or something. Take it this way: everything I don't complain about is just perfect.
First Impressions
The book looks just like all the other O'Reilly books. Of course, that is a good thing, as the O'Reilly standard for quality is quite high. Leafing through the book, I made two initial remarks. First, the table of contents is has a very light feel to it, and is actually very readable. And second, there is a lot of long source code listings in the book, which is sort of a pet peeve of mine. I hate page breaks in source code, especially when the pages are duplex and bound together, like they are in a book.
An Overview
A bit less than a dozen pages at the beginning are dedicated to
installing Twisted and getting started. The next dozen pages walk us
through some very basic examples showing how to write TCP clients and
servers, including an echoserver.py
example, which is practically
identical to the one you can find in the tutorials of the Twisted
project itself.
Writing web clients, both with getPage()
and by subclassing
HTTPClientFactory
, is covered on 13 pages. Server-side web things
are given 25 pages, but note that this does not contain
twisted.web2
, Nevow
, or just about anything beyond basic
twisted.web.resource.Resource
. There is about a page worth of
explanation on page 54 on how the web server things covered in the
book are old and will be deprecated, and says that anything newer is
going through too many changes to document in a book. That is, do not
buy this book to help you do web things.
REST, XML-RPC
, SOAP
and pb
are all crammed in about twenty
pages, basically implementing the same simple dict
manipulation
calls with each mechanism.
Twisted's authentication mechanism, twisted.cred
, gets 20 pages of
its own, and the topic is discussed quite thoroughly.
Writing simple clients for POP3
, SMTP
, and IMAP
are covered, with a
few pages each. Server-side implementations for the same protocols are
demonstrated in 5-14 pages each.
NNTP
, including a client and server implementation and some
background, is given ten pages.
Using Twisted's implementation of SSH
, known as Conch
, gets
explained in 10 pages for server side and a few pages for client side.
The final 15 pages of the book briefly explains twistd
, the
concept of dropping root privileges, and talks about logging.
There are practically no tables or figures in the book, right now I
can't find anything except the twisted.cred
overview on page 92,
which is basically identical with the one in the actual Cred HOWTO
.
The Python in the Book
The book is written in clear and concise English, with surprisingly few typos. That is very good and makes reading the explanations quite easy. But, when it comes the technical books, I'm more concerned about the technical things. When the topic is programming, the examples are what fail or save the book.
All in all, I'm quite happy with the examples in this book. It is mostly readable and easy to follow. There are, however, a few things that left me wishing for more.
Representing source code listings in a book form is always
challenging. What ruins parts of this book is the tendency to show big
chunks of code, spanning multiple pages. I constantly lost my place
when a class or a function was broken between pages, and found myself
having to turn back two pages before being reminded of the name of the
class I was looking at. For contrast, Foundations of Python Network Programming
managed to introduce its concepts iteratively, with most
of the source code listings being less than one page.
Some of the things I encountered in the listings I considered a bit unpythonic, or they seemed old-fashioned to me. Here are some examples, with my preferred forms below them. These repeated through the book.
# Part of example 5-1, `wiki.py`, on page 65.
def hasPage(self, path):
return self.pages.has_key(path)
def hasPage(self, path):
return path in self.pages
or even just
def __contains__(self, path):
return path in self.pages
# Part of example 8-1, `smtpserver.py`, on page 124.
if not user.dest.domain in self.validDomains:
...
if user.dest.domain not in self.validDomains:
...
# Part of example 5-1, `wiki.py`, on page 66.
def deletePage(self, path):
del(self.pages[path])
def deletePage(self, path):
del self.pages[path]
# Part of example 7-3, `smtpsend.py`, on page 113.
attachment.add_header('Content-Disposition', 'attachment',
filename=os.path.split(filename)[1])
attachment.add_header('Content-Disposition', 'attachment',
filename=os.path.basename(filename))
# Part of example 7-5, `imapfolders.py`, on page 117.
def __gotMailboxList(self, list):
return [boxInfo[2] for boxInfo in list]
def __gotMailboxList(self, mailboxes):
return [name for flags, delimiter, name in mailboxes]
# Part of example 8-4, `imapserver.py`, on page 141.
def initMetadata(self):
if not self.metadata.has_key('flags'):
self.metadata['flags'] = {} # dict of message IDs to flags
if not self.metadata.has_key('uidvalidity'):
self.metadata['uidvalidity'] = random.randint(1000000, 9999999)
if not self.metadata.has_key('uids'):
self.metadata['uids'] = {}
def initMetadata(self):
self.metadata.setdefault('flags', {})
self.metadata.setdefault('uidvalidity', random.randint(1000000, 9999999))
self.metadata.setdefault('uids', {))
# Part of example 10-1, `sshserver.py`, on page 174.
publicMethods = filter(
lambda funcname: funcname.startswith('do_'), dir(self))
commands = [cmd.replace('do_', '', 1) for cmd in publicMethods]
PREFIX = 'do_'
commands = [cmd[len(PREFIX):] for cmd in dir(self) if cmd.startswith(PREFIX)]
Sometimes the data structures used are not really suitable for the task at hand.
# Part of example 8-4, `imapserver.py`, on page 143.
... gather message UIDs in list allUIDs ...
for uid in messageSet:
...
if uid in allUIDs:
sequence = allUIDs.index(uid)+1
...
which walks through the list a lot, once for if and once for
index
.
In at least one case, poor data structure choice actually produces visible defects:
# Part of example 5-7, `pb_wiki.py`, on page 84.
...
neededPages = []
for page in allPages:
...
for pageName in wikiWords:
if not self.wikiData.hasPage(pageName):
neededPages.append(pageName)
If multiple WikiNames point to a missing page, neededPages
will
have duplicates. The sample output on page 86 actually shows such a
case. Just making neededPages
a Set
would fix this, make the
example a bit faster, and most of all would be the right data type for
the task. Missing pages have no inherent ordering, but the question
"is this page missing?" is asked a lot.
Throughout the book, I kept wondering about the file modes. Why open a file for writing if you only read from it? But my point with this example was the "don't look before you jump, just fail gracefully" idiom. Just let the file open fail, handle the error, and if it was caused by file not existing, use the default content.
# Part of example 5-1, `wiki.py`, on page 65.
if os.path.exists(filename):
self.pages = pickle.load(file(filename, 'r+b'))
else:
self.pages = {'WikiHome': 'This is your Wiki home page.'}
# My version of `wiki.py`.
self.pages = {'WikiHome': 'This is your Wiki home page.'}
try:
f = file(filename, 'rb')
except IOError, e:
if e.errno == errno.ENOENT:
pass
else:
raise
else:
self.pages = pickle.load(f)
f.close()
There are other similar things, like closing and reopening a file for no gain. Oh, and relying on Python GC to close the file early so the content written is actually flushed to the filesystem!
# Part of example 3-4, `validate.py`, on page 30.
def showPage(pageData):
# write data to temp .html file, show file in browser
tmpfd, tmp = tempfile.mkstemp('.html')
os.close(tmpfd)
file(tmp, 'w+b').write(pageData)
webbrowser.open('file://' + tmp)
...
# My version of `validate.py`.
def showPage(pageData):
"""Write data to temp .html file, show file in browser."""
tmpfd, tmp = tempfile.mkstemp('.html')
f = os.fdopen(tmpfd, 'w')
f.write(pageData)
f.close()
webbrowser.open('file://' + tmp)
...
There's also a bit of clumsy regexp use in the wiki example.
# Part of example 5-1, `wiki.py`, from pages 65-66.
reWikiWord = re.compile(r'\b(([A-Z][a-z]+){2,})\b')
...
wikiWords = [match[0] for match in reWikiWord.findall(pageData)]
for word in wikiWords:
pageData = pageData.replace(
word, "<a href='%s'>%s</a>" % (word, word))
return pageData
Why define two matching groups if you don't want more than one?
# My version of `wiki.py`.
reWikiWord = re.compile(r'\b((?:[A-Z][a-z]+){2,})\b')
...
for word in reWikiWord.findall(pageData):
pageData = pageData.replace(
word, "<a href='%s'>%s</a>" % (word, word))
return pageData
Why do a clumsy manual loop if the substitution isn't any smarter than that?
# My version of `wiki.py`, refactored.
reWikiWord = re.compile(r'\b(([A-Z][a-z]+){2,})\b')
...
s, count = reWikiWord.subn(r'<a href="\1">\1</a>', pageData)
return s
Of course, the proper thing to do would be to actually output valid
XML, regardless of input data. This is a trend throughout the book,
everything involving HTML is just tag soup. Hopefully a later book
on Nevow
will clean things up.
Even outside HTML, the output formats of the examples are a mess.
Most of the examples embedding user data in the output do nothing to
escape potentially malicious strings. Adding quotes is generally done
with "foo '%s'" % evil
, when the even simpler "foo %r" % evil
would work a lot better.
In example 5-1, wiki.py
, class WikiData
consists almost purely
of application logic, but then gets a getRenderedPage
method that
renders it to HTML. Cleaner separation of responsibilities would have
been, well, cleaner. The same example also manages to disappoint me by
putting operations like "edit" and "save" in the same namespace as
wiki pages -- that's a mistake I expect to see in a poorly-written PHP
application, not in a book that sings praise to REST and Twisted.
Throughout the book, docstrings are rare, fully lowercase, and mostly appear to be left there by an absent-minded programmer. While I don't feel comments to be needed in a cleanly written program, the layout of comments almost made me see red:
# Part of example 6-2, `dbcred.py`, on page 96 (without the indentation bugs).
...
return (simplecred.INamedUserAvatar,
simplecred.NamedUserAvatar(username, fullname),
lambda: None) # null logout function
Oh, and example 8-4, imapserver.py
, claims on page 144 that def _addedMessage(self, _, flags): ...
is a Twisted idiom. I feel its
scope is wider than that -- Deferred
s just tend to make it quite
common, with the mandatory first argument to callbacks.
The Twisted in the Book
This is a book on Twisted, so one expects the examples to be written with a deep understanding of Twisted fundamentals and best practices. And with this book, it really seems to be so. Most of the examples are delightful to read and show good grasp of the concepts behind the Twisted framework. This was a big relief for me, personally, as I was a bit disappointed in the Twisted-using examples in Foundations of Python Network Programming. See my review of it for more on that subject.
I need to get this said out loud:
<blink>
DO NOT BUYTwisted Network Programming Essentials
TO LEARN TO WRITE WEB APPS
</blink>
The whole chapter 4, Web Servers
, is just about horrible, and most
things in it should not see the light of day again. Don't even think
about subclassing Request
to manage your web app presentation.
Please ignore any mention of NOT_DONE_YET
. Skip chapter 4 and read
Nevow
or twisted.web2
examples, instead. They may not be as
nicely written, but at least they don't do active damage to your
understanding of the topic.
The book mostly shows small and simple examples, and only explains
things that are absolutely required to read the examples. The
exception to this seems to be Cred, which is covered quite well.
Deferred
s are given a couple of pages, but I failed to find any
motivation for their existence. Most of the examples seem to be
totally sequential in nature, taking no real advantage of Twisted,
apart from the protocol libraries. Even the server applications are
never shown doing anything in parallel. A small example that runs
multiple operations in parallel would have served nicely here. For
example, see ldap_parallelsearch.py
from my Ldaptor
project,
which opens 5 parallel TCP connections each doing 10 asynchronous
requests.
The book never really bothers to explain interfaces. In the only real
mention of them, outside of being sentinel values Checker
s can
use to ensure they are used with the right kind of credentials, is on
page 193, and even that is confused: "You can iterate through the
current services in your application by wrapping the application
object in the service.IServiceCollection
interface".
Things like unit testing Twisted Protocol
s, reactor.callLater
and reactor.spawnProcess
are not even mentioned in the book.
The explanation of pb
on page 83 seems to be a bit confused. The
book claims pb
is not "language- and platform-neutral", does
not need to "interoperate with anything else", and "uses serialized
Python objects". While clearly aimed to be pythonic, and to work well
with Python objects, none of these seems to be true. pb
is
cross-platform, cross-language, and completely capable of
interoperating with just about anything where the protocol gets
implemented. The serialization works for many kinds of things, not
just Python objects. At least one Java implementation
is known to
exist.
The book also claims that pb
is "conservative about what it
exposes to the network", which as such is true, but the examples only
manage to show the exact same self.remote_*
-style access control
as all the other RPC mechanism have shown. Naturally the book refers
to the fact that with pb
you can access complex Python objects
over the network, and the code needs to pre-register the classes with
pb
to control what it knows is safe to receive; XML-RPC and SOAP
don't have such need, because they are limited to transporting only
most primitive types. But the examples do not show this at all.
Many examples, especially the ones dealing
with twisted.cred
, overly complicate things by using Deferred
s for
synchronous actions.
# Part of example 6-1, `simplecred.py`, on page 89.
def requestAvatarId(self, credentials):
username = credentials.username
if self.passwords.has_key(username):
if credentials.password == self.passwords[username]:
return defer.succeed(username)
else:
return defer.fail(
credError.UnauthorizedLogin("Bad password"))
else:
return defer.fail(
credError.UnauthorizedLogin("No such user"))
That's just overly complex. Portal
uses maybeDeferred
exactly
to allow individual checkers to use Deferred
s where they make
sense, but not require them to (and to catch even accidentally
triggered exceptions).
# My version of `simplecred.py`.
def requestAvatarId(self, credentials):
passwd = self.passwords.get(credentials.username)
if passwd is None:
raise credError.UnauthorizedLogin("No such user")
if passwd == credentials.password:
return credentials.username
else:
raise credError.UnauthorizedLogin("Bad password")
Speaking of simplecred.py
, the book defines a class which is
practically identical to InMemoryUsernamePasswordDatabaseDontUse
,
apparently to be able to show a simplistic Checker
in action.
However, I would have hoped it to at least point to
InMemoryUsernamePasswordDatabaseDontUse
, with a short discussion
on really avoiding it in real application. Or, even better,
implementing a simple file-backed Checker
. The difference in the
complexity of source would not have been that big, I think. This
continues in example 8-3, pop3server.py
, on page 135, which
actually uses a file storing usernames and passwords -- except it,
too, reads the file fully into memory, and reimplements
InMemoryUsernamePasswordDatabaseDontUse
once more!
There are also style differences. I dislike the way Deferred
s are
handled, for example on page 72:
# Part of example 5-2, `rest_client.py`, on page 72.
def test(self):
return self.createPage("RestTestTmp").addCallback(
lambda _: self.deletePage("RestTestTmp")).addCallback(
lambda _: self.createPage("RestTest")).addCallback(
lambda _: self.getPage("RestTest")).addErrback(
self.handleFailure)
Gah. There's just no way I am going to bother figuring out what that
actually does, and whether the parens are actually balanced. And by
the time I read the last line, I have already forgotten the
return
. Compare with this:
# My version of `rest_client.py`.
def test(self):
d = self.createPage("RestTestTmp")
d.addCallback(lambda _: self.deletePage("RestTestTmp"))
d.addCallback(lambda _: self.createPage("RestTest"))
d.addCallback(lambda _: self.getPage("RestTest"))
d.addErrback(self.handleFailure)
return d
Page 25 does note that this is a matter of personal opinion, but I stand firmly in the belief that form shown in the book is bad for readability of source code.
I also feel strongly that starting and stopping the reactor is a task
for twistd
or a top-level main function. Almost every example
in the book does something like this:
# Part of example 3-4, `validate.py`, on page 30.
def showPage(pageData):
...
webbrowser.open('file://' + tmp)
reactor.stop()
def handleError(failure):
print "Error:", failure.getErrorMessage()
reactor.stop()
if __name__ == "__main__":
...
postRequest.addCallback(showPage).addErrback(handleError)
reactor.run()
# My version of `validate.py`.
def showPage(pageData):
...
webbrowser.open('file://' + tmp)
def showError(failure):
print "Error:", failure.getErrorMessage()
if __name__ == "__main__":
...
postRequest.addCallback(showPage)
# the book only talks about logging in last example, so
# let this be.. real code would say d.addErrback(log.err)
postRequest.addErrback(showError)
postRequest.addBoth(lambda _: reactor.stop())
reactor.run()
Sometimes this leads to even more confused results:
# Part of example 7-2, `pop3download.py`, on page 109.
def handleError(error):
print error
print >> sys.stderr, "Error:", error.getErrorMessage()
reactor.stop()
if __name__ == "__main__":
...
f.deferred.addCallback(
lambda _: reactor.stop()).addErrback(
handleError)
...
That is, if reactor.stop()
raises an exception, print it out and
try again?
Worse than style, I feel this book, as so many others, fails to introduce readers to the proper mindset of writing robust networked applications.
Some of the examples in the book are prone to data loss on race conditions and abnormal terminations, especially when dealing with files. There are no attempts to use temporary files or such. 1
# Part of example 5-1, `wiki.py`, on page 65.
def save(self):
pickle.dump(self.pages, file(self.filename, 'w+b'))
Now, you may say that these are merely examples, and adding all that checking would needlessly overcomplicate them. Humbug, I say. Here's all that is needed to make that safe (in this case):
# My version of `wiki.py`.
def save(self):
tmp = '%s.tmp' % self.filename
f = file(tmp, 'wb')
pickle.dump(self.pages, f)
os.fsync(f)
f.close()
os.rename(tmp, self.filename)
Okay, so it changed one line to six. So make a class AtomicFile
from it, and reuse that through the book.
# My version of `wiki.py`, refactored to use `AtomicFile`.
def save(self):
f = AtomicFile(self.filename)
pickle.dump(self.pages, f)
f.close()
Besides, there's a 9-page IMAP server example, any readability worries should be concentrated right there. After 9 pages of source code, we get 4 pages of explanation, and then the topic switches to NNTP. The example is way too long to understand as a whole, and the book provides absolutely nothing to help you chunk it up to digestible pieces. The whole experience felt very tedious, and still we did not get to see an IMAP server one would actually want to run.
Similar non-atomic file operations are found also in example 8-4,
imapserver.py
, on page 142, which not only writes a pickle file
non-atomically, but will also lose badly if a user opens more than one
simultaneous connection -- and almost all IMAP clients do so.
Even worse is the remote manipulation of entries in a list based on
indexes in example 6-4, pbcred.py
, on page 103.
Unrelated to Twisted, but just horribly bad form, is how on pages 105, 118 and elsewhere through the book, you are expected to type your password on the command line. Please stop that, some people don't actually know better and will emulate the behaviour in their production applications.
On page 26, example 3-2, webcat2.py
, downloads a web page to a
temporary file, only to read it back line by line and print to
standard output. Why not just stream the results directly to standard
output? And why bother being line-based? And while I'm at it, why open
a file with mode r+b
, if you are only going to read the file?
Similarly but even worse, example 8-1, smtpserver.py
, on page 124,
delivers messages to maildirs with a custom class
MaildirMessageWriter
that buffers the message in memory, when all
the maildir mechanisms in Twisted fully handle streaming the message
directly to disk, and the whole class seems totally unnecessary.
Also in smtpserver.py
, LocalDelivery.receivedHeader
confuses
the identity of the SMTP
client host, as seen in the HELO
command, with its own identity, and thus adds an invalid Received
line to the message. This confusion is continued on pages 127 and 128,
where the book says the first argument to receivedHeader
is "... a
tuple containing two strings: the server name by which the client
addressed the server when it said HELO
, ...". The HELO
line
identifies the client, not the server. Page 127 also correctly
explains the format of the Received
to be "FROM domain BY
domain", but the example uses a different order. Also, the "with
ESMTP" part is a lie, as the message could be sent with plain SMTP.
Example 8-2, smtpgoogle.py
, on pages 129-131, is an email
autoresponder with absolutely no loop prevention or rate limiting.
Please oh please never run it for real. Having recently implemented a
well-behaving autoresponder
as part of the Scalemail
project, I hope
I can assure you that smtpgoogle.py
is still a long way from one.
Example 7-5, imapfolders.py
, on page 117 uses Deferred.called
,
which I feel is an abstraction violation, fiddling with the internals
of Deferred
, and down right ugly. Similarly example 7-6,
imapdownload.py
on page 120. The use case is detecting whether a
connectionLost
event was caused by an error or a clean logout
(though I don't see imapfolders.py
ever calling
self.logout()
). I feel that should be done with a state flag in
the class object, not by fiddling with Deferred
internals. Also,
as example 7-2, pop3download.py
contained no such logic, I feel
compelled to ask why it was added here, and not earlier? The examples
also don't bother to call IMAP4Client.connectionLost
.
Page 63 says embedding small HTTP servers in random application is handy. I wish there would have been some thought given to things like what IP address and port to listen on, even how to integrate to be a part of a "bigger" HTTP server. Maybe that will have to wait until a book on Twisted web development comes out, some day. I would have appreciated a small reminder on security of administrative HTTP servers, too.
Then there's this unimaginably weird thing:
# Example 11-1, `reverse.py`, on page 188.
class ReverserService(internet.TCPServer):
def __init__(self):
internet.TCPServer.__init__(self, 2323, ReverserFactory())
# Example 11-2, `reverse_app.py`, on page 188.
...
reverserService = reverse.ReverserService()
...
why on earth would you do that, instead of just reverserService = TCPServer(2323, reverse.ReverserFactory())
?
If you use Maildir++
-style folder structure, beware that the
maildir examples in the book use the Dovecot
-style layout.
For example, example 8-4, imapserver.py
, on page 141 writes
IMAP metadata to file named .imap-metadata.pickle
, which
in Maildir++
-style layout would be a maildir.
Comparing the Books
Twisted Network Programming
is about half the width of Foundations of Python Network Programming
, with 213 pages compared to 512 pages.
Compared to Foundations of Python Network Programming
, Twisted Network Programming Essentials
never gives you any understanding
of the wire protocols, and only explains how to use various Twisted
implementations. For example, the the former has a table that lists
data types known to XML-RPC, whereas the latter just tells you the
name of the O'Reilly book to buy for more information. Neither of the
books actually explains much about the underlying protocols.
As both Foundations of Python Network Programming
and Twisted Network Programming Essentials
talk about writing IMAP clients with
twisted, lets compare them, deathmatch style.
Example tlist.py
vs. example 7-5, imapfolders.py
:
-
FoPNP
scores one point by separating business logic from itsIMAP4Client
subclass. -
TNPE
scores one point from not callingreactor.stop
from inside aClientFactory
. But gets a minus for calling it from the function that prints the folders to standard output. -
TNPE
scores one point from at least trying to handleconnectionLost
sanely. -
TNPE
scores one point for separating display of results from actual processing (FoPNP
comes close, but still, the printing is inside a business logic object).
FoPNP
1 point, TNPE
3-.
Example tdownload.py
vs. example 7-6, imapdownload.py
:
-
things already mentioned above will not gain points again
-
one point to
FoPNP
for managing the overall structure of itsDeferred
chain in one place, thus making the program more readable. -
one point to
FoPNP
for parallelizing the message downloads, just in case that might help, and to demonstrate the power of asynchronous operations
FoPNP
2 points, TNPE
0.
Final score: FoPNP
3 points, TNPE
3-.
Unfinished Things in the Book
Some parts of the book feel like they slipped through in the final rush to get it to print.
Example 7-2, pop3download.py
, on page 109, seems like it was
supposed to contain more, but was edited at the last moment, and is
now both broken and in conflict with the surrounding text. The sample
run on page 110 shows lines like "Downloading message x of y",
when the actual source contains no such message. The DeferredList
returned from _gotMessageSizes
is never checked for errors. The
error handling is referred to on page 119, which makes this seem all
the more an error in the final edits. The source contains misspelled
identifier retreivers
. Errors catched by handleError
are
printed twice, once to standard output, once to standard error.
Example 7-2, pop3download.py
, on page 108, contains a use of
chainDeferred
, with pretty much no explanation. Example 7-5,
imapfolders.py
, on page 117, again uses chainDeferred
, and now
the text on page 119 spends multiple paragraphs explaining the
function, including the false claim "that [chainDeferred
] hadn't
been used in any of the previous examples".
Example 8-3, pop3server.py
, on page 135, contains an interesting
comment "null logout function (FIXME: explain why)".
Pages 117 and 139 both explain where the IMAP protocol is defined, in slightly different ways. They also can't agree on whether it is "more than 100 pages" or "more than 80 dense pages" long.
Conclusion
Twisted Network Programming Essentials
is a good book. If you want
to learn Twisted, and feel you need a book to do so, go ahead and buy
it, you won't regret it.
However, a "Twisted Bible" it isn't, and will not become. It is a
skinny book with some introductory examples of the most trivial bits
of: twisted basics, web clients, different RPC methods, mail clients
and servers, NTTP and SSH, with a good dose of twisted.cred
added
for spice. It is written in a very introductory style, except for the
twisted.cred
chapter, which actually gives some food for thought.
I honestly do not expect to refer back to it much. I do expect to introduce many people to Twisted with it, though. Don't expect it to teach you about networks or protocols, though; there's no space for that in a book this slim.
Overall, the book is of good quality, especially in the prose and layout. Some of the source code was hard to read, and I feel the presentation of the examples could have been a lot better, but all in all I believe the examples will do their job just fine.
Parts of the book seemed unfinished and sometimes actually conflicted with themselves, giving the image of a tired author trying to hurry things just before a deadline. The book also ended quite abruptly.
Perhaps the biggest thought the book left in my mind is how big
Twisted and related projects really have become. After all this time
with this book, I still want to see books that link Twisted to unit
testing and test driven development, walk through twisted.web2
and
a version of Nevow ported to it (including Athena, NuFox and
Mantissa), talk about the VFS layer, Shtoom, Divmod, Vertex, Ldaptor
after an API rework, and so much more.
Suggestions for Errata
As is my habit, I kept notes while reading the book, including typos etc. Here's a list of those, mostly to help the publisher should they choose to print a second edition of the book:
-
Throughout the book: The underscores in
__main__
look different from other underscores. -
Page 23, at least on line 2: typo
LineReciever
should beLineReceiver
. -
Page 66, example 5-1,
wiki.py
: literal string in the middle of a function, should probably be a comment or docstring. -
Page 84,
pb_wiki.py
: methodremote_getReporter
should return WikiReporter, not RemoteWikiReporter. Spotted by Ed Suominen. -
Page 89, example 6-1,
simplecred.py
:INamedUserAvatar
would be more readable with explicitpass
. -
Page 96, example 6-2,
dbcred.py
:_gotQueryResults
indentation is broken. -
Page 99, example 6-3,
multiavatar.py
:INewUserAvatar.setName
should not have argumentself
(see the z.i README.txt for more). Also consider usingAttribute
. -
Page 99, example 6-3,
multiavatar.py
:requestAvatar
indentation is broken. -
Page 108, "and then downloads each message to an mbox file": for some reason, I read that as one message per mbox.
-
Page 112, "The current specification for SMTP is defined in RFC 2821": err, surely the RFC is the specification, while the protocol itself may be defined in it.
-
Page 117, example 7-5,
imapfolders.py
: suddenly switches to prefixing methods with two underscores, where everything seen before was single underscore. No explanation given. -
Page 119: the explanation claims that
chainDeferred
is equal to separateaddCallback
andaddErrback
calls, while the actual implementation usesaddCallbacks
. In this case, this mostly affects backtraces, but confusing these methods has caused bugs and should be avoided. -
Page 123, "..., decide what to do with the incoming message based on the email address, ...": as this means destination, it should be a plural, "based on the email addresses". SMTP is one sender to multiple recipients.
-
Page 123, "Example 8-1 accepts email for all addresses in a given domain ...": the actual example uses a list of domains, so this should be plural.
-
Pages 124-125, example 8-1,
smtpserver.py
:_getAddressDir
is called withstr(...)
and does"%s" % ...
. Surely it will be string soon -- maybe once more for luck? -
Page 126: the Thunderbird command line argument
--ProfileManager
is printed as not double-minus but em dash or such. -
Page 128, "... a tuple with the hostname used in by the client ..."
-
Page 133, third paragraph: the real reason to really avoid delaying mail acceptance is because any connection loss, for any reason, that occurs between a client sending the end of message and server responding to it usually causes duplication of messages.
-
Page 134, example 8-3,
pop3server.py
: suddenly adds support for aself.debug
flag, with no explanation why it was added here but never before. -
Page 136, figure 8-4: the image is down right pathetic in how small it is compared to the box it sits in.
-
Page 139, "... tools to store, organize, and mail on a central server.": missing word after and?
-
Page 142, example 8-4,
imapserver.py
:_assignUIDs
linewrap causes broken indentation,getFlags
indentation is broken. -
Page 178, "the server is given a copy of a user's private key": surely, but surely, you mean public key.
-
Page 179, example 10-2,
pubkeyssh.py
:requestAvatarId
doesraise failure.failure(error.ConchError(...))
. I'm guessing you meantreturn failure.Failure(error.ConchError(...))
, but actuallyraise error.ConchError(...)
would be even better. See earlier comments about example 6-1 . -
Page 184, example 10-4,
sshclient.py
:
-
Don't get me started on pickle. ↩︎