A song for the lovers › Twisted interactive console

A song for the lovers › Twisted interactive console

Twisted interactive console

As you all already know TwistedMatrix is great to write asynchronous event-driven network oriented programs: define how your protocol responds in case of events, attach some callbacks if you need them, wrap it in a factory and activate the reactor.

The reactor runs a giant loop in which events are processed in a non-blocking fashion. Sometimes, though, everything a man needs it’s just to make it stop. At least for a while, at least for the sake of getting data from the user.

The most prominent example of a command line client program that needs to stop and wait is a REPL. The following kind ofREPL is a bit unorthodox, you’ll see.

The secret sauce (at least the one I found) to write such kind of interactive program in Twisted istwisted.internet.stdio.StandardIO. It connects your protocol to standard input and output:

class Repl(basic.LineReceiver):     delimiter = '\n'     prompt_string = 'cmd> '       def prompt(self):         self.transport.write(self.prompt_string)       def connectionMade(self):         self.sendLine('Welcome to Console')         self.prompt()       def lineReceived(self, line):         # blank line         if not line:             self.prompt()             return           self.issueCommand(line)       def issueCommand(self, command):         # send the command to the server         d = sendCmd("%s%s" % (command, self.delimiter))         d.addCallback(self._checkResponse)       def _checkResponse(self, args):         success, num_lines, data = args         if num_lines > 20:             # use less to display the response             self.lessify(data)         else:             self.sendLine(data)         self.prompt()       def lessify(self, data):         p = subprocess.Popen(["less"], stdin=subprocess.PIPE)         p.communicate(data)       def connectionLost(self, reason):         reactor.stop()

When the protocol it is connected to the standard output with

stdio.StandardIO(TaskConsole()) reactor.run()

the program displays the prompt, hence when a line is received from the standard input it is sent to the other protocol attached on the network and a callback is registered for the response. I also decided to use less to display the response if it’s more than some lines but that’s a detail.

The sendCmd function instantiate the networked protocol and its factory:

def sendCmd(cmd):     factory = CFactory(cmd)     reactor.connectTCP('127.0.0.1', 1234, factory)     return factory.deferred

Then, when the server replies with some content we check to see if everything went ok and the reconstruct the whole response sending it back the REPL:

class Client(basic.LineReceiver):     delimiter = '\n'       def connectionMade(self):         # send the command received by the cmdline to the server         self.sendLine(self.factory.cmd)         self.buffer = []         self.cmd_success = True       def lineReceived(self, line):         # basic check error/success         if line.startswith('OK'):             return         if line.startswith('ERR'):             self.cmd_success = False             return           if line == 'END':             # join the response at the end of it             self.responseFinished(                 len(self.buffer), "\n".join(self.buffer))             self.buffer = []         else:             self.buffer.append(line)       def responseFinished(self, num_lines, data):         # disconnect         self.sendLine('quit')         self.transport.loseConnection()           # send back the response to the REPL         self.factory.deferred.callback((             self.cmd_success, num_lines, data))     class CFactory(protocol.ClientFactory):     protocol = Client       def __init__(self, cmd):         self.cmd = cmd         self.deferred = defer.Deferred()

How cool is that? Not really to be honest. It has a big gigantic fault: every time you input a line a connection to the server is opened and closed. That’s bad, really bad.

It didn’t take too long to create a version that use just one connection:

def connectionMade(self):     self.sendLine('Welcome to Console')     self.factory = CFactory()     self.connector = reactor.connectTCP(         '127.0.0.1', 1234, self.factory)     self.prompt()

We store the connector and the factory. issueCommand does not open a connection anymore, just:

def issueCommand(self, command):     self.connector.transport.write("%s%s" % (command, self.delimiter))     self.factory.deferred.addCallback(self._checkResponse)

We write directly to the transport of the connector (and not the one connected to the stdout) and register the callback on the factory’s deferred.

That’s better in my opinion and a nice start. I know that within twisted.conch.stdio there’s something more evolved. I’ll try to look into it when I have more time.

You can find the first version (bad) and the second version (better) online.

What do you think about this try?