Cocoa for Scientists (Part XX): Python Scripters...Meet Cocoa

Author: Drew McCormack
Web Site: www.mentalfaculty.com

The MacResearch web site wasn’t the only thing to get an overhaul in the last few days. Some of you may also have noticed a rather significant upgrade to your operating system. We will be covering the aspects of Leopard that we think are most relevant to scientists and scientific developers in the coming weeks and months. And to kick it all off, I want to begin with a tutorial introducing one of my favorite new features: BridgeSupport. I’ll do this by showing you how you can write a hierarchical data browser in around 250 lines of code, using Python and Cocoa. The app created here will be for browsing the jobs on an Xgrid controller, but could easily be adapted to browse any form of hierarchical relationships or data.

What is BridgeSupport?

BridgeSupport is a new technology in Leopard that provides a standard means of bridging scripting languages to the Objective-C and C APIs in Mac OS X. The way it works is this: you can use the tools provided by Apple to extract the important interface information for an Objective-C or C based framework into an XML file. This XML file can then be read by a scripting language like Python, so that it knows how to use the native code. It was possible to bridge scripting languages to Cocoa before Leopard, but it was always done in an ad hoc manner that was not very satisfactory. With Leopard, Apple has invited scripting languages like Python and Ruby to become first-class citizens by including them on every Mac; providing a bridging standard; and adding support to the Xcode tool suite.

In Control

The technology underlying BridgeSupport is interesting, but it is not really necessary to know how it works in order to use it. If you know Python (or Ruby), you can start developing a Cocoa app right away. The bridge is for the most part invisible to the developer — it’s almost as if Cocoa was written from the ground up in Python.

To demonstrate how it works, we are going to develop a simple Xgrid browser called ‘In Control’ (see screenshot). You can try it out on a Leopard system by downloading it here. The way it works is this: the user enters an address for the Xgrid controller, along with a password, and then clicks the Refresh button. In Control uses the command line tool xgrid to retrieve information from the controller about the grids that are defined, and the jobs it is maintaining, and displays that in an outline view.

Although In Control is written for use with Xgrid, the same basic design could be used to build other hierarchical data browsers. For example, it wouldn’t require too much effort to adapt In Control to browse a HDF file.

Creating a Cocoa-Python Application

You’ll need to install Leopard, and the Xcode 3.0 developer tools, if you want to follow along. The tools are included as a separate installer package on the Leopard install disc.

Begin by opening Xcode, and create a new project by choosing File > New Project… Locate the Application group in the New Project panel, and select Cocoa-Python Application. Click Next, and enter the name ‘In Control’ for the project. Select the directory where you want the project to be located, and click the Finish button.

It’s worth taking a look around the project that Xcode provides for a Cocoa-Python application. The main routine is still written in Objective-C; if you look in the Other Sources group in the Groups & Files source list, you will find the file main.m. It begins by initializing the python source search path (i.e. PYTHONPATH), and then starts up Python with this code:

Py_SetProgramName("/usr/bin/python");
Py_Initialize();
PySys_SetArgv(argc, (char **)argv);

const char *mainFilePathPtr = [mainFilePath UTF8String];
FILE *mainFile = fopen(mainFilePathPtr, "r");
int result = PyRun_SimpleFile(mainFile, (char *)[[mainFilePath lastPathComponent] UTF8String]);

The C function PyRun_SimpleFile is used to run the Python script main.py. The main.py file is very simple: It imports the application delegate class, and then runs the event loop in AppKit.

#import modules required by application
import objc
import Foundation
import AppKit

from PyObjCTools import AppHelper

# import modules containing classes required to start application and load MainMenu.nib
import In_ControlAppDelegate

# pass control to AppKit
AppHelper.runEventLoop()

The In_ControlAppDelegate class is imported purely for the purpose of loading it. Until it is loaded, the class will not exist in the run time, and cannot be used. Because the In_ControlAppDelegate class is needed in the MainMenu.xib nib file, it needs to be loaded before the nib file is opened. If there are any other classes that you need to load in your application early in the launch process, you can simply import them in main.py to force them to load.

Writing the Model Classes

Now that we have covered the project template code, we can move on to writing our own. We’ll begin with a few simple model classes: one to represent an Xgrid controller; one to represent a grid on a controller; and one to represent jobs running on a grid.

To create a file for the Controller class:

  1. Select the Classes group in the Groups & Files source list.
  2. Choose File > New File…
  3. In the Cocoa group, select Python NSObject subclass and click the Next button.
  4. Enter ‘Controller.py’ for the file name, and click Finish.

Repeat this procedure to create files called Job.py and Grid.py.

Select the Controller.py file in Groups & Files, and enter the following in the editor:

from Foundation import *
from objc import YES, NO
import objc

class Controller(NSObject):

    address = objc.ivar('address')
    grids = objc.ivar('grids')

    def initWithAddress_(self, address):
        self = super(Controller, self).init()
        if self:
            self.address = address
            self.grids = []
        return self

    def label(self):
        return self.address

    def isLeaf(self):
        return NO

    def children(self):
        return self.grids

    def numberOfChildren(self):
        return len(self.grids)

This class is quite simple: It has an instance variable called grids that is used to store a list of Grid objects for the controller, and an address variable for storing the network address of the controller’s host. To make the instance variable visible on the Objective-C side of the bridge, and to make it work properly with Cocoa features like bindings, you need to define it with the ivar function in the objc module, as shown.

The methods included are mostly to support the outline view that will be used to browse through the grids and jobs on the controller. For example, the label method is used to provide a string to represent the controller in the outline view, and the methods children, numberOfChildren, and isLeaf are required to tell the outline view whether a controller has children, how many there are, and what the children are. For a Controller object, the children are Grid objects.

I haven’t included any accessor methods in this code. You could, but to keep everything brief, in this example I will just set instance variables directly. Once you have declared your instance variables with the ivar method, key-value observation should work fine, meaning that when you set a variable, the interface will update appropriately.

There are a few other subtleties in the code above that are related to the PyObjC bridge between Python and Cocoa. Firstly, in order to use the YES and NO constants in Python code, we have to import them from the objc module. Secondly, in order to access the super class, you have to use the special super function, passing in the class and self variable. Lastly, note the naming convention used for PyObjC methods: wherever you would have a colon in the Objective-C name of a method, you must instead use an underscore. For example, the Objective-C method initWithName: becomes initWithName_ in Python. This is to cover up for the fact that Python and Objective-C have fundamentally different method invocation syntax, with Objective-C mixing the method name with the arguments, and Python using the more traditional name-followed-by-arguments approach. This looks a little ungainly at first, but you soon get used to it. (Don’t forget that Xcode supports code-completion to help you out, even in Python code.)

The code for the Job and Grid classes is similar. Job.py should look like this

from Foundation import *
from objc import YES, NO
import objc

class Job(NSObject):

    identifier = objc.ivar('identifier')
    name = objc.ivar('name')

    def initWithName_identifier_(self, name, identifier):
        self = super(Job, self).init()
        if self:
            self.name = name
            self.identifier = identifier
        return self

    def label(self):
        return self.identifier

    def isLeaf(self):
        return YES

    def children(self):
        return []

    def numberOfChildren(self):
        return 0

and Grid.py like this

from Foundation import *
from objc import YES, NO
import objc

class Grid(NSObject):

    label = objc.ivar('label')
    jobs = objc.ivar('jobs')

    def initWithLabel_(self, label):
        self = super(Grid, self).init()
        if self:
            self.label = label
            self.jobs = []
        return self

    def isLeaf(self):
        return NO

    def children(self):
        return self.jobs

    def numberOfChildren(self):
        return len(self.jobs)

Controlling In Control

With the models out of the way, we now turn to the controller code. We will use the In_ControlAppDelegate class as the controller for the application. Add the following code to In_ControlAppDelegate.py

from Foundation import *
from AppKit import *
from Job import *
from Grid import *
from Controller import *
import objc

class In_ControlAppDelegate(NSObject):

    controllers = objc.ivar('controllers')

    @classmethod
    def initialize(cls):
        defs = NSUserDefaults.standardUserDefaults()
        defs.registerDefaults_( {u"controllerHost":u"localhost"} )

    def init(self):
        self = super(In_ControlAppDelegate, self).init()
        if self:
            self.controllers = []
        return self

    def refresh_(self, sender):
        NSThread.detachNewThreadSelector_toTarget_withObject_("queryController", self, None)

    def queryController(self):
        import os

        def runCommand(command):
            "Run a command with error checking. Return value is True upon success."
            errorCode = os.system(command)
            if errorCode != 0:
                alert = NSAlert.alertWithMessageText_defaultButton_alternateButton_otherButton_informativeTextWithFormat_(u"Could not contact Xgrid controller.", u"OK", None, None, u"Check the host name and password before retrying.")
                alert.runModal()
                return False
            else:
                return True

        # Autorelease pool for memory management
        pool = NSAutoreleasePool.alloc().init()

        # Get hostname and password for controller from user defaults
        defs = NSUserDefaults.standardUserDefaults()
        controllerHost = defs.objectForKey_(u"controllerHost")
        controllerPassword = defs.objectForKey_(u"controllerPassword")
        if not controllerHost: controllerHost = u"localhost"
        if not controllerPassword: controllerPassword = u""

        # Get list of grids
        gridListCommand = "/usr/bin/xgrid -f xml -h %s -p %s -auth Password -grid list > /tmp/gridList.plist" % \
            (controllerHost, controllerPassword)
        if not runCommand(gridListCommand): return
        gridDict = NSDictionary.dictionaryWithContentsOfFile_(u"/tmp/gridList.plist")
        gridList = gridDict["gridList"]

        # Get jobs per grid
        grids = []
        for gridLabel in gridList:
            jobListCommand = "/usr/bin/xgrid -f xml -h %s -p %s -auth Password -job list -gid %s > /tmp/jobList.plist" % \
                (controllerHost, controllerPassword, gridLabel)
            returnVal = os.system(jobListCommand)
            if returnVal == 0:
                jobDict = NSDictionary.dictionaryWithContentsOfFile_(u"/tmp/jobList.plist")
                if not jobDict: continue
                jobList = jobDict["jobList"]
            else:
                continue

            # Loop through jobs on grid
            jobs = []
            for jobLabel in jobList:
                jobAttrCommand = "/usr/bin/xgrid -f xml -h %s -p %s -auth Password -job attributes -id %s > /tmp/jobAttrDict.plist" % \
                    (controllerHost, controllerPassword, jobLabel)
                returnVal = os.system(jobAttrCommand)
                if returnVal == 0:
                    jobAttrDict = NSDictionary.dictionaryWithContentsOfFile_(u"/tmp/jobAttrDict.plist")
                    if not jobAttrDict: continue
                    jobAttrs = jobAttrDict["jobAttributes"]
                else:
                    continue
                job = Job.alloc().initWithName_identifier_(jobAttrs["name"], int(jobLabel))
                jobs.append(job)

            # Append grid to list
            grid = Grid.alloc().initWithLabel_(gridLabel)
            grid.jobs = jobs
            grids.append(grid)

        # Create and set controller
        controller = Controller.alloc().initWithAddress_(controllerHost)
        controller.grids = grids
        self.controllers = [controller]

        # Clean up autorelease pool
        del pool

(One nasty aspect of the PyObjC bridge is that function names can get very long. Take the method alertWithMessageText_defaultButton_alternateButton_otherButton_informativeTextWithFormat_ from the NSAlert class. This method name is so long, it makes the code above difficult to read. If you can’t read it all, I suggest copying it out into an external editor.)

The In_ControlAppDelegate class has one instance variable, for the Controller model objects. The refresh_ method is actually an action method, that will be called when the user clicks the Refresh button, and the queryController method does the actual Xgrid queries and updates the model objects.

Because the Xgrid query is typically over a network, I perform it in a separate thread. The NSThread method detachNewThreadSelector_toTarget_withObject_ is used for this purpose. One subtlety of this invocation is that it expects to be passed a selector as the first argument. Whenever a selector is expected, the PyObjC bridge can be passed a standard Python string instead.

The queryController method basically just runs the xgrid command line tool a number of times, first to query the controller for its grids, then for the jobs in each grid. It’s not the prettiest of code, and a serious app would be better off using either the XGFoundation framework, or Charles Parnot’s great GridEZ framework. For our purposes though, it’s an interesting demonstration of how you can mix Cocoa and Python in one app. All of the queries are performed using Python’s built in os.system function, that simply runs a shell command. The results in each case are dumped to file, and then read and parsed by NSDictionary’s dictionaryWithContentsOfFile_ method, since the xgrid commands print out property lists.

When all of the grids and jobs have been retrieved, model objects are created to store the details, and combined into an object tree. Each grid gets a list of jobs assigned to its jobs instance variable, and each controller — only one in this case — is assigned a list of grids. Finally, the controller list is stored in the controllers instance variable of the In_ControlAppDelegate object.

In-Your-Face Interface

All that’s left to do now is the UI. We’ll build it all in the new Interface Builder 3.0, and connect it to our Python layer through the standard Cocoa techniques of target/action and bindings. The objects in our interface will have no idea whatsoever that they are talking to Python. We’ll move through this section pretty quickly, because it has little to do with the PyObjC bridge.

Double click the MainMenu.xib file in the Resources group of the In Control Xcode project to launch Interface Builder 3.0. Now layout some Cocoa controls and views in the window provided such that you end up with something like the screenshot below. You can find the new controls in the Library panel; if you can’t see the Library, select Tools > Library. Use the search field at the bottom to locate the classes you need, and drag them into the window.

In Control Window LayoutIn Control Window Layout

You will need an NSOutlineView for the main list. When you have dragged it onto the window, double click the column headers to enter the column titles shown. You will also need two Label instances, for the Controller and Password labels, a Text Field for the controller address, and a Secure Text Field for the password. Lastly, drag out a Rounded Textured Button for the Refresh button, and double click it to enter the text ‘Refresh’.

We are going to need one more object: an NSTreeController. This controller class will act as the intermediary between the NSOutlineView and the tree of model objects we have designed. To create the NSTreeController, locate it in the Library, and drag it onto the MainMenu.xib document window.

To add the refresh: action method in IB, click on the In_ControlAppDelegate instance, and bring up with Inspector by choosing Tools > Identity Inspector. In the Class Actions section, click the + button, and fill in the method refresh:. Now connect the target/action of the Refresh button by control-dragging from the button to the In_ControlAppDelegate object, and select refresh: from the heads-up display (HUD) that appears. (Gotta love IB 3.0!)

To configure the NSTreeController, select it, and choose the first inspector pane. The main thing we need to do is tell it what methods can be used to access information about the children. In our model classes, we defined some methods for this purpose, so now we can fill them in: In the Children field, fill in the method name children; for Count, fill in numberOfChildren; and for Leaf, fill in isLeaf.

NSTreeController InspectorNSTreeController Inspector

To finish off, we have to set a whole lot of bindings. Rather than go through them all individually, I will just give you a table, and leave the rest as an exercise. To set bindings, simply select the bound object, and choose the bindings pane in the inspector, which is fourth from the left, and has a two shapes as its icon.

Bound Object Binding Bind to Controller Key Model Key
Tree Controller Content Array In_ControlAppDelegate controllers
NSOutlineView Content Tree Controller arrangedObjects
NSOutlineView Selection Index Paths Tree Controller selectionIndexPaths
NSOutlineView Controllers & Grids Column Value Tree Controller arrangedObjects label
NSOutlineView Job Name Column Value Tree Controller arrangedObjects name
Controller Text Field Value Shared User Defaults Controller values controllerHost
Password Secure Text Field Value Shared User Defaults Controller values controllerPassword

Because the Job Name column only applies to Job objects, and not to Controller or Grid objects, we need to make sure that if there is no name method, that the controller just ignores it and doesn’t throw an exception. To do this, return to the bindings for the Job Name column, and in Value binding uncheck ‘Raises for Not Applicable Keys’.

Build and Browse

In theory, that completes the exercise. If you are too lazy to try all of this yourself, you can download the In Control project source code here.

Try clicking Build & Go in Xcode, and test the app by connecting to an Xgrid controller. Also test what happens if you enter a non-existent controller.

It’s worth stopping to think about how seamless the PyObjC bridge works, and what you get in return. For example, Cocoa bindings just work, so that modifying the model layer is immediately reflected in the view layer. And, in your Python code, you no longer have to worry about memory management — the bridge takes care of it for you.

There are many more advantages too, but perhaps it is better if you read about those elsewhere, like on Apple’s site, on MacDevCenter, and at the PyObjC site. There are also a few online [1, 2] posts about the support of PyObjC in Leopard. And if you don’t believe you can write ‘real’ software with a scripting bridge, read about Checkout, which was developed with PyObjC.

Leopard ushers in a whole new world of possibilities for developers, and BridgeSupport is a big part of that. In fact, I will be surprised if in 10 years time nearly all new applications are not written in a dynamic scripting language like Python or Ruby. And for scientists who already work with these scripting languages, it is an invitation to start developing great graphical apps in a fraction of the time that it takes in traditional languages like C++ and even Objective-C. Go get ‘em!

AttachmentSize
InControlSourceCode.zip52.21 KB
InControl.zip31.62 KB

Comments

Comment viewing options

Select your preferred way to display the comments and click "Save settings" to activate your changes.

Great tutorial

Thanks, Drew, this is a great tutorial!

Threads and tempfile

Nice tutorial, I'm looking forward to trying PyObjc (again). One thing that caught my eye while skimming over the source code: Shouldn't you be using tempfile.NamedTemporaryFile to create independent tempfiles in your thread?

Tempfile

You're right. The current implementation is not perfect. I think you will also run into race conditions if you press the refresh button while another refresh is already running, for the same reason: each refresh uses the same files.
So, there is plenty of room for improvement, but I hope the main lesson --- how you can use PyObjC to build Python-Cocoa apps --- comes across.

Drew

---------------------------
Drew McCormack
http://www.maccoremac.com
http://www.macanics.net
http://www.macresearch.org

Hi, it's great review. Thank

Hi,
it's great review. Thank you very much. I was looking for trying python and Cocoa for years and now it's really easy. Could you please post the archive. You provided a link to the source, but file is not found.

Links in PyObjC Tutorial

Sorry about the links. You could always use the links at the bottom of the article, but I have fixed the links in the article itself now too.

Drew

---------------------------
Drew McCormack
http://www.maccoremac.com
http://www.macanics.net
http://www.macresearch.org

Hi Drew, thanks for

Hi Drew,
thanks for providing links. I tried your example yesterday and still don't understand how to bind my output to UI. I made a similar project, but regardless what I do I fail to get the output on UI. It may help if you post your output plist file, that I can fake running xgrid command and grab the output in order to debug and learn how to put it on UI.

Thanks a lot,
Valentin.

Plists

Hi Valentin,

You could download XgridLite and turn your computer into an xgrid controller to test it:

http://edbaskerville.com/software/xgridlite/

Otherwise, you could generate your own plists. Here are a few simple examples to help:

The grid list plist looks like this:

<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE plist PUBLIC "-//Apple//DTD PLIST 1.0//EN" "http://www.apple.com/DTDs/PropertyList-1.0.dtd">
<plist version="1.0">
<dict>
	<key>gridList</key>
	<array>
		<string>0</string>
	</array>
</dict>
</plist>

The job list looks like this:

<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE plist PUBLIC "-//Apple//DTD PLIST 1.0//EN" "http://www.apple.com/DTDs/PropertyList-1.0.dtd">
<plist version="1.0">
<dict>
	<key>jobList</key>
	<array>
		<string>161</string>
		<string>165</string>
	</array>
</dict>
</plist>

and the important part of the job attributes plist looks like this

<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE plist PUBLIC "-//Apple//DTD PLIST 1.0//EN" "http://www.apple.com/DTDs/PropertyList-1.0.dtd">
<plist version="1.0">
<dict>
	<key>jobAttributes</key>
	<dict>
		<key>name</key>
		<string>adf_test_runadf.sh</string>
	</dict>
</dict>
</plist>

Hope that helps.

Drew

---------------------------
Drew McCormack
http://www.maccoremac.com
http://www.macanics.net
http://www.macresearch.org

great tutorials... i want more

your tutorials are some of the best on the web, although ironically they are hard to find. my only suggestion is that you have one screen shot for every part of the task...

really why I am posting is to request a tutorial for Python and Interface Builder 3.0 (with xcode 3.0 in Leopard) that includes a graphing utility.

If you had a tutorial on this topic, it might literally be the only one of the (searchable or easily found) web. Plus, it would save me tens of hours.

thanks, and keep up the good work.

-jfd

Debugging

Drew, an inspiring article. I like Python very much and fully agree with your last paragraph. At this moment I see one problem with serious using of PyObjC - XCode can not debug it (or I do not know how to run it). Nevertheless this problem will sooner or later disappear.
Jaromir Siska

Debugging Python

Agreed.

The other issue going forward is that you can't use garbage collection with python. This is not such a problem, because python already has its own form of GC, but it could be an issue if you want to mix a bit of python into a GC Obj-C Cocoa app, for example.

Drew
---------------------------
Drew McCormack
http://www.maccoremac.com
http://www.macanics.net
http://www.macresearch.org

Debugging

Excuse me, what GC stands for?
Jaromir

GC

Garbage collection. It means you don't have to worry about cleaning up objects when you are finished with them. They get cleaned up automatically when they are no longer in use.

Drew

---------------------------
Drew McCormack
http://www.maccoremac.com
http://www.macanics.net
http://www.macresearch.org

How Do You Make Simple Program?

I'm sorry to rain on the parade but this was way too complicated for me. I appreciate you taking the time and effort to put in all the detail about trees but I feel you skipped over one of the most difficult and confusing parts of Cocoa-Python projects: how do you get the .xib file in Interface Builder to attach a controller class to an actual python method call?

I have been working for the last two days and can't even get a button to beep when I click on it. The operation to make such a program is trivial in Objective C and I worked through the tutorials to do it in Obj C in minutes. But in Cocoa-Python, it seems all but damn-near impossible. I don't know what the magic click or keyword or import it is but the damn thing just will not work.

Please help :(

Actions

You need to indicate that your method is an action using @IBAction. Here is an example from a tutorial which I found here:

    # IBactions must be valid objc selectors
    # in this case it's resetMovie:
    @IBAction
    def resetMovie_(self, sender):
        ...

Check out the tutorial -- it looks like it might help with your problem.

Drew

---------------------------
Drew McCormack
http://www.maccoremac.com
http://www.macanics.net
http://www.macresearch.org

Doesn't Objective-C 2.0 have garbage collection?

Leopard's Objective-C 2.0 has garbage collection. Can't it be used with Python and PyObjC?

PyObjc and GC

Apparently it was too difficult to integrate the ObjC garbage collector with the python garbage collector, so you can't use it. But it's not such a big problem, because you will do most of your coding in python, and that has a basic form of garbage collection -- which also works with the ObjC objects you create -- so it is as if you are working in a GC language.

Drew

---------------------------
Drew McCormack
http://www.maccoremac.com
http://www.macanics.net
http://www.macresearch.org