Red Artisan - Blog

Switching Views with a UISegmentedControl - Revisited

2010-06-27T12:00Z

Recently I investigated switching between multiple different views using a UISegmentedControl, similar to iCal or the AppStore application.

Background

The best approach I could find to work at the time was to use a managing view controller, that enclosed an array of sub view controllers. The UISegmentedControl would then switch between these sub views by adding/removing the selected segment’s view as a subview of the managing controller’s view.

While this worked and satisfied one of my main requirements of keeping the logic between view controllers separate, there were a few things that frustrated me about the approach:

The enclosing view controller that managed the sub views was in effect a ‘container’ view controller, and I distinctly remembered Evan Doll’s WWDC 2009 presentation where cautioned against building any style of container view controllers.
To push onto the navigation stack from within a sub view, each sub view needed to have a reference back to the managing container view controller, either via a property and/or custom constructor, as the sub views weren’t created within the navigation hierarchy and hence had a nil navigationController property.
Since the managing view controller enclosed a series of sub view controllers, some in the view hierarchy and some not, view life cycle messages needed to be forwarded to the sub view controllers to be good UIKit citizens.

While attending WWDC 2010 just a few weeks ago, I managed to talk to several UIKit engineers and together we managed to find a much better approach to solving this UI paradigm without requiring a container view controller.

New Shiny

The new approach is to utilize a UINavigationController rather than a managing view controller. However, rather than use the more common navigation controller methods pushViewController:animated: and popViewControllerAnimated:, we will manipulate the navigation view hierarchy directly by modifying the viewControllers property using the setViewControllers:animated: method.

The technique essentially works as follows. Any index change in the designated UISegmentedControl calls upon a method in a custom NSObject descendant controller object of ours. This controller accesses the selected view controller appropriate for the selected segment and installs it into the navigation controller stack directly using the setViewControllers:animated: method.

Finally, since the navigation view hierarchy has been modified directly, we then re-install the segmented control as the title view on the incoming view controller so that further segment changes can be made.

I’ve reimplemented the previous example application I built using the managing view controller with this new pattern, let’s step through it to demonstrate how it all works.

Since we’ll be using standard UINavigationController and UISegmentedControl objects in this application, I’ll skip straight to our custom controller object that accepts a message indicating a change in selected segment index, and does the navigation controller magic.

Interface

:::objective-c
@interface SegmentsController : NSObject {
    NSArray                * viewControllers;
    UINavigationController * navigationController;
}

@property (nonatomic, retain, readonly) NSArray                * viewControllers;
@property (nonatomic, retain, readonly) UINavigationController * navigationController;

- (id)initWithNavigationController:(UINavigationController *)aNavigationController
                   viewControllers:(NSArray *)viewControllers;

- (void)indexDidChangeForSegmentedControl:(UISegmentedControl *)aSegmentedControl;

@end

Here we define the SegmentsController interface to be an NSObject descendant, with storage for the view controllers appropriate for each segment, and a reference to our navigation controller.

A custom constructor accepts the view and navigation controller, and the indexDidChangeForSegmentedControl: is our method that can be invoked when a given segmented control index changes.

Implementation

:::objective-c
@interface SegmentsController ()
@property (nonatomic, retain, readwrite) NSArray                * viewControllers;
@property (nonatomic, retain, readwrite) UINavigationController * navigationController;
@end

@implementation SegmentsController

@synthesize viewControllers, navigationController;

- (id)initWithNavigationController:(UINavigationController *)aNavigationController
                   viewControllers:(NSArray *)theViewControllers {
    if (self = [super init]) {
        self.navigationController = aNavigationController;
        self.viewControllers = theViewControllers;
    }
    return self;
}

- (void)indexDidChangeForSegmentedControl:(UISegmentedControl *)aSegmentedControl {
    NSUInteger index = aSegmentedControl.selectedSegmentIndex;
    UIViewController * incomingViewController = [self.viewControllers objectAtIndex:index];

    NSArray * theViewControllers = [NSArray arrayWithObject:incomingViewController];
    [self.navigationController setViewControllers:theViewControllers animated:NO];

    incomingViewController.navigationItem.titleView = aSegmentedControl;
}

- (void)dealloc {
    [super dealloc];
    self.viewControllers = nil;
    self.navigationController = nil;
}

@end

In the anonymous category we redefine our properties read/write so we can mutate them from within the implementation only, and define our constructor to store references to our given view and navigation controllers.

The meat of the work is done next. Our segmented control will be appropriately configured to call upon indexDidChangeForSegmentedControl: when it’s segment index changes. When this occurs, we retrieve the new index from the segmented control, and the relevant view controller to install (a more complex example could instantiate/cache view controllers to conserve memory), and assign it to the navigation controller via the setViewControllers:animated: message.

Once installed, we then apply the segmented control to the title view of the view controller we just installed into the navigation controller, and are done.

Finally, we implement appropriate memory management methods to de-allocate resources when being released.

Application Delegate Implementation

:::objective-c
- (BOOL)application:(UIApplication *)application didFinishLaunchingWithOptions:(NSDictionary *)launchOptions {

    NSArray * viewControllers = [self segmentViewControllers];

    UINavigationController * navigationController = [[[UINavigationController alloc] init] autorelease];
    self.segmentsController = [[SegmentsController alloc] initWithNavigationController:navigationController viewControllers:viewControllers];

    self.segmentedControl = [[UISegmentedControl alloc] initWithItems:[viewControllers arrayByPerformingSelector:@selector(title)]];
    self.segmentedControl.segmentedControlStyle = UISegmentedControlStyleBar;

    [self.segmentedControl addTarget:self.segmentsController
                              action:@selector(indexDidChangeForSegmentedControl:)
                    forControlEvents:UIControlEventValueChanged];

    [self firstUserExperience];

    [window addSubview:navigationController.view];
    [window makeKeyAndVisible];

    return YES;
}

In this example, I’ve instantiated and configured the segmented control, navigation and segment controllers from within the application delegate, but this could equally be done at a lower level depending on your application.

In particular, the segmented control has its target/action pair set up to point to our indexDidChangeForSegmentedControl: method described above.

The segmentViewControllers methods returns the view controllers relating to each segment (the title property of each view controller is used as the segment’s title via the NSArray arrayByPerformingSelector: extension), and firstUserExperience kicks everything off by selecting and installing the first segment.

:::objective-c
- (NSArray *)segmentViewControllers {
    UIViewController * italy     = [[ItalyViewController alloc] initWithNibName:@"ItalyViewController" bundle:nil];
    UIViewController * australia = [[AustraliaViewController alloc] initWithStyle:UITableViewStyleGrouped];

    NSArray * viewControllers = [NSArray arrayWithObjects:italy, australia, nil];
    [australia release]; [italy release];

    return viewControllers;
}

- (void)firstUserExperience {
    self.segmentedControl.selectedSegmentIndex = 0;
    [self.segmentsController indexDidChangeForSegmentedControl:self.segmentedControl];
}

Summary

Using a UINavigationController based approach has several distinct advantages that I quite like – it doesn’t require a container controller, and hence no custom code for handling memory, rotation, or view life cycle events.

Code-wise it’s much smaller than the previous implementation, and will be a lot easier to maintain. Segment view controllers can push directly onto the navigation controller stack since they’re set within the navigation hierarchy, and no special management of parent view controllers is required.

The full XCode project of the example above is available if you’d like to examine it further. Enjoy.

Multiple Views with a UISplitViewController

2010-06-14T12:00Z

The iPad has ushered in a suite of new awesome user interface paradigms with it’s large screen and enhanced performance.

In this article, I’ll step through using one of the new user interface elements in iPhone SDK 3.2+, the UISplitViewController, in particular managing multiple views with their own navigation controller stack, handling all orientations.

Concept

Since I’m an avid cyclist and the Giro d'Italia just finished, in this article’s we’ll step through an example application that lists the name and distance of each stage.

When tapped, we’ll show a photo from Melbourne’s most popular cycling blog, Cycling Tips, and allow the user to tap through to the Cycling Tips article summarizing the stage (full credits to Cycling Tips, they’ve done an awesome job at covering the Giro this year).

I’ll use landscape orientation terminology to describe the visual aspects of the split view controller in this article, but please assume normal split view controller semantics with the left hand side of the split view appearing in a popup view when in portrait mode. We’ll discuss the code required to get this to work as well.

Model

Since we want to focus on the UISplitViewController, I’ll browse over the data model for this particular application since it is quite small, all the code is available online, and it’s just a single Core Data Stage entity containing attributes relevant to each stage.

I definitely recommend utilizing Core Data for your models where possible, I find it fast and easy to use, and its great for prototyping new ideas.

For data persistence it’s awesome, you can switch between using in-memory storage and SQLlite during the development making it easy to always have fresh content while you’re making changes.

Table view list of Stages

On to the controller code, along the left hand side of the split view controller, we’ll show a table view, with each stage of the Giro being listed in a separate cell.

StageTableViewController Interface

:::objective-c
@interface StageTableViewController : UITableViewController {
    NSFetchedResultsController * stageResultsController;

    NSMutableDictionary        * stageViewControllers;

    UIPopoverController        * popoverController;
    UIBarButtonItem            * popoverButtonItem;
}

@property (nonatomic, retain) NSFetchedResultsController * stageResultsController;

@property (nonatomic, retain) NSMutableDictionary        * stageViewControllers;

@property (nonatomic, retain) UIPopoverController        * popoverController;
@property (nonatomic, retain) UIBarButtonItem            * popoverButtonItem;

- (void)autoselectFirstStage;

@end

@protocol PopupManagingViewController
- (void)showPopoverButtonItem:(UIBarButtonItem *)barButtonItem;
- (void)invalidatePopoverButtonItem:(UIBarButtonItem *)barButtonItem;
@end

The stageResultsController property provides access to our Core Data backed model.

In addition to this we define storage for the known view controllers we present on the right hand side of the split view controller. This effectively caches view controllers we’ve already shown, so if the user taps back to them, they show instantly rather than re-fetch data.

We also define an autoselectFirstStage method that the application delegate can use to automatically select and show the first stage upon startup.

When orientated to portrait, the left hand side of the split view controller disappears and becomes available via a popup. The popoverController property is responsible for showing the view controller inside a popover view when a user taps the popoverButtonItem button.

We also define a protocol our detail view controller can implement to install and remove the button allowing access to the popover when we switch between detail views in portrait mode.

StageTableViewController Implementation

:::objective-c
@implementation StageTableViewController

@synthesize stageResultsController, stageViewControllers, popoverButtonItem, popoverController;

- (void)viewDidLoad {
    [super viewDidLoad];

    self.title = @"Stages";
    self.stageResultsController = [[ContentController sharedInstance] stageResultsController];

    NSUInteger stageCount = [self tableView:self.tableView numberOfRowsInSection:0];
    self.contentSizeForViewInPopover = CGSizeMake(320.0, self.tableView.rowHeight * stageCount);
    self.clearsSelectionOnViewWillAppear = NO;

    self.stageViewControllers = [NSMutableDictionary dictionary];
}

- (void)autoselectFirstStage {
    NSIndexPath * startPath = [NSIndexPath indexPathForRow:0 inSection:0];
    [self.tableView selectRowAtIndexPath:startPath animated:NO scrollPosition:UITableViewScrollPositionNone];
    [self.tableView.delegate tableView:self.tableView didSelectRowAtIndexPath:startPath];
}

In viewDidLoad, we configure the view controller and create references to our model layer’s NSFetchedResultsController. Our autoselectFirstStage method ensures the table view and delegate are informed of our selection.

:::objective-c
- (NSInteger)numberOfSectionsInTableView:(UITableView *)tableView {
    return [[self.stageResultsController sections] count];
}

- (NSInteger)tableView:(UITableView *)tableView numberOfRowsInSection:(NSInteger)section {
    id <NSFetchedResultsSectionInfo> sectionInfo = [[self.stageResultsController sections] objectAtIndex:section];
    return [sectionInfo numberOfObjects];
}

- (UITableViewCell *)tableView:(UITableView *)tableView cellForRowAtIndexPath:(NSIndexPath *)indexPath {

    static NSString *CellIdentifier = @"StageCell";

    UITableViewCell *cell = [tableView dequeueReusableCellWithIdentifier:CellIdentifier];
    if (cell == nil) {
        cell = [[[UITableViewCell alloc] initWithStyle:UITableViewCellStyleSubtitle reuseIdentifier:CellIdentifier] autorelease];
    }

    Stage * stage = [self.stageResultsController objectAtIndexPath:indexPath];
    cell.textLabel.text = stage.name;
    cell.detailTextLabel.text = stage.formattedDistance;

    return cell;
}

We then implement the required UITableViewDataSource protocol methods to return our content to fill the table view when requested.

For some extra shine, we would also want to implement a custom UITableViewCell as well to nicely show all the stage content. I’ll follow this up in a subsequent article.

:::objective-c
- (void)tableView:(UITableView *)tableView didSelectRowAtIndexPath:(NSIndexPath *)indexPath {
    Stage * stage = [self.stageResultsController objectAtIndexPath:indexPath];

    // invalidate the current popover button if one is being used
    UINavigationController * currentStageNavigationController = self.splitViewController.rightSideViewController;
    UIViewController<PopupManagingViewController> * currentViewController = [currentStageNavigationController rootViewController];
    [currentViewController invalidatePopoverButtonItem:self.popoverButtonItem];

    // install the new viewcontrollers array (LHS: stage view controller, RHS: incoming detail navigation controller)
    UINavigationController * stageNavigationController = [self.stageViewControllers valueForKey:stage.name];

    if (!stageNavigationController) {
        StageDetailViewController * detailViewController = [[StageDetailViewController alloc] initWithStage:stage];
        stageNavigationController = [[UINavigationController alloc] initWithRootViewController:detailViewController];
        [detailViewController release];
        [self.stageViewControllers setValue:stageNavigationController forKey:stage.name];
        [stageNavigationController release];
    }

    self.splitViewController.viewControllers = [NSArray arrayWithObjects:self.navigationController, stageNavigationController, nil];

    // dismiss the popover if present, and add the popover button to the incoming detail view controller
    [self.popoverController dismissPopoverAnimated:YES];

    if (self.popoverButtonItem) {
        UIViewController<PopupManagingViewController> * viewController = [stageNavigationController rootViewController];
        [viewController showPopoverButtonItem:self.popoverButtonItem];
    }
}

The only UITableViewDelegate method we’ll implement is tableView:didSelectRowAtIndexPath: for when a user taps a particular stage in the table view.

In tableView:didSelectRowAtIndexPath: we perform a few tasks. Essentially, we want to create a StageDetailViewController instance within a navigation controller, and cache it in case the user selects this table row again.

Once we’ve created (or retrieved a cached version of) our StageDetailViewController we reassign the viewControllers property on the split controller, specifying the new left/right hand side view controllers to be shown.

In this case, we never want the left hand side to change from being the table view of all stages, so position 0 in the returned array is the navigationController reference of the StageTableViewController (be careful to remember this in your apps as you’ll get some funny behaviour if you return the view controller directly).

Position 1 of the array includes the newly created stage detail view controller, inset within a UINavigationController instance.

Once the ‘viewController’ property has been assigned, the split view updates instantly to show our selected view controller in the right hand side of the split view.

In addition to this, if the user selected an item in the table while in portrait mode from a popover, we dismiss the popover view, and install a new popover button in the detail view that the user can tap if they wish to select another item.

Finally as good memory management citizens we also implement viewDidUnload and didReceiveMemoryWarning to release allocated and cached resources respectively.

:::objective-c
- (void)viewDidUnload {
    [super viewDidUnload];
    self.stageViewControllers   = nil;
    self.popoverButtonItem      = nil;
    self.popoverController      = nil;
    self.stageResultsController = nil;
}

- (void)didReceiveMemoryWarning {
  [super didReceiveMemoryWarning];
  [self.stageViewControllers removeAllObjects];
}

The full XCode project also includes the UISplitViewController delegate code to install/remove the popover button when the user changes orientation.

Now that we’ve implemented the left hand side of the split view, let’s move onto the right hand side detail view.

StageDetailViewController interface

The right hand side of the split view controller will initially show a picture from the selected Giro racing stage. We’ll also place this image inside a scroll view so the user can pan and zoom in/out, etc.

We will also install a button within the navigation bar so the user can drill down to the blog post summarizing that stage.

Here’s the interface:

:::objective-c
@interface StageDetailViewController : UIViewController <ImageLoaderDelegate, UIScrollViewDelegate, PopupManagingViewController> {
    Stage                   * stage;

    UIScrollView            * scrollView;
    UIActivityIndicatorView * activityView;

    UIImageView             * stageImageView;
    ImageLoader             * imageLoader;
}

@property (nonatomic, retain) Stage                            * stage;

@property (nonatomic, retain) IBOutlet UIScrollView            * scrollView;
@property (nonatomic, retain) IBOutlet UIActivityIndicatorView * activityView;

@property (nonatomic, retain) IBOutlet UIImageView             * stageImageView;
@property (nonatomic, retain) ImageLoader                      * imageLoader;

- (id)initWithStage:(Stage *)stage;

@end

The ImageLoader property is a helper class I’ve written for asynchronously downloading images over the network, utilizing the excellent ASIHttpRequest library.

StageDetailViewController implementation

:::objective-c
@interface StageDetailViewController ()
- (void)didSelectReadArticle:(id)sender;
@end

@implementation StageDetailViewController

@synthesize stage, stageImageView, imageLoader, activityView, scrollView;

- (id)initWithStage:(Stage *)aStage {
    if (self = [super initWithNibName:@"StageDetailViewController" bundle:nil]) {
        self.stage = aStage;
    }
    return self;
}

- (void)viewDidLoad {
    [super viewDidLoad];

    self.title = [NSString stringWithFormat:@"Giro d'Italia: %@", self.stage.name];

    self.view.backgroundColor = [UIColor scrollViewTexturedBackgroundColor];

    self.scrollView.maximumZoomScale = 5.0f;
    self.scrollView.minimumZoomScale = 1.0f;
    self.scrollView.delegate = self;

    self.navigationItem.rightBarButtonItem = [[UIBarButtonItem alloc] initWithTitle:@"Read Article" style:UIBarButtonItemStylePlain target:self action:@selector(didSelectReadArticle:)];
    self.navigationItem.backBarButtonItem  = [[UIBarButtonItem alloc] initWithTitle:[NSString stringWithFormat:@"Stage %@", self.stage.index] style:UIBarButtonItemStylePlain target:nil action:nil];

    self.imageLoader = [[ImageLoader alloc] initWithURL:self.stage.imageURL];
    self.imageLoader.delegate = self;
    [self.imageLoader start];
}

- (UIView *)viewForZoomingInScrollView:(UIScrollView *)scrollView {
    return self.stageImageView;
}

In the anonymous category we define a didSelectReadArticle: method to be invoked when the taps the button to drill down to read the full blog post summarizing the race stage.

In viewDidLoad, we configure the view’s title and background colour (scrollViewTexturedBackgroundColor looks just awesome on the iPad), the scroll view’s scale and delegate properties, install a ‘Read Article’ button on the right hand side of the navigation bar, and redefine the back button to be the stage number rather than the full stage name.

Finally, we create an ImageLoader instance and start retrieving an image of the specified stage.

Should a user tap the ‘Read Article’ button the following method is called:

:::objective-c
- (void)didSelectReadArticle:(id)sender {
    StageArticleViewController * articleViewController = [[StageArticleViewController alloc] initWithStage:self.stage];
    [self.navigationController pushViewController:articleViewController animated:YES];
    [articleViewController release];
}

This will push an instance of StageArticleViewController onto the navigation stack, which includes a web view configured to load the race stage summary.

Once again we ensure all allocated resources are released when the view is deconstructed.

:::objective-c
- (void)viewDidUnload {
    [super viewDidUnload];
    self.stageImageView       = nil;
    self.imageLoader.delegate = nil;
    self.imageLoader          = nil;
    self.scrollView           = nil;
}

- (void)dealloc {
    [super dealloc];
    self.stage = nil;
}

Here we split deallocation of the view related items from the stage, since the stage was specified in the constructor whereas stageImageView, scrollView and the imageLoader objects were created in viewDidLoad.

Summary

As we can see, UISplitViewController is really useful for master-detail style interfaces and it’s configured similarly to a UITabBarController by assigning it’s viewControllers property.

I’ve skipped over a few parts of the example app to keep the post length under control, such as implementation of the StageArticleViewController, orientation permissions on all our view controllers, the image loader callbacks to add the image to the scroll view upon completion of download, etc. All of this is availablae in the full XCode project however, please feel free to peruse the code online.

Australasian Surf Business Magazine iPhone App

2010-05-30T12:00Z

Its great to see an application you’ve been working on recently be accepted and made available on the Apple iTunes store.

Just last week, we released the Australasian Surf Business Magazine (asbmag) iPhone application!

asbmag’s application, targeted at those working within the surf industry, allows you to keep up to date with the latest industry news and job content on the iPhone, normally available on their website.

The design of the application has been to focus on the user’s experience while reading content on the phone, and integrate with the native capabilities of the device.

Content is locally in Core Data so that the application works well offline. When online, it efficiently retrieves the latest news and job articles from www.asbmag.com via an atom feed, and appropriately updates the device.

If you’re keen to keep up with what’s happening in the Surf Industry, download the app for free and enjoy!

Switching between Views with a UISegmentedControl

2010-05-26T12:00Z

Please note an alternative approach to this article is available in further writing: Switching Views with a UISegmentedControl – Revisited

It’s boutique, shiny and several apps that ship with the iPhone do it, in this article I’ll step through using a UISegmentedControl to toggle between different subviews, each with their own layout, within a UINavigationController.

To see an example of this use case, take a look at the Apple Calendar app, along the bottom of the screen is a toolbar containing a UISegmentedControl, with the segments List, Day and Month. Tapping on any of these segments changes the view to a completely new layout.

The AppStore app also implements this paradigm to an extent – take a look at the Top 25 tab, in the navigation title view there’s a UISegmentedControl with the segment labels Top Free, Top Paid and Top Grossing.

The AppStore variant is a bit simpler and can actually be implemented using a single UITableView and multiple UITableViewDataSource/UITableViewDelegate objects, with the UISegmentedControl switching between each of them and reloading the table upon index changes.

In my case though, I needed to build the full Ferrari, and allow for switching between views with completely different layouts. In addition, it all had to work well within a navigation controller, with elements within these views pushing onto the navigation stack.

Solution

The solution was to create specialized view controllers for each style of view, and programatically add/remove these views as subviews to a managing view controller, in response to selected segment index changes on the UISegmentedControl.

To do all this, one has to create an additional UIViewController subclass that manages the UISegmentedControl changes, maintains a collection of sub view controllers and adds/removes their views on demand.

The advantage of this approach, is that it keeps the business logic between each subview separate, and makes it easy to add additional segments later as your application grows.

Interface

:::objective-c
@interface SegmentManagingViewController : UIViewController <UINavigationControllerDelegate> {
    UISegmentedControl    * segmentedControl;
    UIViewController      * activeViewController;
    NSArray               * segmentedViewControllers;
}

@property (nonatomic, retain, readonly) IBOutlet UISegmentedControl * segmentedControl;
@property (nonatomic, retain, readonly) UIViewController            * activeViewController;
@property (nonatomic, retain, readonly) NSArray                     * segmentedViewControllers;

@end

Here we define a view controller subclass for managing the presentation of multiple subviews. segmentedControl is the UISegmentedControl, either created and assigned in code, or via a Interface Builder. activeViewController represents the view controller currently being presented, and segmentedViewControllers is an array of all view controllers presentable via the segmented control.

I’ll get to the UINavigationControllerDelegate protocol in just a minute..

Implementation

:::objective-c
@interface SegmentManagingViewController ()

@property (nonatomic, retain, readwrite) IBOutlet UISegmentedControl * segmentedControl;
@property (nonatomic, retain, readwrite) UIViewController            * activeViewController;
@property (nonatomic, retain, readwrite) NSArray                     * segmentedViewControllers;

- (void)didChangeSegmentControl:(UISegmentedControl *)control;
- (NSArray *)segmentedViewControllerContent;

@end

In an anonymous category we redefine the interface properties as readwrite for local accessor/mutator methods, and define method signatures for a callback from the segmented control, and a helper method to create the view controllers representing each segment.

:::objective-c
@implementation SegmentManagingViewController

@synthesize segmentedControl, activeViewController, segmentedViewControllers;

- (void)viewDidLoad {
    [super viewDidLoad];

    self.segmentedViewControllers = [self segmentedViewControllerContent];

    NSArray * segmentTitles = [self.segmentedViewControllers arrayByPerformingSelector:@selector(title)];

    self.segmentedControl = [[UISegmentedControl alloc] initWithItems:segmentTitles];
    self.segmentedControl.selectedSegmentIndex = 0;
    self.segmentedControl.segmentedControlStyle = UISegmentedControlStyleBar;

    [self.segmentedControl addTarget:self
                              action:@selector(didChangeSegmentControl:)
                    forControlEvents:UIControlEventValueChanged];

    self.navigationItem.titleView = self.segmentedControl;
    [self.segmentedControl release];

    [self didChangeSegmentControl:self.segmentedControl]; // kick everything off
}

- (NSArray *)segmentedViewControllerContent {
    UIViewController * controller1 = [[ItalyViewController alloc] initWithParentViewController:self];
    UIViewController * controller2 = [[AustraliaViewController alloc] initWithParentViewController:self];

    NSArray * controllers = [NSArray arrayWithObjects:controller1, controller2, nil];

    [controller1 release];
    [controller2 release];

    return controllers;
}

viewDidLoad creates our UISegmentedControl object within the navigation controller title, and installs a target/action pair to call back on didChangeSegmentControl: when the selected segment index changes. It also calls upon segmentedViewControllerContent to return an array containing the view controllers we’ll be toggling between. In this case, view controllers representing Italy (fantastic holiday a few years back) and Australia, where I come from.

We’ll also implement our memory management methods:

:::objective-c
- (void)didReceiveMemoryWarning {
    [super didReceiveMemoryWarning];

    for (UIViewController * viewController in self.segmentedViewControllers) {
        [viewController didReceiveMemoryWarning];
    }
}

- (void)viewDidUnload {
    self.segmentedControl         = nil;
    self.segmentedViewControllers = nil;
    self.activeViewController     = nil;

    [super viewDidUnload];
}

Now onto the beef, when a segment index change occurs, the UISegmentedControl will call back to our didChangeSegmentControl: method, where we can interrogate the segmented control for the new index.

When this changes we need to remove the current active subview from the view hierarchy, and replace it with a new one according to the users segmented control selection.

Since we’re also manipulating the view hierarchy directly, we also need to fire viewWillDisappear:/viewDidDisappear: and their counterparts viewWillAppear:/viewDidAppear: appropriately as well to ensure the outbound and inbound view controllers are notified of their view’s visual status change:

:::objective-c
- (void)didChangeSegmentControl:(UISegmentedControl *)control {
    if (self.activeViewController) {
        [self.activeViewController viewWillDisappear:NO];
        [self.activeViewController.view removeFromSuperview];
        [self.activeViewController viewDidDisappear:NO];
    }

    self.activeViewController = [self.segmentedViewControllers objectAtIndex:control.selectedSegmentIndex];

    [self.activeViewController viewWillAppear:NO];
    [self.view addSubview:self.activeViewController.view];
    [self.activeViewController viewDidAppear:NO];

    NSString * segmentTitle = [control titleForSegmentAtIndex:control.selectedSegmentIndex];
    self.navigationItem.backBarButtonItem  = [[UIBarButtonItem alloc] initWithTitle:segmentTitle style:UIBarButtonItemStylePlain target:nil action:nil];
}

@end

The final part extracts the title of the selected segment, and creates a ‘back’ UIBarButtonItem with it’s name matching that title. This ensures that when we push onto the navigation stack from within one of these subviews, the navigation item back button matches the name of the selected segment.

We also pass on any view controller life cycle methods to the active subview:

:::objective-c
- (void)viewWillAppear:(BOOL)animated {
    [super viewWillAppear:animated];
    [self.activeViewController viewWillAppear:animated];
}

- (void)viewDidAppear:(BOOL)animated {
    [super viewDidAppear:animated];
    [self.activeViewController viewDidAppear:animated];
}

- (void)viewWillDisappear:(BOOL)animated {
    [super viewWillDisappear:animated];
    [self.activeViewController viewWillDisappear:animated];
}

- (void)viewDidDisappear:(BOOL)animated {
    [super viewDidDisappear:animated];
    [self.activeViewController viewDidDisappear:animated];
}

Navigation Controllers

Interestingly, if we place this managing view controller within a UINavigationController, the managing view controller won’t actually receive the viewWillAppear:/viewDidAppear: events from the system. To be notified of when this occurs inside a navigation view hierarchy, we need to implement the UINavigationControllerDelegate methods to be informed when a view has been pushed on or off the navigation stack.

Without these methods bizarre side effects can occur, such as UITableView’s within a segments subview not knowing when to appropriately de-highlight the selected row.

:::objective-c
- (void)navigationController:(UINavigationController *)navigationController didShowViewController:(UIViewController *)viewController animated:(BOOL)animated {
    [viewController viewDidAppear:animated];
}

- (void)navigationController:(UINavigationController *)navigationController willShowViewController:(UIViewController *)viewController animated:(BOOL)animated {
    [viewController viewWillAppear:animated];
}

Pushing onto the navigation stack

The final piece is to allow pushing onto the navigation stack from within one of the managed subviews.

Each subview’s UIViewController has an implicit navigationController property that you can use to send the pushViewController:animated: message to add an additional view controller to the navigation hierarchy.

In our case though, since each subview’s view controller has been instantiated outside of the navigation hierarchy, their navigationController reference will be nil.

The other observation is that we don’t actually want to push onto the navigation stack from within the subview – we want to push onto the navigation stack from the managing view controller.

The solution to this, is to pass the managing view controller to the subviews, to correctly allow pushing onto the navigation stack from within the subview. There’s several ways to do this, in the code above, I’ve defined a custom view controller initializer that accepts a managing view controller reference.

Conclusion

What I particularly like about this approach is that it separates the code and behaviour of each subview into separate view controllers, and assembles them together in a neat and compact manner.

Separate view controllers follow Apple’s single screen full of content per view controller paradigm, and pushing onto the navigation controller via the managing view controller yields a comfortable user experience.

An example XCode project of all this in action is also available if you’d like to step through the details, enjoy.

Updated

Example uploaded to github
Added pass of didReceiveMemoryWarning thanks to Jonah Williams

Scrolling with UIScrollView

2010-05-23T12:00Z

Just about every iPhone/iPad application needs them, scrollable views to let your users pan and/or zoom over more content than can be shown on one screen at a time.

There’s several approaches out there for making your content scrollable with a UIScrollView, some of them quite complex with overlapping views in Interface Builder, others recommending heavy use of code for layout.

The method I’ve found the quite effective of late in terms of code and ease of design in Interface Builder, is to add a UIScrollView to your UIViewController subclass' UIView, and create a separate UIView in your XIB, where the underlying full content can be added. Upon viewDidLoad, you can add this UIView to the UIScrollView’s subview property, and set the contentSize and zoom properties appropriately.

Here’s the steps:

Add UIScrollView to your UIViewController subclass' UIView

Code

Define the scroll view in your class interface and synthesize the property in the implementation.

:::objective-c
@interface MyViewController : UIViewController {
    UIScrollView * scrollView;
}

@property (nonatomic, retain) IBOutlet UIScrollView * scrollView;

@end

Interface Builder

Add the UIScrollView to your XIB, connecting it to your view controller via the outlet. The UIScrollView can be placed within any other decorations on the view such as navigation bars, tool/tab bars.

Create a separate UIView for your full content

Add an additional property for the view that will hold the content to be added to the scroll view.

Code

:::objective-c
@interface MyViewController : UIViewController {
    UIScrollView * scrollView;
    UIView       * contentView;
}

@property (nonatomic, retain) IBOutlet UIScrollView * scrollView;
@property (nonatomic, retain) IBOutlet UIView       * contentView;

@end

Interface Builder

Create the view in interface builder, and connect it your contentView outlet. This view doesn’t have to be within standard iPhone view dimensions, it can be of any size, since the scroll view will allow us to pan over it. Add all the content you wish the user to be able to pan over to this view.

Configure the UIScrollView to pan over your content

Add the contentView as a sub view of the scroll view, and set the contentSize property on the scroll view to be the bounds of the content view, or if you have any dynamic text/etc, calculate the height appropriately.

Code

:::objective-c
@synthesize scrollView, contentView;

- (void)viewDidLoad {
    [super viewDidLoad];

    // set the scrollview content and configure appropriately
    [self.scrollView addSubview:self.contentView];
    self.scrollView.contentSize = self.contentView.bounds.size;
}

Additionally, if you’d like pinch zooming, you can set the maximumZoomScale/minimumZoomScale properties, and viewForZoomingInScrollView: in your view controller as the UIScrollView delegate.

Also, let’s not forget to be good memory management citizens and release both the scrollview and contentview properties in viewDidUnload (iPhone 3.x), or didReceiveMemoryWarning (iPhone OS 2.x).

Code

:::objective-c
- (void)viewDidUnload {
    self.scrollView  = nil;
    self.contentView = nil;

    [super viewDidUnload];
}

Summary

I find this pattern useful for several reasons, mainly that it keeps the design and dimensions of content view separate from view that contains the UIScrollView, and there’s less complexity within the XIB.

It doesn’t force any Interface Builder pain with overlapping views or magic code to add scrolling to an existing view, and works well when you’d like to retrofit a UIScrollView into an existing XIB content.

The XCode project used to build this article and screenshots, etc, is also available.

Updated:

Thanks to Nathan de Vries for mentioning memory management.
Example uploaded to github

Getting Started with MacRuby

2009-09-01T12:00Z

For the those dual personality developers like myself who love Ruby, Rails and other awesome Ruby tools, but also get a big kick out of Mac and iPhone development, MacRuby is a really exciting project worth taking a look at. Originally a port of Ruby 1.9 to the Cocoa/Foundation frameworks under Mac OSX, MacRuby is now a fully fledged Ruby environment, with an LLVM based interpreter and DSL driven UI toolkit.

MacRuby is still under active development, and isn’t finished yet, but it’s Ruby and standard library compatibility is already quite impressive.

From the perspective of Ruby development, MacRuby can be used to build and run your Ruby and when supported Rails applications. From the perspective of a Mac applications developer, MacRuby can be used to build fully-fledged OSX desktop applications that interface with all the usual Mac development frameworks such as Core Data, Core Image and Cocoa, etc. User interfaces can be designed using Interface Builder, or using HotCocoa, a Ruby based DSL for describing Cocoa based UI’s.

Technically, MacRuby is implemented using the Objective-C common runtime and garbage collector, and the Core Foundation framework. This means that under MacRuby, Ruby objects are NSObjects (eg. a Ruby String is a NSString, likewise Ruby Hashes are NSDictionary’s, without any bindings or conversion required), Ruby classes are Objective-C classes, fully interoperable and interchangeable with each other, and instead of using the Ruby 1.9 garbage collector, the Objective-C 2.0 garbage collector is in full effect.

Most recently with MacRuby 0.5+ and the current development version of MacRuby, the YARV based interpreter has been removed and a new LLVM code generating interpreter has been implemented for performance and further optimisations down the track.

In this particular post I’ll work through the installation of MacRuby on your system, several follow up posts will discuss developing applications using MacRuby for Ruby and OSX desktop developers.

Installing

To get MacRuby running on your machine you currently need to build and install it from source. The build process is straightforward, but does take a bit of time. I’ll also assume you have the latest version of XCode installed. Here’s what you need to do.

Building LLVM

LLVM is required to build the latest MacRuby source, in particular a specific revision of LLVM for compatibility reasons, so even if you have it installed already as part of Snow Leopard or Ports, you might need to build it again. The particular revision required is 89156.

LLVM is hosted at http://www.llvm.org in Subversion, however personally I found it much easier to check out using a git mirror of the repository rather than attempt it from SVN (which for me took well over an hour just for the checkout that failed before completion resulting in a hosed source tree).

To checkout LLVM using git, and create a branch you can build, based off the right SVN revision number, perform the following commands:

$> git clone git://repo.or.cz/llvm.git
$> cd llvm
$> git checkout -b macruby-reliable 39a0f07ef82b6cc70ce87c038620921d87297ced
$> env UNIVERSAL=1 UNIVERSAL_ARCH="i386 x86_64" CC=/usr/bin/gcc CXX=/usr/bin/g++ ./configure --enable-bindings=none --enable-optimized --with-llvmgccdir=/tmp
$> env UNIVERSAL=1 UNIVERSAL_ARCH="i386 x86_64" CC=/usr/bin/gcc CXX=/usr/bin/g++ make
$> sudo env UNIVERSAL=1 UNIVERSAL_ARCH="i386 x86_64" CC=/usr/bin/gcc CXX=/usr/bin/g++ make install

Note that 39a0f07ef82b6cc70ce87c038620921d87297ced above corresponds to the git commit hash of svn commit 89156, you can tell this by looking at the output of git log and searching for 89156 in the git-svn-id section of the log message.

Alternatively, if you really need to check out llvm using Subversion, you can use the following command:

svn co -r 89156 https://llvm.org/svn/llvm*project/llvm/trunk llvm

Compiling and installing LLVM will take a while (45 minutes or more depending on your system), feel free to grab a cup of coffee – I had a nice Cappuccino:

2719.14 real      2429.54 user       183.13 sys

Building MacRuby

After installing LLVM you can now go ahead and build MacRuby itself. MacRuby now has an official git repository you can use to check out the source from. There is also an SVN repository however I’ll always favour git over Subversion:

$> git clone git://git.macruby.org/macruby/MacRuby.git
$> cd MacRuby
$> rake
....
/usr/bin/install -c -m 0755 libyaml.bundle /Library/Frameworks/MacRuby.framework/Versions/0.6/usr/lib/ruby/site_ruby/1.9.0/universal-darwin9.0
cd ext/fcntl
/usr/bin/make top_srcdir=../.. ruby="../../miniruby -I../.. -I../../lib" extout=../../.ext hdrdir=../../include arch_hdrdir=../../include install
/usr/bin/install -c -m 0755 fcntl.bundle /Library/Frameworks/MacRuby.framework/Versions/0.6/usr/lib/ruby/site_ruby/1.9.0/universal-darwin9.0
cd ext/zlib
/usr/bin/make top_srcdir=../.. ruby="../../miniruby -I../.. -I../../lib" extout=../../.ext hdrdir=../../include arch_hdrdir=../../include install
/usr/bin/install -c -m 0755 zlib.bundle /Library/Frameworks/MacRuby.framework/Versions/0.6/usr/lib/ruby/site_ruby/1.9.0/universal-darwin9.0
./miniruby instruby.rb --make="/usr/bin/make" --dest-dir="" --extout=".ext" --mflags="" --make-flags="" --data-mode=0644 --prog-mode=0755 --installed-list .installed.list --mantype="doc" --sym-dest-dir="/usr/local"
installing binary commands
installing command scripts
installing library scripts
installing headers
installing manpages
installing data files
installing extension objects
installing extension scripts
installing Xcode templates
installing Xcode 3.1 templates
installing samples
installing framework
installing IB support
$> sudo rake install

After installing you can test your MacRuby build by using the bundled spec suite:

$> rake spec:ci

Please note that this installs the latest version of MacRuby under development which is a moving target, however being on the bleeding edge means you can update to the latest source at any time with the latest features and fixes, and makes it much easier for contributing back to the project with patches, etc.

Later, if you’d like to update your installation, just return to the source directory, update your source from the git repository and reinstall.

$> git pull origin master
$> rake
$> sudo rake install

MacRuby Usage

Now that you have MacRuby installed, you can start using it. gems, ri, irb, etc, are all provided as part of your MacRuby installation with a mac* prefix so they don’t conflict with your existing MRI based installation (eg. macri, macirb, macgem, etc). In addition to this further interesting extensions such as XCode templates are included to get started building graphical Cocoa based apps using Interface Builder, etc.

$> macruby -v
MacRuby version 0.6 (ruby 1.9.0) [universal-darwin10.0, x86_64]

$> macirb
irb(main):001:0> puts "Hello World"
Hello World
=> nil
irb(main):002:0>

Summary

We’ve stepped through the installation of the latest version of MacRuby on your system, including it’s primary dependency LLVM, and described how to familiarise yourself with its environment. In future posts, I’ll write further about how to get started creating Cocoa applications using Mac OSX technologies such as XIBs, Bindings, Core Data, and Core Image with your apps, and also how to use HotCocoa, the nice and concise Ruby DSL for building OSX UI’s in Ruby.

If you can’t wait till then there’s a few further resources I’d recommend taking a look at:

The MacRuby site
The MacRuby Example Applications
Rich Kilmer’s MacRuby and HotCocoa presentation from the Ruby on OSX conference in Amsterdam

There’s also @macruby on Twitter where regular updates are posted, an IRC channel, and two mailing lists. I also follow the Github mirrors to see each commit that’s made to the MacRuby source, together with all the other projects I’m actively following.

Looking forward to writing more MacRuby posts in the future, enjoy MacRuby!

Updated to include newer LLVM revisions and MacRuby 0.6 release (25th May 2010)

RabbitMQ

2009-08-31T12:00Z

Last week I was privileged to present at our local Melbourne Ruby/Rails user group with fellow CLEAR Interactive colleague Daniel Neighman. Daniel and I gave a talk about RabbitMQ, the exciting AMQP based messaging platform.

We focused on discussing how RabbitMQ and AMQP came into existence and its architecture. I also showed a few demo applications I’d prepared, one a Rails application that used RabbitMQ to resize and process images via Core Image in the background, the other, a RubyCocoa Desktop client that posted surf report measurements to a fanout exchange that drove a video news feed of surfer quotes.

The slides for the presentation are available at slideshare.

The example applications I demonstrated during the talk are available as a GitHub project as well.

Big thanks to Nick Marfleet for organising and Square Circle Triangle for hosting the night, looking forward to next month already!

Comma, CSV for all

2009-03-10T12:00Z

CSV can be quite uninspiring at times, but as I’m sure many of you are all too familiar, many modern applications still require parsing and generation of CSV to interface with legacy systems and/or desktop software, notably Excel.

One of my Ruby on Rails clients required CSV data generation to support an ‘export to excel’ feature – so I embarked on a journey to look at the various CSV gems/plugins available at the time to export our data.

The result of this adventure gave birth to Comma, a small and simple (just over 60 lines implementation) gem that adds CSV generation support to arbitrary Ruby objects.

Using a declarative approach, you specify the output CSV format naming attributes, methods, associations, etc, all within a block with optional header names. Comma traverses these definitions to fetch model data, with conventions inferring headers when not specified using sensible defaults.

I had a few particular requirements while researching, which led to Comma’s development:

Support pure Ruby objects

I wanted to export arbitrary instances to CSV, not just ActiveRecord derived objects, and hence didn’t want to use a plugin specific to Rails, or one that had internal knowledge of ActiveRecord or similar models for inferring information such as associations and attributes.
Flexibility

Transparency across associations, attributes and methods – they should all be treated the same. Some of the plugins I looked at required different configuration to name methods or associations to use, as opposed to attributes. I wanted to be able to cleanly define where the data for export should come from, and have Comma transparently access to it (after all, Ruby’s #send mechanism provides the base foundations for this).
Multiple CSV output formats per class

One class we have requires several CSV output formats, one for delivery to end users, and another for escrow purposes. I wanted to be able to define multiple output formats per class, and be able to call upon them when required.
Integration

We’re using Ruby on Rails, so integration with Rails would be useful, particularly at the controller level, which should be DRY and able to ‘render :csv => @objects’.
Simplicity

CSV export shouldn’t be that hard on the plugin/gem implementer, nor the plugin/gem user – ideally I wanted to be able to define a CSV configuration (with an optional name) using a declarative syntax that names what should be exported, and have that same definition used for data access and header name generation.

An example use of Comma follows:

class Book < ActiveRecord::Base

  # ================
  # = Associations =
  # ================
  has_many   :pages
  has_one    :isbn
  belongs_to :publisher

  # ===============
  # = CSV support =
  # ===============
  comma do

    name
    description

    pages :size => 'Pages'
    publisher :name
    isbn :number_10 => 'ISBN-10', :number_13 => 'ISBN-13'
    blurb 'Summary'

  end

end

Annotated, the ‘Comma’ description includes:

# starts a Comma description block, generating 2 methods #to_comma and #to_comma_headers for this class.
comma do

  # name, description are attributes of Book with the header being reflected as 'Name', 'Description'
  name
  description

  # pages is an association returning an array, :size is called on the association results, with the header name specifed as 'Pages'
  pages :size => 'Pages'

  # publisher is an association returning an object, :name is called on the associated object, with the reflected header 'Name'
  publisher :name

  # isbn is an association returning an object, :number_10 and :number_13 are called on the object with the specified headers 'ISBN-10' and 'ISBN-13'
  isbn :number_10 => 'ISBN-10', :number_13 => 'ISBN-13'

  # blurb is an attribute of Book, with the header being specified directly as 'Summary'
  blurb 'Summary'

end

Notice above how attributes and associations are all specified and treated the same, header names are reflected from the method names using sensible conventions unless provided directly, and more complex combinations of data can be grouped together into methods if required.

Multiple descriptions can be specified with a named Comma block:

# ===============
# = CSV support =
# ===============
comma do  # implicitly named :default

  name
  description

  pages :size => 'Pages'
  publisher :name
  isbn :number_10 => 'ISBN-10', :number_13 => 'ISBN-13'
  blurb 'Summary'

end

comma :brief do

  name
  description
  blurb 'Summary'

end

You can specify which format you’d prefer as an optional parameter to #to_comma.

If you’re using Ruby on Rails, your controllers automatically gain Comma-fu.

class BooksController < ApplicationController

  def index
    respond_to do |format|
      format.csv { render :csv => Book.limited(50) }
    end
  end

end

Comma is licensed under the MIT License, and can be installed directly from github’s gem server.

sudo gem install crafterm-comma

Please feel free to contact me if you have any questions and/or feedback regarding Comma.

Distributed Image Processing with Airbrush

2008-11-02T11:00Z

Airbrush is a lightweight distributed processing tool, that Rails applications can use to communicate with and offload heavy processing of images, and/or other tasks while they continue processing requests.

Early this year, one of my clients started experiencing issues with their Rails application that managed the content for approximately 40 sites. The problem was in the area of image processing, with very large images, many several hundred megabytes in size that would be uploaded by the site’s administrators for publishing on various sorts of media.

When an image was uploaded several previews were being generated, and this process was bringing the system to a halt, consuming all resources of the Mongrel processing the request, to the point where the virtual server hosting that process would kill it off as a rampant process. Even testing some of the offending images would bring our MacBook Pro’s to a grinding halt, with memory use soaring into swap, causing everything to slow down to a snails pace.

We employed various tools for the platforms we were testing on, strace under Linux and later dtrace under Mac OS X. We noticed one example image, 10mb in size, would allocate 700mb of memory while it was being read by the image library (RMagick 2.x at the time). Image Science and even Quartz on my Mac OS X Tiger install exhibited similar behaviour.

After much research and testing, we found many of the offending images to be in non-RGB colour profiles, and to include all sorts of meta data (one even included an entire XML formatted Mac OSX plist file in its header). Installing profile management and pre-processing metadata alleviated much of the memory exhaustion pain.

The production environment consisted of many smaller Ubuntu Linux virtual private servers (~256mb ram, etc), so we decided to isolate the processing of images into a dedicated slice, so that could be shared across all the application servers, and could be configured to any specifications we required to handle the scale content being rendered.

This gave birth to Airbrush, which has been in use now for several months now with great success, and Square Circle Triangle, the client who paid for its development, has allowed us to open source Airbrush for all to use under the MIT license.

Airbrush was designed to abstract the three main roles in its architecture – the listening, the processing and the publishing of results from incoming jobs. This was done primarily to allow us to provide any style of access to Airbrush’s services now and in the future (eg. a queuing system, webservice, etc), and to allow processing of any job type, not just related to image processing (eg. bulk emailing, report generation, etc).

Around the time Airbrush was architected, Starling, a memcache derived persistent queue implementation was released, and became the perfect fit for Airbrush’s first listener implementation.

To get up and running with Airbrush, first install the following gems:

$> gem install starling airbrush rmagick

Then, create a memcache queue using Starling (specifying the queue and pid file locations):

$> starling -q /var/tmp/starling -P /var/tmp/starling.pid
I, [2008-11-02T11:58:13.443012 #77820]  INFO -- : Starling STARTUP on 127.0.0.1:22122

Then, start any number of Airbrush server instances:

$> airbrush -v
Sun Nov 02 11:58:22 +1100 2008: Accepting incoming jobs

‘v’ indicates verbose operation, so that you receive extra logging information. Several other options can be passed to Airbrush such as the memcache server location and port, job poll frequency and a log target, run ‘airbrush -h’ for further details. Both Starling and Airbrush’s default to the localhost as the memcache server on port 22122, so if you are running both Starling and Airbrush locally the defaults will be fine.

To send a preview request to an Airbrush server, you can use the example airbrush-example-client command included with the Airbrush gem:

$> airbrush-example-client -i leaves_desktop.jpg -o resized
Sending leaves_desktop.jpg for preview processing

This sends the image ‘leaves_desktop.jpg’ to be resized into two smaller previews, with filenames starting with ‘resized’.

Back on the Airbrush server side, you’ll notice some further logging output when this happens:

Sun Nov 02 12:02:16 +1100 2008: Processing generate-previews
Sun Nov 02 12:02:18 +1100 2008: Processed previews ({:filename=>"leaves_desktop.jpg", :sizes=>{:small=>[300], :large=>[600]}, :image=>"[FILTERED]"})
Sun Nov 02 12:02:18 +1100 2008: Published results from generate-previews
Sun Nov 02 12:02:18 +1100 2008: Processed generate-previews: 0.806346 seconds processing time

Indicating a successful job. Should an error occur, both the client and server will report what happened.

Internally, the API request to create several image previews is:

<< client = Airbrush::Client.new(memcache_host) client.process( ‘generate-previews’, :previews,

:image => File.read(OPTIONS[:image]), 
:sizes => { :small => [300], :large => [600] } )

This creates an instance of the Airbrush client, and instructs it to process a given job via the #process method. The parameters to #process specify a unique job id (also used as the return memcache queue name for any results the job may provide), the job name (:previews in this case), and arguments to the job (two sizes in this example, for the creation of ‘small’ and ‘large’ previews, with a longest edge of 300 and 600 pixels respectively). Airbrush will calculate the resultant dimensions using the aspect ratio of the image.

An Airbrush server reads this job request from the Starling memcache queue, processes it and places the results back on the queue for the client to read either immediately, or at a later time.

Processing is a simple Ruby class with method names matching job names, here’s an example taken from Airbrush’s RMagick based image processor with the resize and previews job implementations:

<< module Airbrush module Processors

module Image
  class Rmagick < ImageProcessor
    filter_params :image # ignore any argument called 'image' in any logging

    def resize(image, width, height = nil)
      width, height = calculate_dimensions(image, width) unless height

      process image do
        change_geometry("#{width}x#{height}") { |cols, rows, image| image.resize!(cols, rows) }
      end
    end

    def previews(image, sizes) # sizes => { :small => [200,100], :medium => [400,200], :large => [600,300] }
      sizes.inject(Hash.new) { |m, (k, v)| m[k] = crop_resize(image, *v); m }
    end

    # ... snip ...

  end
end

end end

The parameters to each method are extracted from the options has passed in as arguments to the named job (similar to how Merb can extract values from the params[] hash automatically).

The current RMagick based image processor supports resizing, cropping and multiple preview generation, with images in RGB and CMYK colour profiles. ‘before’ and ‘after’ filters are also supported to pre or post process images (eg to add a watermark or filter out metadata amongst other things) as well. A Core Image based processor is also available.

The distributed nature of memcache allows us to daisy chain as many airbrush servers as we’d like to handle anticipated load, and we can spread this across any number of VPS or real servers as we’d like (even other platforms such as X-Serve’s with dedicated video processing hardware). Another added benefit of using Starling to handle the incoming job queue is that Airbrush servers can be increased/decreased without affecting the queue reliability – even when no Airbrush servers are running, jobs will simply be added to the Starling queue and wait until one is started.

Airbrush is available as a gem on GitHub and Rubyforge, and is under the MIT license. We’re excited to see it released, and keen to continue its development. Please also feel free to contact us with any suggestions or ideas to make Airbrush more useful for everyone.

Sprinkle Update!

2008-08-03T12:00Z

Sprinkle, the new provisioning tool I recently released has been progressing really well over the past few months. Its been great to see the focus of Sprinkle to succinctly describe software installation via an elegant Ruby DSL language improve so well.

Last Thursday at our monthly Melbourne Ruby User Group meetup, I had the pleasure of demonstrating Sprinkle to the local community. I’ve uploaded the slides for those interested:

Gem packages for Sprinkle are available from both GitHub and Rubyforge, and include several new features that have been added sine my original post about the project.

Deployment/command delivery

Support for Vlad in addition to Capistrano, allowing you to leverage upon your existing Vlad configuration
Direct or gateway tunneled net/ssh connection support, allowing you to configure Sprinkle to communicate with remote hosts via net/ssh directly if you desire
Support for provisioning your local machine using Sprinkle by running any commands locally

Installers

Support for automatically verifying installations pre- and post-install to detect any installation errors, and also to optimize the installation order to skip packages already installed.
Support for installing build dependencies of APT packages
Support for installing RPM packages
Support for installing GEM packages from a specific remote repository (eg. github) and into a specific local repository

Documentation

Full RDoc coverage across all classes, also published online
New blog posts & screencast demonstrating Sprinkle

Examples

Many new examples have been contributed showing several ways of using Sprinkle to provision packages from Git, Ruby, Rails, Sphinx, through to Phusion, etc.

Plus many other contributions from people around the world helping with specs, bugs and other improvements which has been great.

I’d really like to thank everyone who has jumped in and helped with the project in any way over the past few months, I really appreciate your support!

Sprinkle Some Powder!

2008-05-27T12:00Z

Provisioning a brand new server or VPS slice can be quite tricky, tedious and time consuming, particularly if done manually with changing software versions and configurations. In the Rails world, most of us are using virtual private servers which are instantiated from base operating system images, it takes only a few minutes to create a slice, however installing the rest of your server’s stack, be it Rails, Merb or another framework is where the work begins. Provisioning in this sense, is installing all software required post operating system install.

Several free and commercially available tools already exist to help automate the installation of software however most fall into two styles of design:

Task based, where the tool issues a list of commands to run on the remote system, either remotely via a network connection or smart client.
Policy/state based, where the tool determines what needs to be run on the remote system by examining its current and final state.

Task based solutions are usually quite easy and fast to get up and running, but can be problematic as the user has to define all of the commands manually (not to mention get them right with testing). Policy/state based solutions have much more intelligence about how to modify and adapt the remote system, but often require specialized software to run remotely.

Sprinkle is a prototype tool I’ve been working on recently in this space that merges both concepts together, using a Ruby domain specific language to declaratively describe the state of the remote system. Using Sprinkle, provisioning your brand new remote server or slice can be automated using pre-defined and/or customized scripts from a single machine at your fingertips.

Sprinkle reads a script that defines a set of packages, a set of policies that define what packages should be installed on what roles of target machines, and a deployment section that defines the delivery mechanism for communicating with remote machines, and any default settings.

Packages can have relationships between each other to support dependencies. Virtual packages are also supported allowing you to define a role that a package (or multiple) fulfills, with the user or Sprinkle selecting which concrete package should be used at runtime. Packages can also support arbitrary installer types, allowing you to install packages from source, gems, apt, or any other installer you’d like to employ. Installer types know what commands need to be issued to install packages, so all that needs to be specified in a script is the installer type and metadata about the package itself.

Policies allow you to group sets of packages together, and specify a target role of systems the policy is applicable for. This is useful for example when multiple packages make up the complete installation of a stack (eg. Rails requires the rails gem, but it also requires ruby, an application server, a web server, and potentially a database).

In essence, Sprinkle is about defining a domain specific meta-language for describing and processing the installation of software.

Example Sprinkle Script

Here’s an example Sprinkle deployment script, annotated to explain each section:

# Annotated Example Sprinkle Rails deployment script
#
# This is an example Sprinkle script configured to install Rails from gems, Apache, Ruby and
# Sphinx from source, and mysql from apt on an Ubuntu system.
#
# Installation is configured to run via capistrano (and an accompanying deploy.rb recipe script).
# Source based packages are downloaded and built into /usr/local on the remote system.
#
# A sprinkle script is separated into 3 different sections. Packages, policies and deployment:


# Packages (separate files for brevity)
#
#  Defines the world of packages as we know it. Each package has a name and
#  set of metadata including its installer type (eg. apt, source, gem, etc). Packages can have
#  relationships to each other via dependencies.

require 'packages/essential'
require 'packages/rails'
require 'packages/database'
require 'packages/server'
require 'packages/search'


# Policies
#
#  Names a group of packages (optionally with versions) that apply to a particular set of roles:
#
#   Associates the rails policy to the application servers. Contains rails, and surrounding
#   packages. Note, appserver, database, webserver and search are all virtual packages defined above.
#   If there's only one implementation of a virtual package, it's selected automatically, otherwise
#   the user is requested to select which one to use.

policy :rails, :roles => :app do
  requires :rails, :version => '2.0.2'
  requires :appserver
  requires :database
  requires :webserver
  requires :search
end


# Deployment
#
#  Defines script wide settings such as a delivery mechanism for executing commands on the target
#  system (eg. capistrano), and installer defaults (eg. build locations, etc):
#
#   Configures sprinkle to use capistrano for delivery of commands to the remote machines (via
#   the named 'deploy' recipe). Also configures 'source' installer defaults to put package gear
#   in /usr/local

deployment do

  # mechanism for deployment
  delivery :capistrano do
    recipes 'deploy'
  end

  # source based package installer defaults
  source do
    prefix   '/usr/local'
    archives '/usr/local/sources'
    builds   '/usr/local/build'
  end

end

# End of script, given the above information, Spinkle will apply the defined policy on all roles using the
# deployment settings specified.

Given such a script, Sprinkle will apply the specified policy rails to the target machines identified by the role app, where the policy rails is composed of packages for the rails gem itself, an application server, webserver, search daemon, and the ruby runtime.

Currently, Sprinkle uses Capistrano internally for communicating with remote systems, however this is pluggable as well, allowing for just about any concievable delivery mechanism in the future. The deployment section above identifies Capistrano as the delivery mechanism, specifying a local deploy.rb script that defines what roles are available, and what machines are defined within those roles.

This particular script breaks the package section up into multiple files, here are some of the actual package definitions (complete example available here):

package :ruby do
  description 'Ruby Virtual Machine'
  version '1.8.6'
  source "ftp://ftp.ruby-lang.org/pub/ruby/1.8/ruby-#{version}-p111.tar.gz" # implicit :style => :gnu
  requires :ruby_dependencies
end

package :rubygems do
  description 'Ruby Gems Package Management System'
  version '1.0.1'
  source "http://rubyforge.org/frs/download.php/29548/rubygems-#{version}.tgz" do
    custom_install 'ruby setup.rb'
  end
  requires :ruby
end

package :rails do
  description 'Ruby on Rails'
  gem 'rails'
  version '2.0.2'
end

package :sphinx, :provides => :search do
  description 'MySQL full text search engine'
  version '0.9.8-rc2'
  source "http://www.sphinxsearch.com/downloads/sphinx-#{version}.tar.gz"
  requires :mysql_dev
end

package :apache, :provides => :webserver do
  description 'Apache 2 HTTP Server'
  version '2.2.6'
  source "http://apache.wildit.net.au/httpd/httpd-#{version}.tar.bz2" do
    enable %w( mods-shared=all proxy proxy-balancer proxy-http rewrite cache headers ssl deflate so )
    prefix "/opt/local/apache2-#{version}"
    post :install, 'install -m 755 support/apachectl /etc/init.d/apache2', 'update-rc.d -f apache2 defaults'
  end
  requires :apache_dependencies
end

Each package includes a description, optional version, optional list of dependencies and an installer type (also optional allowing for meta-packages).

Source installers are particularly intelligent and will download, configure and install source archives from a remote location directly on the target machine. They assume GNU style source archives by default (ie. tar.gz/tar.bz2 compressed archives, configure script and make, make install style semantics), however are completely customziable to support any arbitrary build style (rubygems for example does this above), with pre and post commands.

The Apache installer for example, specifies a few extra source installer options such as a set of —enable options, an alternate installation prefix and a series of post installation commands to be executed.

With this example configuration, lets take a look at actually using Sprinkle to provision a remote server.

Usage

Sprinkle supports several command line options:

Usage
=====

$> sprinkle [options]

Options are:

  -s, --script=PATH                Path to a sprinkle script to run
  -t, --test                       Process but don't perform any actions
  -v, --verbose                    Verbose output
  -c, --cloud                      Show powder cloud, ie. package hierarchy and installation order
  -h, --help                       Show this help message.

where you can name the script to be processed, enable testing mode or verbose output, and/or examine the cloud of packages and operations that will be performed.

Viewing the powder cloud!

Sprinkle calculates all operations to be performed on remote servers upfront which is nice, as it allows you to inspect what modifications will be made to the system before any are actually performed. Lets inspect the powder (ie. package) cloud for the above script:

$> sprinkle -c -t -s rails.rb
--> Cloud hierarchy for policy rails

Policy rails requires package rails
        Package rails requires rubygems
                Package rubygems requires build_essential
                Package rubygems requires ruby
                        Package ruby requires build_essential
                        Package ruby requires ruby_dependencies

Policy rails requires package appserver
Selecting mongrel_cluster for virtual package appserver
        Package mongrel_cluster requires rubygems
                Package rubygems requires build_essential
                Package rubygems requires ruby
                        Package ruby requires build_essential
                        Package ruby requires ruby_dependencies
        Package mongrel_cluster requires mongrel
                Package mongrel requires rubygems
                        Package rubygems requires build_essential
                        Package rubygems requires ruby
                                Package ruby requires build_essential
                                Package ruby requires ruby_dependencies

Policy rails requires package database
Selecting mysql for virtual package database

Policy rails requires package webserver
Selecting apache for virtual package webserver
        Package apache requires build_essential
        Package apache requires apache_dependencies

Policy rails requires package search
Selecting sphinx for virtual package search
        Package sphinx requires build_essential
        Package sphinx requires mysql_dev

--> Normalized installation order for all packages: build_essential, ruby_dependencies, ruby, rubygems, rails, mongrel, mongrel_cluster, mysql, apache_dependencies, apache, mysql_dev, sphinx

-c indicates that Sprinkle should print the powder cloud (ie. the output above) -t indicates that we’re operating in test mode, so we won’t actually perform any remote commands -s identifies the Sprinkle script that should be processed

Above we can see that the policy rails required packages rails, appserver, database, webserver and search. Note that all of these packages bar rails are actually virtual packages, so Sprinkle has selected an appropriate implementation of each virtual package automatically based on the supplied package definitions. If more than one package provided an implementation of a virtual package, then the user would be given the opportunity to select which one they prefer.

Under each package is a textual representation of that package’s dependency tree, including all sub-dependencies, etc. Dependencies are packages that need to be installed first before a higher level package can be installed. You’ll notice that several packages have the same dependencies, eg. both rails and mongrel require ruby, which has its own dependencies as well. Sprinkle will install all packages in reverse dependency order so that lower level dependencies are installed before higher level packages, and it will also normalize the final package list to remove duplicates so that packages aren’t installed multiple times unnecessarily. This is the final line in the output above which lists the actual packages to be installed and order of installation.

Provising a remote system

To actually provision a remote server we simply remove the testing (and if desired cloud) flags from the command issued above and Sprinkle will process the configuration and provision the remote system. Note for the moment, you’ll need to ensure that your SSH keys are appropriately installed on the remote server under a user that has enough privileges to install software (generally the root user):

$> sprinkle -s rails.rb
--> Installing build_essential for roles: app
--> Installing ruby_dependencies for roles: app
--> Installing ruby for roles: app
--> Installing rubygems for roles: app
--> Installing rails for roles: app
--> Installing mongrel for roles: app
--> Installing mongrel_cluster for roles: app
--> Installing mysql for roles: app
--> Installing apache_dependencies for roles: app
--> Installing apache for roles: app
--> Installing mysql_dev for roles: app
--> Installing sphinx for roles: app

After the command is finished, all of the requested software will have been applied on your target system.

If you’d like to see more action printed as commands are run, specify the —verbose (-v) flag. Internally, Capistrano tasks are dynamically defined and executed at runtime for each package’s installation, using a Capistrano configuration file to identify the actual roles and hostnames associated with those roles to communicate with. The verbose option will display Capistrano activity in addition to the usual Sprinkle output.

An extra benefit of leveraging Capistrano is that you can actually provision multiple servers/slices simultaneously and in parallel if desired.

I want!

If you’re interested in downloading and experimenting with Sprinkle, you can clone and/or watch the project at GitHub, or download it from GitHub’s gem server using:

$> sudo gem install crafterm-sprinkle --source http://gems.github.com/

The official Rubyforge gem server will also be updated over the coming days as well. If you download the source, you can create a gem package for installation by:

$> rake package
$> sudo gem install -l pkg/sprinkle-0.1.0

Prerequsites

Installing the Sprinkle gem will also install all pre-requisite gems such as active support, highline and capistrano. The only other pre-requisite is that you have SSH connectivity to the remote system you wish to provision, preferably with SSH keys in place to prevent passwords being asked for.

Summary

Sprinkle is a new prototype tool that you can use to provision your servers/slices. Its declarative policy/state based approach for specifying how a remote system should be provisioned with intelligent logic to support dependencies, multiple installer types and remote installation is quite compelling. Sprinkle is a young project and while operational still in development, with several limitations. Currently, only Ubuntu/Debian has been tested as a target deployment platform, operating system abstraction and other platforms will be tested and supported in the future, along with several new features that are in the pipeline.

I’m most certainly open to ideas, suggestions and thoughts about how Sprinkle can be improved and generally made better for the community, and I really welcome any bug reports, patches and suggestions. Please feel free to contact me with any comments at all.

Twilight Theme for Emacs!

2008-05-27T12:00Z

Recently I’ve been returning to my Unix heritage and using Emacs as my day to day editor. Its been quite an enlightening journey returning to such a powerful editor with its limitless customization and features (probably worthy of a whole separate post), although one things I do miss from TextMate are the colour themes used for syntax highlighting. Emacs has a package called color-theme which can be used to customize font faces/colours however in comparison to the styled themes TextMate offers, most of default ones available don’t quite compare.

In TextMate I used the Twilight theme quite often, so, after examining the color-theme package format, I’ve ported across the Twilight theme to Emacs, and have released it as a small project on GitHub. Its definitely a work in progress, most of the colours within Ruby mode are working as expected, but please feel free to improve anything that needs some fine tuning (eg. Rails keywords, and particularly non-Ruby modes such as dired, ERB, SCM, etc). Patches are most certainly welcome.

Ruby && DTrace!

2008-05-18T12:00Z

Performance, memory and runtime analysis of software has always been a tricky subject, often requiring special debug versions of code or application specific parameters to determine what’s going on. Additionally, developer debugging information can often clutter source code making it harder to see the intent and design of source.

D-Trace offers an interesting solution to this problem, by dynamically instrumenting your application at runtime to enable probes to report various pieces of information as to how your application is running. DTrace is a part of Solaris, but is now also available under Mac OS X Leopard. A typical installation of DTrace can offer 20,000 different types of probes (or more depending on what applications are running), from kernel level information, all the way to application specific data.

Applications, such as the Ruby interpreter can also define their own domain specific probes, last year Joyent added support for D-Trace probes to MRI (Matz-Ruby), allowing developers to analyze runtime behaviour of their Ruby based applications. Starting with this particular commit, I’ve also started adding compatible D-Trace probes to Rubinius.

What can you do with DTrace?

The list of uses for DTrace are endless, as it provides the means to gain answers to many questions about how your application is behaving. Some practical questions DTrace can answer include:

Tracing execution flow through your application as it steps through each method/class
Determining runtime performance analysis, working out what methods are the most expensive (excellent for 80/20 performance analysis)
Heap analysis, determining what objects are consuming the most memory
Garbage collection impact, determining how often the garbage collector is running an impacting your applications performance
and much more…

Getting started with DTrace and Ruby

To get started with Ruby and DTrace, you’ll need a compatible operating system, eg. Mac OS X Leopard, and you’ll need to get the Ruby source to build MRI with Joyent’s DTrace patches. Luckily, if you’re using Mac OS X Leopard, Apple has already appropriately patched their bundled version of Ruby to include DTrace patches for you, and no extra compilation is necessary. DTrace itself is also included by default under all Mac OS X Leopard operating system installs.

DTrace primer

There is a wealth of information available on the internet that I’d certainly recommend taking a look at to learn using DTrace in depth (in particular Sun’s DTrace admin guides). Essentially to interact with DTrace, you write a script in a language called ’D', which defines what probes you’re interested in, and what to do with the data when the probe fires. This script is then read and bytecode compiled by DTrace’s command line and user land libraries, and then passed to the DTrace virtual machine running inside of the kernel to be interpreted. Probes are enabled, and appropriately fired, with data being collected according to your scripts for analysis.

Anatomy of a DTrace script

provider : module : function : name
/ predicate /
{
  action
}

Above is generic breakdown of a DTrace script (parts of it bearing similarity to an awk script). Most parts of the script are optional as you’ll see further, such as the module or function name, or even the action.

Probes are grouped by providers, of which there are many (io, pid, objc, profile, to name a few). module and function’s meaning are somewhat dependant on the provider being used. The name parameter identifies the actual name of the probe that is to be fired.

The predicate is identifies a clause that must evaluate to true for the probe to fire and allows for conditional firing of probes.

The action contains arbitrary instructions to be performed when the probe fires.

To probe your application you also need root privileges on the machine you’ll be running DTrace on, this is due to kernel level interaction of DTrace. Usually you’ll run DTrace and pass it the name of a D script (or include the script on the command line if it’s brief), either with a command to run, or a process ID of an application that’s already running that should be attached to. For example:

$> sudo dtrace -s profile.d -c 'ruby -e "puts 'gday australia!'"'

or:

$> ps aux|grep ruby
crafterm 29877   0.0  0.0   590472    188 s001  R+    5:04pm   0:00.00 grep ruby
crafterm 29875   0.0  1.2   622564  25016 s001  S     5:04pm   0:00.85 ruby
$> sudo dtrace -s profile.d 29875

Ruby Provider Probes

To see how many probes and what Ruby probes are available, we can ask DTrace to print their specifics to the console, eg:

$> sudo dtrace -l | wc -l
31569

indicating I currently have 31569 probes available to query, this number will change depending on what applications are running at the time of running the dtrace command. Ruby specific probes can be found by:

$> sudo dtrace -l | grep ruby
21427  ruby53816   libruby.1.dylib                          rb_call0 function-entry
21428  ruby53816   libruby.1.dylib                          rb_call0 function-return
21429  ruby53816   libruby.1.dylib                   garbage_collect gc-begin
21430  ruby53816   libruby.1.dylib                   garbage_collect gc-end
21431  ruby53816   libruby.1.dylib                           rb_eval line
21432  ruby53816   libruby.1.dylib                      rb_obj_alloc object-create-done
21433  ruby53816   libruby.1.dylib                      rb_obj_alloc object-create-start
21434  ruby53816   libruby.1.dylib                   garbage_collect object-free
21435  ruby53816   libruby.1.dylib                        rb_longjmp raise
21436  ruby53816   libruby.1.dylib                           rb_eval rescue
21437  ruby53816   libruby.1.dylib                 ruby_dtrace_probe ruby-probe

As you can see from the list in the last column of the output, probes are available between method invocations, runs of the garbage collector, creation and destruction of objects and exceptions. The last probe actually allows the application writer to fire an arbitrary ruby probe containing application specific data from ruby code. A full list of probes and arguments supplied to them is available at Joyent’s Ruby provider wiki page.

Example 1: Tracing execution flow through your application

Lets start by tracing the execution flow through a small Ruby program.

Simple Ruby Program

class World
  def say(message)
    puts message
  end
end

world = World.new
world.say('hello')

This small Ruby program creates an instance of the World class, sends it the say message with hello as a String parameter which is printed to the console.

Execution Flow D script

ruby$target:::function-entry
{
   printf("%s:%s\n", copyinstr(arg0), copyinstr(arg1));
}

ruby$target:::function-return
{
   printf("%s:%s\n", copyinstr(arg0), copyinstr(arg1));
}

This particular script enables the function-entry probe on the Ruby provider, and prints the first and second arguments passed to the probe in a C-style printf command. The first and second arguments passed are provider specific, but in the Ruby provider’s case for these probes they are always the class/module and method names being executed. The $target variable enables the probe on the PID of the command specified via the -c parameter to DTrace itself.

Results

$> sudo dtrace -q -F -s execution-flow.d -c "ruby hello.rb"
CPU FUNCTION
1  -> rb_call0                              Class:inherited
1  <- rb_call0                              Class:inherited
1  -> rb_call0                              Module:method_added
1  <- rb_call0                              Module:method_added
1  -> rb_call0                              Class:new
1    -> rb_call0                            Object:initialize
1    <- rb_call0                            Object:initialize
1  <- rb_call0                              Class:new
1  -> rb_call0                              World:say
1    -> rb_call0                            Object:puts
1      -> rb_call0                          IO:write
1      <- rb_call0                          IO:write
1      -> rb_call0                          IO:write
1      <- rb_call0                          IO:write
1    <- rb_call0                            Object:puts
1  <- rb_call0                              World:say

Taking a look at the results we can almost visually ‘see’ how the script was parsed and executed. First Class was ‘inherited’, this is part of the creation of our ‘World’ class, then we defined World#say (invoking the Module:method_added method) to contain some operations. We then created a new instance of our class (Class:new and Object:initialize to create and construct our object), and then invoked World#say which we can see calls Object:puts and IO:write.

(In this particular case, the program is small and the instructions simple, however just add “require ‘rubygems’” to the top of the source and re-run the DTrace script again and you’ll quickly be overwhelmed with too much information – writing effective DTrace scripts is an art, but it’s well worth learning to ensure you get the answers you’re looking for)

Example 2: Individual Method Performance

Lets use DTrace to take another look at our application from a different perspective and see what methods are most expensive. To do this we’ll use the function entry and return probes to capture a time stamp interval for each method call. We’ll also use an aggregate DTrace variable to store a running average of how long each method takes so that multiple method calls are recorded together and averaged across the count of method invocations, and we’ll print the results according to most expensive method execution time.

Method Performance DTrace script

ruby$target:::function-entry
{
  self->start = timestamp;
}

ruby$target:::function-return
/self->start/
{
  @[copyinstr(arg0), copyinstr(arg1)] = avg(timestamp - self->start);
  self->start = 0;
}

This script introduces a few more DTrace constructs, associative arrays and aggregate functions.

We enable two probes, function entry and function return on our Ruby program. When any method is entered, we capture a timestamp and store it in the ‘start’ variable. When any method is exited, we gather another timestamp, subtract it from the entry, and pass it to the avg DTrace aggregate function to be averaged. The average execution time is then stored in an associative array, indexed by the module/class and method name (arg0 and arg1 respectively). Finally, we reset ‘start’ to zero once its no longer required. A predicate is also set on the function return probe to only fire if we have a start timestamp value, which prevents us from seeing any errors if we attach our DTrace script to an application that is already running (since after attaching, a return probe could fire for which we have no start timestamp).

Results

$> sudo dtrace -s timestamps.d -c "ruby hello.rb"
Object                   initialize                     9185
Module                   method_added                   10021
Class                    inherited                      25323
IO                       write                          98956

DTrace automatically prints data collected unless you supply a custom output format (eg. by using the C-style printf DTrace function as in the method flow example above) which in this case is the associative array, indexed by class/module and method name. From these results we can see that IO#write was the slowest of the methods called, taking an average of 98956 nanaseconds to run, most likely due to its interation with the rest of the system and IO nature.

Example 3: Quantized Method Performance

Average values can often be affected by a few large values during program startup, so lets take a closer look and see what values consititue the calculation the averages above. To do this we’ll use the DTrace quantize aggregate function, which will provide us with a distribution breakdown of each individual component within the average. We’ll also specifically target the IO#write method, and update our Ruby program to print ‘hello’ 10 times to collect some more data over a period of invocations.

ruby$target:::function-entry
/copyinstr(arg0) == "IO" && copyinstr(arg1) == "write"/
{
  self->start = timestamp;
}

ruby$target:::function-return
/copyinstr(arg0) == "IO" && copyinstr(arg1) == "write" && self->start/
{
  @[copyinstr(arg0), copyinstr(arg1)] = quantize(timestamp - self->start);
  self->start = 0;
}

Results

$> sudo dtrace -s timestamps-q.d -c "ruby hello.rb"

  IO                                                  write
           value  ------------- Distribution ------------- count
            4096 |                                         0
            8192 |@@@@@@@@@@@@@@@@@@@@                     10
           16384 |@@@@@@@@@@@@@@@@                         8
           32768 |@@                                       1
           65536 |                                         0
          131072 |@@                                       1
          262144 |                                         0

Here we see the distribution of how long each invocation of IO#write took. There was one invocation that particuarly long, over 131k nanoseconds, with most landing between 8k and 16k nanoseconds. From here we could step further into the runtime and determine which particular IO#write call was slower than the others by inspecting the user stack inside the virtual machine, and/or by enabling probes in lower level providers.

Example 4: Memory allocation

To profile memory allocation we need to enable the object-create-start, object-create-done, and object-free probes. Creation of objects is separated into two probes to allow you to determine exactly how long it takes to construct an object.

First, lets create a simple balance script to check that objects are being allocated and deallocated correctly within the Ruby runtime. The script we’ll use will create an associative array and index a counter by object type. Each time an object is created we’ll increment the counter, conversely each time an object is freed we’ll decrement the counter.

ruby$target:::object-create-done { @[copyinstr(arg0)] = sum(1); }
ruby$target:::object-free        { @[copyinstr(arg0)] = sum(-1); }

Results

$> sudo dtrace -s object-balance.d -c "ruby hello.rb"

  NoMemoryError                                                     1
  SystemStackError                                                  1
  ThreadGroup                                                       1
  World                                                             1
  fatal                                                             1
  Object                                                            3

Glancing over the results we can see that Ruby is creating an instance of several error classes and a thread group during the run of our application (perhaps during startup), we can also see a single instance of our World class, and three other Objects that have also been allocated. Our particular hello.rb script is quite small, and probably finishes executing before the garbage collector has had a change to reclaim any unused objects. If you run this script over a large application though, you’ll see a line for each Object in the application, and essentially a reference count of how many have been created and freed. In an ideal application, after garbarge collector has finished, all objects (except those required to keep the application running) types will be listed with the value ‘0’ alongside it indicating a corresponding deallocation for each allocation.

Example 5: Inspecting memory allocation points

The object-create-start/done probes also provide the source file and line number of where the allocation was made in Ruby script which we can use. For example if our World class came from another developer’s library and we wanted to find out where it was allocated we could use the following script:

ruby$target:::object-create-start
/copyinstr(arg0) == "World"/
{
  printf("%s was allocated in file %s, line number %d\n", copyinstr(arg0), copyinstr(arg1), arg2);
}

Results

$> sudo dtrace -s object-balance.d -c "ruby hello.rb"

  World was allocated in file hello.rb, line number 7

which matches our source file.

Example 6: Inspecting stack traces

We also saw above that several other ‘Object’s are created within the C portion of the Ruby interpreter upon startup. We can also inspect where these objects were created by saving the user C stack inside the virtual machine at the point of allocation.

ruby$target:::object-create-start
/copyinstr(arg0) == "Object"/
{
  @[ustack(4)] = count();
}

This particular script uses the DTrace ustack function to access the actual runtime stack of the Ruby interpreter in userland at the point in time when the probe fired. We then use the stack as an index into an associative array and store the number of times an Object type was created at the same point in the interpreter. This example really shows how flexible associative arrays can be in DTrace, by using a full strack trace as an index.

Results

$> sudo dtrace -s object-user-stack.d -c "ruby hello.rb"

          libruby.1.dylib`rb_obj_alloc+0x90
          libruby.1.dylib`Init_Object+0x130b
          libruby.1.dylib`rb_call_inits+0x15
          libruby.1.dylib`ruby_init+0x14f
            1

          libruby.1.dylib`rb_obj_alloc+0x90
          libruby.1.dylib`Init_IO+0x1059
          libruby.1.dylib`rb_call_inits+0x6a
          libruby.1.dylib`ruby_init+0x14f
            1

          libruby.1.dylib`rb_obj_alloc+0x90
          libruby.1.dylib`Init_Hash+0x903
          libruby.1.dylib`rb_call_inits+0x51
          libruby.1.dylib`ruby_init+0x14f
            1

Here we can see three separate stack traces indicating a Hash, IO and Object being created as part of the ruby_init method inside the Ruby interpreter.

Summary

DTrace is a very powerful framework, allowing you to really hypothesise and ask arbitrary questions about the behaviour of your system and applications. Often, the answer to one question will lead to another, and this is very much in the sprit of DTrace. The ability to script questions and format results allows you to slice behavioural data from any perspective and depth from your application, all the way to the operating system kernel.

DTrace scripts are the key to reducing complexity and understanding the true behaviour of your application at runtime, and I certainly recommend learning as much about the D script format and language as you can. Fine tuning your script to return the exact data you’re after can be an art, but its well worth learning so that you can specify exactly what data you are after, and not be cluttered with too much information obscuring the information you are searching for.

Collections of commonly used DTrace scripts are available as part of the DTraceToolkit, in particular several very useful and high quality Ruby DTrace scripts. I also recommend taking a look at them to see how probes can be used in combination with each other, also in multi-threaded environments.

In future articles I’ll step further into using DTrace via Instruments, and also look at instrumenting your Rails or Merb application to collect runtime data about the performance of your web applications.

DTrace at Cocoaheads

2008-05-09T12:00Z

Last night we had our monthly Melbourne Cocoaheads meetup which was great. There were several talks given including one about the AUC, and I also gave a talk about DTrace, the popular dynamic tracing framework for Mac OS X Leopard and OpenSolaris.

I’ve uploaded the slides from my talk to slideshare for those interested – since most of the talk was a demo there are only a few slides included, however the links towards the end of the presentation provide some good references to more information about DTrace, the D language and using DTrace on your system.

Ruby Cocoa & Core Image at Cocoaheads

2008-04-11T12:00Z

Last night saw our first Melbourne Cocoaheads meet up for the year. An awesome night was had by all, there were several talks given by various people including myself about RubyCocoa & Core Image, XCode Organiser, Python, and the iPhone.

I gave a presentation about Ruby Cocoa and some of the Core Image related research I had been working on to integrate it into Attachment Fu for processing file uploads in Rails applications. I’ve uploaded the slides from my talk if anyone is interested in them.

| View | Upload your own