fnokd!

Don’t forget, in a few weeks at the Red Hat offices in Raleigh, there will be an unconference-style Ruby BarCamp on October 18th.

My plan is to put together a few clumps of slides, prepare a cluster on EC2, and see if anyone wants to hear about JBoss Rails.  I’ll of course put them online sometimes before/during/after the camp, since that’s part of the rules.

Rysiek asked me to elucidate my clustering on OSX…

By default, it seems that OSX provides exactly 1 localhost address, unlike RHEL.  I normally make use of 127.0.0.1/24 when I’m deploying on Linux, and figured I’d do the same on OSX.

To accomplish this, you need to create some new localhost IPs to play with.

sudo ifconfig lo0 alias 127.0.0.10 up
sudo ifconfig lo0 alias 127.0.0.11 up
sudo ifconfig lo0 alias 127.0.0.12 up

Now, when you run your AS with ./run.sh, just pass -b 127.0.0.10 or -b 127.0.0.11 etc, and the entire stack will bind to that IP address.

At this point, I’m allowing each AS instance to share a work directory and such, which is probably not exactly the safest thing to be doing. Normally each cluster node would be a discrete machine with its own JBOSS_HOME.

For the apache config, in /private/etc/apache2/httpd.conf, I uncommented the line to allow vhost file loading:

# Virtual hosts
Include /private/etc/apache2/extra/httpd-vhosts.conf

Then, in the httpd-vhosts.conf file I fixed it up to include my virtual host for the head-end load-balancing stuff.

The <VirtualHost> is only barely conformant, and even the DocumentRoot is gratuitous, really. It looks like this:

<VirtualHost *:80>
  ServerName app.local.ballast
  DocumentRoot "/Users/bob/public_html"

  RewriteEngine on

  RewriteMap    lb      'prg:/Users/bob/bin/load_balancer local.ballast 1 3'
  RewriteRule   ^/(.*)$ ${lb:$1}           [P,L]
</VirtualHost>

What that is doing is passing each requested path to the load_balancer script I’ve borrowed from I-don’t-know-’where. It returns an augmented URL pointing to one of my localhost-bound nodes.

#!/usr/bin/env perl
##
##  lb.pl -- load balancing script
##

$| = 1;

$name   = $ARGV[0];         # the hostname base
$first  = $ARGV[1];         # the first server (not 0 here, because 0 is myself)
$last   = $ARGV[2];         # the last server in the round-robin

$cnt = 0;
while (<STDIN>) {
    $server = sprintf("node%d.%s", $cnt+$first, $name);
    $cnt = (($cnt+1) % ($last+1-$first));
    print "http://$server:8080/$_";
}

If a path of /foo/bar is handed to it on STDIN, it’ll return something in the format of http://node1.local.ballast:8080/foo/bar or http://node2.local.ballast:8080/foo/bar.

Through the magic of /etc/hosts those friendly names point to the localhost bound AS cluster nodes, along with app.local.ballast pointing to good old traditional 127.0.0.1, where httpd is normally listening. That’s my head-end from the httpd-vhosts.conf.

127.0.0.1       localhost app.local.ballast

127.0.0.10      node1.local.ballast
127.0.0.11      node2.local.ballast
127.0.0.12      node3.local.ballast

Fire up a browser and I surf to http://app.local.ballast/, apache answers, and immediately and invisibly proxies the request to port 8080 of one of my localhost-bound JBoss AS instances.

Nice.

JBoss on Rails will indeed cluster!

After modifying and dropping my jboss-rails.deployer into an ‘all’ configured server of JBoss AS 5, and firing up 3 instances on my localhost (non-trivial on OSX…):

10:43:28,409 INFO  [RPCManagerImpl] Received new cluster view: [127.0.0.10:63740|2] [127.0.0.10:63740, 127.0.0.11:63747, 127.0.0.12:63749]
10:43:28,435 INFO  [RPCManagerImpl] Cache local address is 127.0.0.12:63749
10:43:28,469 INFO  [ComponentRegistry] JBoss Cache version: JBossCache 'Poblano' 2.2.0.GA

And I’ve got 3 nodes running the same Rails app, all sharing a cookie and a JBossCache cache. Nick Sieger’s JRuby-Rack handles binding the Rails session to the actual servlet session, and JBossCache takes care of the rest.

A little 8-line perl round-robinning load-balancer is wired up through mod_rewrite in my Apache httpd.conf to throw requests to each of the nodes. Anything set in the session is immediately available at the next request which lands at a different node.

Further down the line, we can look at a clustered cache for caching AR models and view fragments. Not too shabby.

It should be fairly easy to create a nice Amazon EC2 AMI with Fedora+AS5+jboss-rails, plus some better Rake/capistrano tasks, and make for quick cluster deployment.   Any EC2 experts wanting to jump in?

Since JBoss AS 5 is built on top of Microcontainer, it’s effectively a network of beans just doing their respective jobs.  You have already probably noticed that it ships with 3 included configurations: minimal, default, and all.

Unfortunately, they’re awefully far apart along the spectrum of configuration options.  The minimal configuration barely gets the container running, while the all configuration includes, well, everything.  For most apps, it’s safe to start with the default configuration.  But default to who?  It’s the 80% case.  But the 80% of the world who gets by with default, it still probably includes way more than they need.  Of course, each user needs a different subset of that 80%.

Given that the MC is managing a graph, it seems like we should be able to actually perform a solve to determine what is or is not required.  Or at least get a good idea.  Bonus points if someone can then twiddle conf/, deploy/ and deployers/ to tidy things up.

Anyhow, for the jboss-as-rails project, I’m kicking it old-school, and going with default.  I’m jamming it into git, and hopefully with some help, we can get it pared down to what we need.

The jboss-as-rails project is simply a configuration of AS that includes only what we need, along with the jboss-rails.deployer from the jboss-rails project (notice the subtle naming difference…).

Here’s a picture to see how it’ll all ultimately fit together.

The plugin will offer rake tasks for managing the included AS, deploying your app.  I’ll also produce an AS-free version of the plugin, assuming you want to manage your own AS separately.

And poke around jboss-as-rails, see what we can rip out.

When digging through code, I find it’s good to print it out, wander off away from the machine, and do some reading.

But if you print any reasonable-sized chunk of code from Eclipse, you’ll be carrying around several reams of paper, since Eclipse apparently thinks we’re all blind illiterates.

For the past decade, my friend & mentor Jim Crossley and I have been passing the same encscript alias back and forth.  I recently had to request it from him again.  (Mental note: put it in SCM somewhere…)

alias print='enscript --color=blackwhite -E -T4 -M Letterdj -C -2 -fCourier@5/5 -r $*'

It’ll spew out your code in landscape mode, syntax-highlighted, two columns of 90 lines each (180 lines total per page), with line-numbers and a nice-looking header.

Many times, this allows you to fit an entire class on a single sheet without any gratuitous wrapping due to Eclipse’s 24pt font usage.

Tomorrow is my first real status update call with my boss, Sacha Labourey.  I’ve been anxious to deliver something, to prove I hadn’t gone completely pudding-brained during my tenure as management.

This morning, it all finally came together in a pleasing fashion, causing me to hoot and holler loud enough to scare the cats and probably some cows.

I’ve just pushed an ugly-but-working deployer targeting JBoss-AS 5.0.0.CR2 (the latest and greatest!)

It’s not very consumable at this point, as it’s just a deployer, not a nice Rails plugin with a set of Rake tasks.  Heck, it doesn’t even undeploy yet.

But adding the deployer to your server’s deployers/ directory allows you symlink live RAILS_ROOTs into your deploy/ directory, and be running on JBoss.

Live.  In-situ.  Edit your controllers or views as you like, and your changes are immediately reflected in the running instance.  Just like with ./script/server.  It does not even have to redeploy your app.  The rails framework is handling the magic reloading.

It’s taken me some time to dig through the innards of JBoss-Microcontainer, and a few false starts, but I finally figured out a super simple deployment process.

I’d previously been trying to manipulate a RAILS_ROOT into a synthetic Java WAR archive, and shoe-horn things around that.  But I have the freedom to go lower than that, so the jboss-rails deployer just sets up a Catalina context appropriately, without regard to WEB-INF or other non-Rails stuff.  There’s no need for that cruft.  Likewise, I can directly control and manipulate the classpath, so the RAILS_ROOT does not even have to have any JRuby bits in it.

The example application (src/test/ballast) is a virgin rails app with ActiveRecord disabled so I don’t have to deal with database-driver gems just yet.

Once deployed, a Rails app looks like pretty much any other web-app.  The jboss.rails.deployment domain contains deployment objects for each rails app.  And jboss.web contains all the webby bits floating around.

I need to go  back and remove the dead-end code I’ve left in my wake, and update the tests I’d disabled while in a coding flury (bad Bob!)  I plan to put together an easy-to-consume plugin gem which contains an nicely-configured AS along with the jboss-rails deployer pre-installed, along with rake tasks to start/stop AS, and deploy your app.  I’d also like to give clustering a whirl, and see what we can do.

It’s been an excellent 3 weeks back as an engineer.

Since I’ve been back on the job, writing Java code lately, that means I’ve been testing Java code lately.

After living in the land of Ruby with RSpec, thinking about JUnit did not excite me.  Thankfully I found the rspec-maven-plugin, which integrates straight into the maven test process.

I like rspec because it removes a whole lot of the cruft involved in writing tests.  When I return to the specs weeks later, I can still read the intent.  So often, with JUnit tests named with cryptic method names, it’s difficult to ascertain exactly what’s intended to be tested.

The beta-4 plugin on the repository is based on jruby-1.1.2, but in SVN I’ve updated it to support 1.1.4.  I think there’s still some more improvements to be made, such as respecting maven.test.skip and getting rid of the need for JRUBY_HOME pointing to a full installation. (Note: I did not write this plugin, I’ve just started contributing to it).

After a little configuration of the plugin in your pom.xml…

<plugin>
  <groupId>org.codehaus.mojo</groupId>
  <artifactId>rspec-maven-plugin</artifactId>
  <configuration>
    <!-- jrubyHome points to the JRuby installation you wish to use (usually ${env.JRUBY_HOME}) -->
    <jrubyHome>${env.JRUBY_HOME}</jrubyHome>
    <!-- sourceDirectory references where your RSpec tests reside -->
    <sourceDirectory>${basedir}/src/test/specs</sourceDirectory>
    <!-- outputDirectory specifies where the RSpec report should be placed -->
    <outputDirectory>${basedir}/target</outputDirectory>
    <!--<skipTests>true</skipTests>-->
  </configuration>
  <executions>
    <execution>
      <id>test</id>
      <phase>test</phase>
      <goals>
        <goal>spec</goal>
      </goals>
    </execution>
  </executions>
</plugin>

Now, just start flinging out specs in ./src/test/specs/.

This is a portion of a test against a JBoss-VFS VirtualFileHandler implementation, written in Java in WarRootHandler.java:

describe WarRootHandler do

  before(:each) do
    @context = RailsAppContextFactory.getInstance.createRoot( "ballast", File.dirname( __FILE__ ) + '/../ballast' )
    @root = @context.get_war_root_handler
  end

  it "should have a rails:// URL" do
    @root.to_uri.to_string.should eql( "rails://ballast/" )
  end

  it "should be resolvable through java URL handlers" do
    url = java.net.URL.new( "rails://ballast/" )
    url.to_s.should_not be_nil
  end

  it "should delegate for WEB-INF requests" do
    web_inf = @root.get_child( 'WEB-INF' )
    web_inf.should_not be_nil
    jboss_rails_yml = web_inf.get_child( 'jboss-rails.yml' )
    jboss_rails_yml.should_not be_nil
  end
end

In order to deploy a Rails app, I’ve had to learn the innards of Microcontainer’s deployer framework.  After a few wrong turns, I feel like I’ve finally gotten a handle on it.

While we’re all used to dropping in an .ear or a .war, and might think in terms of deploying these archive formats, that’s ultimately one step removed from true deployment through MC.

Within MC, when you deploy a .war or an exploded WAR directory, the first step is something recognizes that the chunk you’re deploying is roughly shaped like a WAR.  I’ll address that phase of deployment in a future post.

Knowing that the deployment is a WAR also tells MC to look in WEB-INF/ for meta-data descriptors, such as web.xml and jboss-web.xml.  This is where true deployment of components starts.  Deployment runs through a series of stages, with deployers setup to match particular files and stages, doing the right things at the right time.

One of the earliest stages is the PARSE stage.  A deployer can be bound to this stage to be given an early chance to match, parse, and act upon any meta-data file.  For normal WAR deployment, the WebAppParsingDeployer does exactly that.  There’s a nice hierarchy of classes to make parsing XML descriptors such as web.xml super simple.

The WebAppParsingDeployer is the bridge from a web.xml file sitting on the filesystem or in an archive to the MetaData deployment bits. The parser reads web.xml, and produces a WebMetaData object associated with the deployment.  The WebMetaData is simply a nice object-tree representing anything you can denote in web.xml.

We also might have a jboss-web.xml meta-data in our WAR, and that is parsed during the PARSE stage by the JBossWebAppParsingDeployer.  This deployer, like the previous, reads the XML file and creates, in this case, a JBossWebMetaData object.

Once we’ve parsed these .xml files, the container has enough information to build up the classpath for the component.  Some of these deployers have also thrown off or modified some ClassLoadingMetaData, which describe paths that should be added to the classpath.

As the container enters the CLASSLOADER stage of deployment, other magic occurs to actually set up the classpath.

In the end, it’s the JBossWebMetaData that drives the ultimate deployment, but what if we don’t have a jboss-web.xml?  That’s where the MergedJBossWebMetaDataDeployer comes in.  It looks for a WebMetaData, and a JBossWebMetaData if one has been parsed, and merges them into a singular JBossWebMetaData.  I think it also mixes in any defaults that you have set for server-wide settings.

Additionally, as jboss-web.xml is parsed by JBossWebAppParsingDeployer, it will perform the merge itself.  Additionally, magic is occuring to merge any annotation-based meta-data.

I’m a little fuzzy on the ins and outs of the CLASSLOADER stage at this point, but magic occurs there.

And our app still isn’t deployed yet.  But we’re getting there.

Finally, we enter the REAL stage of deployment, which fittingly-enough, is where the actual deployment occurs.  Hooray!

Our TomcatDeployer is hanging out there, waiting for JBossWebMetaData objects to appear.  When it sees one, it goes to work setting up information for Tomcat to deploy a web-app.  It configures everything in Tomcat from the information other deployers figured out from web.xml and jboss-web.xml and embodied in the MetaData.

It jams it into Tomcat, hits the big red “go” button, and port 8080 is serving you web-app.

Finally.

In general, to deploy in AS5 using Microcontainer, you need some MetaData bits, and perhaps a bag of files/classes/resources.  Nothing says they have to be bundled into a .war, or include some j2ee XML deployment descriptor.  If you have other magical ways of bundling MetaData and resources, you’re good to go.

Of course, Ales or Adrian may tell me I’m completely wrong.  That’s always a possibility.  In fact, I’m sure I’ve got some things wrong, in reverse order, and otherwise mixed-up.

Here’s a picture for you, though.

Update:

While discussing this on the Microcontainer user’s forum, I discovered that there are indeed several errors and inconsistencies in the above.

Update #2:

New image, slightly new text to match the image better.  Comments and clarifications still welcomed.

Code comprehension.  It’s important when you jump into someone else’s code.

Eclipse makes it easy, with F3, command-T, and shift-command-G.

Very quickly, you can jump through a maze of classes and interfaces, diving into details or seeing the higher hierarchy.  I forgot how nice Eclipse can be.  Particularly if you’ve got the viPlugin.

72 open files (and bless the JBossAS guys for including Eclipse projects and source jars).

I’m in the process of moving to some farmland in Virginia.  It’s out in the boonies.

How far in the boonies?  No cable.  No DSL.  GSM is sketchy.

Thankfully, I’ve moved my personal development to Git as noted previously, which works wonderously.  With no TCP in the air, I can still commit, and push my complete history on to the repository-of-record when I do find some radio waves.

So, I spend my mornings at Starbucks using the heck out of their wifi and the afternoons in a small 4-square house surrounded by cows.  I’ve seen various strategies and tools for turning off the internet for some predetermined amount of time, to allow you to focus without distractions.  I’ve indeed found The Farmhouse Method to be great for focusing on coding, and not browsing porn blogs.

Next week, though, I’ll take a half-step closer to being somewhat online.  WildBlue delivered a dish and a modem to bounce some bidirectional internet signals off a satellite.  At a stellar 1 watt broadcasting power.  It’s not going to be awesome, particularly coming off 6mbps DSL.  The latency on interactive traffic is also debilitating.  The speed of light is just too slow bouncing all the way up to the bird and back, plus the return trip for the echo.

I also plan to evaluate Alltel’s “wireless internet” EVDO service, supposedly unlimited, but also probably not funneling between the mountains to my valley (or “cove” as we call them here in the south).

But then again, I get to wake up and drink coffee with the free-range cattle.  That’s a trade-off I’m willing to make.

And there’s always Starbucks.