Using Neo4j Graph Databases With ColdFusion

After last week, I decided to put off picking a new frontend platform for my Semantic Web rubric project and focus a bit on the server backend.

Since this is just a proof-of-concept project at this point I can afford to take some risks in choosing technologies. I’ve been following the developments around using graph databases for storing data, especially for Semantic Web applications. One project that kept coming up was Neo4j, a graph database engine built in Java. I figured now was a good time to try it out. My server-side logic is built in ColdFusion, and integrating open source Java projects like Neo4j into CF applications is generally a snap.

Aside from one hiccup, porting Neo4j’s 1-minute Java “Hello World” example to CFML proved to be fairly straightforward. The process I used to get this working is detailed below. I’d suggest that you skim over the Java example before continuing – I’m sure I left out some of the exposition.

First add the Neo4j Jar files to the ColdFusion server:

  • Download the Neo4j “Apoc” distribution and unpack it somewhere convenient. I’m using Mac OS X, so I put things like this in ~/lib/neo4j-apoc-1.0
  • Add the Neo4j JAR files to the ColdFusion classpath. Log into your ColdFusion Administrator, and select Server Settings -> Java and JVM. Enter the path to the lib folder in your Neo4j distribution in ColdFusion Class Path
  • Restart your ColdFusion server. If you’re at all nervous, log back in to the ColdFusion Administrator and verify that the Neo4j jars are indeed listed on your classpath.

Once this is complete, you can initialize a new database for your ColdFusion app. Decide where you want the CF server to create the Neo4j data files and pass that to the object’s init() method. I put mine in a folder under /tmp on Mac OS X.

<cfset dbroot = "/tmp/neo4jtest/" />

<cfset graphDb = createObject('java',
                  "org.neo4j.kernel.EmbeddedGraphDatabase") />
<cfset graphDb.init(dbroot & "var/graphdb") />

[Aside for non-ColdFusion folks: CF doesn't instantiate Java objects quite how you'd expect. The call to CreateObject() just gets a handle on the class itself. Calling init() on the resulting handle actually instantiates the class via the appropriate constructor.]

Just as in the Java example, it’s good to surround your connection with a try/catch block that will close your database connection if you throw an error. As I was working with Neo4j I would periodically lock up my database and not be able to connect without restarting CF. Adding a CFTRY/CFCATCH block cleared this right up.

<cftry>
   <cfset tx = graphDb.beginTx() />

   <cfscript>
     tx.success();
     WriteOutput("Success.");
   </cfscript>

   <cfset tx.finish() />

  <cfcatch type="any">
     <cfset graphDb.shutdown() />
     <cfdump var="#cfcatch#">
   </cfcatch>
</cftry>

<cfset graphDb.shutdown() />

Where things got really sticky was the use of Java enumerations to declare the available relationship types for the graph:

 /* Java code */
 public enum  MyRelationshipTypes implements RelationshipType
 {
    KNOWS
 }

To my knowledge there’s no way to declare something like this in standard CFML. I likely could have wrapped this in a Java class of some sort and loaded it through CreateObject(), but that wouldn’t have been true to the spirit of ColdFusion. So I dug around in the Neo4j docs and found an answer: relationships can be created dynamically at runtime from a static method on the class org.neo4j.graphdb.DynamicRelationshipType. I created an instance of DynamicRelationshipType for the “KNOWS” relationship and loaded it into a Struct, anticipating caching them in Application scope for a real application.

 relationship = CreateObject("java",
                             "org.neo4j.graphdb.DynamicRelationshipType");
 MyRelationshipTypes = structNew();
 MyRelationshipTypes.KNOWS = relationship.withName( "KNOWS" );

It might be interesting to see if these relationship enumerations could be generated and compiled by something like JavaLoader. I’m not yet aware of any downsides with dynamic relationships besides the obvious lack of compile-time checking.

The rest of the exercise follows without any real suprises:

 firstNode = graphDb.createNode();
 secondNode = graphDb.createNode();
 relationship = firstNode.createRelationshipTo( secondNode,
                                         MyRelationshipTypes.KNOWS );

 firstNode.setProperty( "message", "Hello, " );
 secondNode.setProperty( "message", "world!" );
 relationship.setProperty( "message", "brave Neo4j " );

 WriteOutput( firstNode.getProperty( "message" ) );
 WriteOutput( relationship.getProperty( "message" ) );
 WriteOutput( secondNode.getProperty( "message" ) );

And there you have it! A quick and dirty Neo4j application built with CFML.

I’ve put a little work into developing a Neo4j helper class that hides some of these warts in a nice clean CFC. As soon as I can get eGit to behave I’ll post the files on GitHub.

Synchronized Web development workflow

It’s been six years since I switched from HomeSite to Eclipse+CFEclipse as my primary ColdFusion development environment. At the time, my switch was primarily driven by my switch to Mac for development, but the desire for integrated support for version control (e.g. Subversion) directly within in the IDE helped with that as well.

One of the things that has long bugged me about developing ColdFusion apps locally on a dev box (i.e. not on a shared network server) is the need to place project files directly in a Web root somewhere – for example, C:\InetPub\wwwroot on Windows/IIS or in C:\ColdFusion9\wwwroot if you use ColdFusion’s built-in Web server. This throws off my game in two ways:

  1. Browsing to where your files live (in Finder/Explorer, or via the command line) inevitably adds an extra step to every task
  2. Placing the project home outside your user space on the OS makes it more likely you’ll lose the files when upgrading/uninstalling/migrating.

Pain point one could be tackled by sprinkling my system with aliases, shortcuts and/or symbolic links. This reeks of configuration, and would be something I’d need to duplicate on any system I use.

Pain point two there is really the kicker. Sure, using version control keeps you from losing your work, but rebuilding the workspace after a system migration can take a long time. I rarely migrate applications when I get a new system; I prefer to just move the user files and reinstall the apps manually. This periodically cleanses the system and keeps me up to date on patches, even on apps I don’t use often.

I tried keeping my actual project files in a workspace somewhere convenient, such as in my home directory or in the root of the drive (e.g. C:\workspace\MyColdFusionApp)  and then copy the files to the server root to test them. I tried both manual copying and even Subversion commands, but I couldn’t keep that up for long. Part of the benefit of developing anything locally on your system is removing the step of uploading your code to a server to test it, and I was basically backsliding towards that kind of process.

But the idea was sound, and I looked around for something that would painlessly synchronize two folders in different parts of the drive – the project files in my workspace, and the files in a folder under the Web root. My last resort would be to use something like rsync, but I looked around for some sort of plugin for Eclipse – something that could keep the preferences as part of the Eclipse project and/or workspace and be easy to migrate and hard to lose.

With a little digging I found FileSync – a plugin which really fit the bill. It’s open source, and if you can look past the, ah, unpolished Web site of its creator the plugin works pretty well. When I save a file in my workspace, it painlessly gets pushed out to the Web server root for testing.

The plugin also appears to work with network drive targets, so you may be able to use it to publish changes out to a preview or QA server automatically, but you should probably be using some sort of version control for that. :)

April 14, 2010Permalink

Why many Microformats begin with ‘h’

I’ve been spending some quality time with several Microformats as part of my work for DealerPeak. We’ve been adding semantic attributes, including hCard, hProduct, and hListing, to the pages generated by the DealerPeak Automotive Dealership CRM/CMS system.

During a recent redesign of the car listing page, I was adding hCard microformat information to the dealer contact information block. As I was reviewing the hCard specification, I came across the following text:

The root class name for an hCard is “vcard”. An element with a class name of “vcard” is itself called an hCard.[1]

This distinction struct me as a bit odd, but I didn’t think too much of it because I had a deadline.

Over the past few days I’ve been working with one of the other developers again on some Microformat ideas, this time implementing some of the hProduct and hListing elements into a similar page. Sudden inspiration struck me – the ‘h’ is an ASCII-safe lower-case Mu (μ) – the SI prefix for “micro”!

[1] http://microformats.org/wiki/hcard#Root_Class_Name

FlexBuilder 3 for academia

Since FlexBuilder 3 was released last week, I decided to hunt around for the ‘free to academics’ offer and see if it applies to FlexBuilder 3. Here’s where it lives now:

http://www.flexregistration.com/

Interesting things to note:

  1. It apparently does apply to staff too (woo!)
  2. You can request more than one license if you’re using it for a class or a lab. That will be a real time-saver.
  3. You have to scan your ID and submit it with the form. Uhh… scan? paper? I don’t have a scanner hooked up anywhere, so I took a photo of my ID with the iSight on MacBook Pro and tried that, thumb included free of charge.

The action page tells you to allow a few weeks for processing the request, but mine was approved the next day…. that’s service!

Heading to CodeMash ’08

I’m really looking forward to CodeMash. The slate of speakers and topics looks fantastic; It’s really nice to look at a conference schedule and see a lot of topics that are totally new to me.

One thing I’m curious about is Scala. I’ve been working with a research group lately on a project using Intelligent Agents, and through that was introduced to the idea of Functional Programming. Somehow I missed seeing this in my undergraduate days, though I remember my peers complaining about Scheme in one of the senior-level computer science courses.

Some of the talks on Groovy and Grails seem interesting, too. The Ruby on Rails movement has certainly sparked some innovation in the Web Development community, and I like seeing those ideas cross-ported into the technologies I have more of an affinity for, such as Java and ColdFusion. Having recently built a somewhat painful full-scale Java application, there may be something useful here.

Advice for jumping into Java Web Services?

So I’ve landed on a new project. We’ve been asked to port a well-studied
scientific algorithm into a Web Service, hoping to link the calculated
results into a networked client for visualization (likely Google Earth).

Now, porting the algorithm (from MATLAB) should be relatively
straightforward. What I’m unsure about is where to start building the Web
Service! I’ve previously just used ColdFusion for anything that needed to serve a
data feed (like sending RSS or feeding a Flash or Flex app), but the requirements from the
contracting agency really point to something more portable like Java, Python or Ruby. Since I’m most familiar with Java, and there have been a few articles here and there on Java Web Services, that seemed to be a likely path.

I’ve read about SOAP, REST, and XML-RPC, and the Apache Axis library. Then I found Axis2, and heard about CXF from a colleague. Can anyone
offer any advice as to where to start? Does anyone even build Web Services by hand anymore?

Oh, and Barry… I know I still owe you those 8 things you asked about.

Problems (and a Few Solutions) Installing Adobe CS3

So we got our media for Adobe Creative Suite 3 last week, and I’ve been fighting with the installs on various systems for the last few days. Some notes from the battlefield:

That popping sound you heard was sudden obsolescence

Adobe Premiere Pro and Soundbooth are only supported on multicore Intel Macs. So much for our “multimedia lab” of 2GHz Dual G5 PowerMacs.

You want how much RAM and disk space?

Be prepared to clean up your hard disks, especially on notebooks — CS3 Master want’s almost 20GB of disk space to install. Web Premium for Windows also complained on our XP systems, which have only 512MB of RAM. I can’t really fault them for that, though… XP alone takes that much RAM to function. ;)

Blank Popup Window on Mac Pro Install

I fought with installing Master Collection on my Mac Pro for a few days. The install would get to the end of disc 1 and pop up a blank alert box, mutely asking me to put in the second disc. There were no buttons to click on, and nothing I did could get the install to continue. I had to force-quite the Setup app and clean up my system.

I was eventually able to complete the install by copying the contents of all four discs to a single “payload” folder on my hard disk and running the install from there, as described here.

Turns out the culprit may have been the Safari 3 beta.

Apollo, we have liftoff!

At last! Adobe Apollo, the new cross-platform runtime for building rich internet applications is now available on Adobe Labs:

http://labs.adobe.com/technologies/apollo/

I’ve been reading and hearing about Apollo for what seems like a year… I’ll have to take some of my off-week between classes to give this a test drive by converting some of my Flex 2 applications to Apollo applications.

Holy Xeons!

Wow… ColdFusion MX on the MacPro is fastfastfast. It took a while to get it set up since it isn’t technically supported, but wow. WOW I say.

I skimped at least on one step: I didn’t recompile mod_jrun on this box. Since I’d gone through that process on the MacBook Pro, I just copied my existing Intel binary .so file to the new machine, and everything seems to be running just fine.

I had to mangle several lines in the config files I copied over from my G5, as the CF files now live in /Applications/JRun4 rather than /Applications/ColdFusionMX due to the recommended multi-server install.

The big drag again was adding our homebrew SSL CA certificate to the system keychain to let our development applications authenticate against Active Directory. I think we should just pony up the cash and get a verified certificate this year.