onsdag 23. november 2011

Show gradle dependencies as graphwiz png

A few weeks ago I needed to show my team an overview of all the dependencies between our sub projects. I found a solution where I generated a graphwiz dot file. The caveat here was that I had to install graphwiz and run another command to generate a png.

Today I was reading through "Making Java Groovy", MEAP authored by Ken Kousen. I saw an example using the googles charting api. A quick check revealed they had experimental support for creating a png from a dot graph !

So my gradle task ended up looking as follows:

reportsDir = new File("build/reports")
compileDepsPng = file("$reportsDir/compileDeps.png")

task dependenciesPng() {
  inputs.files subprojects.configurations.compile
  outputs.files compileDepsPng
  doFirst {
    if(compileDepsPng.exists()) compileDepsPng.delete()
    if(!reportsDir.exists()) reportsDir.mkdirs()
  
  }
  doLast {
    dotGraph = "digraph Compile{"
    subprojects.each {subproject ->
      subproject.configurations.compile.dependencies.each {dependency ->
      if(dependency instanceof ProjectDependency) {
 dotGraph += "\"$subproject.name\" -> \"$dependency.name\"" 
      }   }
    }
    dotGraph += "}"
  
    def chartParams = [cht: 'gv', chof: 'png', chl: dotGraph]
    def url = "http://chart.googleapis.com/chart?" 
    url += chartParams.collect {k,v -> "$k=${URLEncoder.encode(v)}"}.join('&')
  
    compileDepsPng.withOutputStream{out ->
      out << new URL(url).openStream()
    }   
  }
}





fredag 7. oktober 2011

Textmate made friendly for running Groovy snippets

I seem to remember Scott Davis doing a groovy presentation at JavaZone in 2008 or was it 2009, using Textmate. Didn't think much of it at the time. I had enough editor confusion going on,  switching from eclipse to idea.

Earlier this week a colleague and I was working on a Groovy introduction presentation and we had decided on using textmate. So we downloaded the capable Groovy textmate bundle from Github. I had been using the Groovy console and found the performance for running small groovy snippets to be rather unbearable with the textmate plugin though. You got the startup cost of firing up the jdk+groovy and it opened a new window. This was not what I remembered, and it certainly wouldn't work for a code driven presentation. A quick google found a youtube video with Dierk König. Aha of course Groovyserv to the rescue.


So the short version:
  1. Got Homebrew ? Well then just  brew install groovyserv
    1. No ? Install it (if you don´t have xcode either... I suggest you download groovyserv manually unless you have the time to spare. People complain about maven downloading the internet, xcode feels pretty close)
  2. In textmate select bundles->bundle editor->edit commands
  3. Add a command for running the current file. The following settings might work for you:
    1. Save current file (usually what I want)
    2. Commands : /usr/local/Cellar/groovyserv/0.9/libexec/bin/groovyclient -encoding "UTF-8" "$TM_FILEPATH"
    3. Input : Entire document
    4. Output: Show as tooltip (worked out real nice in the presentation)
    5. Activation : cmd+h (or something that suits you... that particular combination has a somewhat different behaviour in the GroovyConsole btw)
  4. Similarily if you just want to run a selection just change command to something like:
    1. /usr/local/Cellar/groovyserv/0.9/libexec/bin/groovyclient -c "UTF-8" -e  "$TM_SELECTED_TEXT"
  5. Unlike the console it doesn't output the last executed statement, so if you are doing oneliners you better remember println ! (If anyone has any tips for that one in Textmate, feel free to comment on the post)

You are all set to run nice little groovy snippets with near instant responses (well the first time you try it, groovyserv will incur a heavy startup cost. But after that you are good to go in kickass speed).

tirsdag 13. september 2011

Tapestry5 and GORM

So I've had some fun lately trying to get Tapestry and Gorm to play nicely together. To make the learning experience a bit more realistic I started doing a small sample app. Its loosely inspired by a domain I'm scratching the surface of in my current consulting gig. I've had a ball learning more about groovy and Gorm, and it has been a nice refresher on my already fading Tapestry skills.

Domain modeling with Gorm has been an absolute pleasure (especially from someone coming from hibernate/jpa). It's just so neat and concise... and there is also all these nice little freebies you keep discovering.  Working with Tapestry (and the cool jquery flavored add-on tapestry5-jquery) was about as productive as I remembered. With the ability to write my pages/components in Groovy, the experience was even better. When the limitations of the java api became a burden, I always had the option of putting a thin layer of groovy in front to DSL'ify it to meet my expectations better. I suspect I will keep playing with this app to see where it leads. I'm also tempted to do a cousin implementation using grails sometime soon as well.

Did I have any problems along the way ? Sure did, but mostly in terms of things like:

  • Unifying logging from all those external dependencies (yeah its a lot I'm afraid, and Gorm is the main culprit!). To make life hard I figured I wanted to use logback. Once I finally got my dependencies right, with bridges and all, it now works fine
  • Setting up Spock tests with transactional support (proved to be a lot simpler than I initially thought) 
  • Intellij compilation error (which isn't really and error, but disrupts the flow). Keep getting a system.err when compiling groovy code. Unfortunately intellij insists its an error even though the compile actually succeeds. Something todo with groovy stubs and/or ast transformations I think. 

If time permits I'm planning to do a few future posts that dig into some details from the sample app. 


If you are interested in the sample app, feel free to pull it from github at https://github.com/rundis/TapestryGorm

lørdag 6. august 2011

A groovy infatuation growing


I guess for the last 2 years I've been interested and intrigued by the groovy ecosystem. However customers in Norway that are groovy ready aren't exactly lining up in my experience. I was fortunate enough to help a customer last year introduce the build toolkit #gradle (love it). It gave me an opportunity to at least get a taste of groovy in real life projects.  Since then I've been looking for opportunities to introduce smatterings of groovy wherever applicable.

During the summer I had grandiose plans for various hobby projects, but forgot that family sort of requires most of that holiday spare time. I've immensely enjoyed reading the "Groovy in action" second edition Meap, and its triggered a lot of other groovy related readings.

Grails pick and choose ? Yay!
Anyways I have been playing around bit,  and one of the things that has been paying dividends is looking at a possible web stack of #Gorm and #Tapestry5.
Why not just grails ? Well.... ignorance might be part of it, but I really do like the component oriented/ event oriented kind of web frameworks like tapestry, wicket and vadin. Maybe the fear of leaving java too far behind is getting the better of me as well. Yeah and well I have had some very positive experiences with using Tapestry. There is that. 

Web frameworks are written by backend people
Silly to couple JS so tightly with a web framework you say ? I'm sure there is some merit(sometimes a lot) in that, however I don't always have the luxury of having an JS expert next to me, and frankly most of the aforementioned frameworks have made leaps in "liberating" themselves from specific js frameworks. Specifically I've been trying out tapestry in a jquery dressing and enjoyed it so far. Use what you like of the client side stuff and customize the rest, it might just work out pretty well for you. (I'm no fan of struts of spring mvc btw).

Sample app
I'm currently working on a sample groovy app using tapestry, gorm, spock and gradle. Managed to "unify" logging using logback (groovy friendly) and I'm sure geb might be considered as well. Regardless, the general experience so far is: Most things are just so pleasant and amazingly productive to work with. IDE support has been above all expectations (thx jetbrains/idea) and documentation etc have been plentiful. 

Bye java ?
I've really enjoyed java in the past and still believe the jvm has a bright future (unless oracle messes it up big time). However java the language is just ... well ... lagging and not nearly as ... groovy.

A blog post about the sample app will be coming. But so far I can reveal that #tapestry5 and #gorm are playing fairly happily together :-)

good night
- and yes I did write this up in a "got it to work" euphoria

"${magnus}"

mandag 28. februar 2011

Incremental builds - great potential, reliant on trust

The first time I evaluated gradle as a candidate build system I was intrigued by its incremental build feature. Having worked with multiprojects in maven previously I must admit I was pretty hardwired into using "mvn clean install". With maven that felt as the only safe bet way to ensure that any changes in subprojects didn't break the overall build. However for large multiproject builds it sure felt as I was spending a lot of time waiting for a range of build activities that shouldn't have been necessary.
What if my build system could be clever enough to do just the activities it must, skip the others and still ensure the overall build integrity ? Gradle came with a promise to go a long way doing just that, so obviously I had to try it out.


Sample multiproject:

  • dumpling : Root multiproject (packaged as pom in maven )
  • dumpling-util : A utility project (jar)
  • dumpling-core : Application domain (jar, depends on dumpling-util)
  • dumpling-webapp: Web app  (depends on dumbling-core, and dumpling-util transitively)


To see how gradle and maven behaved I performed some simple trials. For each step I observed the main activities that was reported.


Sample scenario 1: Non breaking change

Precondition:

  • maven: mvn clean install
  • gradle: gradle clean build

Changed a source file in dumpling-util. The change is internal to the class. Then for maven and gradle I did following:

  • maven: mvn install
  • gradle: gradle build



dumpling-util observations:
Taskmvn observationgradle observation
resources copied resources (not needed) skipped
compile compiled changed file compiled changed file
test resources copied resources skipped
test compile no work compiled test sources
jar created created
install installed to local repo n/a



dumpling-core observations:
Task
mvn observation
gradle observation
resources
copied resources (not needed)
skipped
compile
none
full recompile (due to input file dumpling-core jar had changed)
test resources
copied resources
skipped
test compile
no work
full recompile (due to input file dumpling-core jar had changed)
jar
created
skipped (input classfiles and resources same checksum as previous run)
install
installed to local repo
n/a



dumpling-webapp observations:
Task
mvn observation
gradle observation
resources
copied resources (not needed)
skipped
compile
none
full recompile (due to transitive input file dumpling-core jar had changed)
test resources
copied resources
skipped
test compile
no work
full recompile (due to transitive input file dumpling-core jar had changed)
war
created
created (due to transitive input file dumpling-core jar had changed)
install
installed to local repo
n/a

Comparing the two one might at first conclude that both did unnecessary work. 

Maven: Copied resources and test resources for all subprojects eventhough none of them had changed. For dumpling-core it created the jar eventhough not a single file in the jar had changed.
For dumpling-core and dumpling-webapp it didn't do any work compiling java sources ar test sources. This might seem ok, but next scenario will prove this to be untrue.

Gradle: It seems unnecessary to do some of the activities that gradle does, but there is a reasonable explanation behind. In gradle there is a concept of inputs and outputs for tasks. A task may have a defined set of inputs (files) and a defined set of outputs (files). For each execution of a task gradle calculates checksums for the inputs and the outputs. If gradle finds that the checksums have changed since last run, the tasks will be run (unless other factors such as explicit disabling of tasks has been configured). So an example would be dumpling-util:test compile. This task is executed because its input files from dumpling-util:compile has changed.

Sample scenario 2: Breaking change

I changed the interface of a method for a class in dumpling-util, and updated its test case accordingly. The method in question is used by a class in dumpling-core, so dumpling-core needs to change...
Then for maven and gradle I did following:
  • maven: mvn install
  • gradle: gradle build

dumpling-util observations:
Task
mvn observation
gradle observation
resources
copied resources (not needed)
skipped
compile
compiled changed file
compiled changed file
test resources
copied resources
skipped
test compile
compiled changed test source file
compiled test sources
jar
created
created
install
installed to local repo
n/a


dumpling-core observations:
Task
mvn observation
gradle observation
resources
copied resources (not needed)
skipped
compile
none
full recompile (due to input file dumpling-core jar had changed). 

BUILD Failed due to compiler error on class in dumpling-core that uses changed interface method of class from dumpling-util
test resources
copied resources
n/a
test compile
no work
n/a
jar
created
n/a
install
installed to local repo
n/a


dumpling-webapp observations:
Task
mvn observation
gradle observation
resources
copied resources (not needed)
n/a
compile
none
n/a
test resources
copied resources
n/a
test compile
no work
n/a
war
created
n/a
install
installed to local repo
n/a


It instantly becomes obvious that the incremental build features of maven is rather limited. Gradle on the other hand fails fast where it should. Obviously maven has never advertised great support for incremental builds, but the comparison helps put things into perspective.

Parting words
The potential of huge savings in time due to incremental builds is obvious. However vitally important for the feature is the level of trust you can put into it as well. In gradle you have great flexibility of creating your own tasks, if these rely on file inputs and outputs be sure to consider using the inputs/outputs mechanisms available to you. But do take care  when defining them, otherwise you might end up with a trust issue and reverting back to "clean build" :-)




fredag 18. februar 2011

Gradle friends with Sonar

A little while ago a colleague of mine asked me if I had done a Sonar analysis of my current project. Well I hadn't so I started to investigate a little on what my options were given my build was setup with Gradle. After reading up a bit on the gradle forums, it appeared that it wasn't all that trivial. In particular getting my emma coverage available for sonar seemed to be a stumbling block. The sonar version at the time also seemed to be fairly tied to maven, a build system I sort of left behind last year.

Anyways yesterday I thought I'd check up on the announced plans for providing gradle support for sonar. Not quite there yet, but there was a snapshot version with ant support. Gradle and Ant being pretty good friends, I decided to give it a go.

Short story is that it worked. However I couldn't manage to get the testcoverage from emma in sonar, so I switched my setup to use cobertura (did slow my build quite a bit actually). Another remaining issue is all the annoying error messages from the pmd plugin in sonar. Lots of exceptions related to it for some reason insisting that I'm using jdk 1.4, but the fact is that I'm using jdk 1.6.

My build.gradle setup:
 apply plugin: 'java'  
 configurations {  
   sonarLibs  
    coberturaLibs {extendsFrom testRuntime}  
 }  
 dependencies {  
   //.. other deps omitted  
   coberturaLibs 'net.sourceforge.cobertura:cobertura:1.9.3'  
   sonarLibs "org.codehaus.sonar-plugins:sonar-ant-task:0.1-SNAPSHOT"  
 }  
 def cobSerFile = "${project.buildDir}/cobertura.ser"  
 def srcOriginal = "${sourceSets.main.classesDir}"  
 def srcCopy = "${srcOriginal}-copy"  
 test {  
   useTestNG()  
   maxParallelForks = 2  
   afterTest {descriptor, result ->  
     println "${descriptor.className}.${descriptor.name}"  
   }  
   /** Cobertura setup **/  
   if (project.hasProperty('coverage') && ['on', 'true'].contains(project.properties.coverage)) {  
     systemProperties["net.sourceforge.cobertura.datafile"] = cobSerFile  
     doFirst {  
       ant {  
         // delete data file for cobertura, otherwise coverage would be added  
         delete(file: cobSerFile, failonerror: false)  
         // delete copy of original classes  
         delete(dir: srcCopy, failonerror: false)  
         // import cobertura task, so it is available in the script  
         taskdef(resource: 'tasks.properties', classpath: configurations.coberturaLibs.asPath)  
         // create copy (backup) of original class files  
         copy(todir: srcCopy) {  
           fileset(dir: srcOriginal)  
         }  
         // instrument the relevant classes in-place  
         'cobertura-instrument'(datafile: cobSerFile) {  
           fileset(dir: srcOriginal,  
               includes: "**/*.class",  
               excludes: "**/*Test.class")  
         }  
       }  
     }  
     doLast {  
       if (new File(srcCopy).exists()) {  
         // replace instrumented classes with backup copy again  
         ant {  
           delete(file: srcOriginal)  
           move(file: srcCopy,  
               tofile: srcOriginal)  
         }  
         // create cobertura reports  
         ant.'cobertura-report'(destdir: "${project.buildDirName}/test-results",  
             format: 'html', srcdir: "src/main/java", datafile: cobSerFile)  
         ant.'cobertura-report'(destdir: "${project.buildDirName}/test-results",  
             format: 'xml', srcdir: "src/main/java", datafile: cobSerFile)  
       }  
     }  
   }  
 }  
 task runSonar << {  
   ant.taskdef(name: "sonar", classname: "org.sonar.ant.SonarTask", classpath: configurations.sonarLibs.asPath)  
   ant.sonar(workdir: ".", key: "no.sample:sampleapp", version: '1.0') {  
     sources {  
       path(location: "src/main/java")  
     }  
     tests {  
       path(location: "src/test/java")  
     }  
     // tell sonar to use existing cobertura reports  
     property(key: "sonar.dynamicAnalysis", value: "reuseReports")  
     property(key: "sonar.cobertura.reportPath", value: file("build/test-results/coverage.xml"))  
   }  
 }  

So to run the sonar goal with coverage I basically make the following command;

$gradle build runSonar -Pcoverage=on



And voila: