mandag 26. mai 2014

A Groovy Light Table client - Step 4: Exploring new avenues with Gradle

Background

This is the fourth post in my series "A Groovy Light Table client". A blog series about steps I take when trying to build a Groovy plugin for Light Table.

Initial ponderings

Gradle ships with a Tooling API that makes it fairly easily to integrate with your Gradle projects. Initially I thought that Gradle integration should be a separate plugin that other jvm language plugins could depend on, starting with the Groovy plugin. However after much deliberation I decided to start out with bundling the gradle integration with the Groovy plugin. There is certainly a degree of selecting the easy option to that decision. However I still consider the integration exploratory and I'm not sure how it will pan out. I've settled for a strategy of keeping it logically fairly separate, with a mind to separating gradle specifics out to its own plugin when things become clearer.

Classpath Integration for Groovy "REPL"

In part 3 I talked about some REPL like features where variables that result in bindings are stored in a editor session and used as input to the next evaluation. Since then I've also added the feature of caching method definitions (albeit as closures so I'm sure there are gotchas to that approach as well). 

Anyways wouldn't it be nice If I could also explore my project classes and my projects third party library dependencies in a REPL like fashion ? Hence the idea of providing a Gradle integration. With the Tooling API I should be able to retrieve a class path. So this is where i started. 
Before anyone potentially asking; I will not bother with maven or ant at any point in time, I'll leave that to someone else.

Retrieving the class path as a list from a Gradle project

// Step 1: Connecting to project
def con = GradleConnector.newConnector()
  .forProjectDirectory(projectDir)
  .connect()

// Step 2: Get hold of a project model, for now a IdeaModel provides what we need
def ideaProject = con.model(IdeaProject)
  .addProgressListener(listener)
  .get()

// Step 3: Get list of dependencies
def deps = ideaProject.children
  .dependencies
  .flatten()
  .findAll { it.scope.scope == "COMPILE" }
  .collect {
    [
      name   : it.gradleModuleVersion?.name,
      group  : it.gradleModuleVersion?.group,
      version: it.gradleModuleVersion?.version,
      file   : it.file?.path,
      source : it.source?.path,
      javadoc: it.javadoc?.path
    ] 
  }

def classpathList = deps.file + [new File(projectDir, "build/classes/main").path]

The above code is actually wrapped in a class. Connection and model instances are cached for performance reasons.
  1. We connect to our gradle project. If the project ships with a gradle wrapper (which it should IMO), the gradle connector will use that version (download the distribution even if need be). Otherwise it will use the gradle version of the tooling-api. At the time of writing that's 1.12
  2. The tooling api doesn't really expose as much information by default as you might wish. However it ships with an IdeaModel and an EclipseModel that provides what we need for the purposes of creating a class path. As an Idea user the IdeaModel seemed the right choice ! There is also added a progress listener, which is a callback from the api reporting progress. The progress listener returns each progress event as a string to Light Table so that we can display progress information
  3. We basically navigate the model and extract information about dependencies and put it in a list of maps for ease of jsonifying (useful later !). The location of our projects custom compiled classes are added manually to the class path list (ideally should have been retrieved from the model as well...)

Adding the class path list to our groovy shell before code invocation

private GroovyShell createShell(Map params) {
def transform = new ScriptTransform()
def conf = new CompilerConfiguration()
  conf.addCompilationCustomizers(new ASTTransformationCustomizer(transform))
  conf.addCompilationCustomizers(ic)

  if(params.classPathList) {
    conf.setClasspathList(params.classPathList)
  }

  new GroovyShell(conf)
}


Its basically just a matter of adding the class path list to the CompilerConfiguration we initialise our GroovyShell with. Sweet !
Voila your groovy scripts can invoke any class in your project´s class path.


This addition basically resulted in version 0.0.4

Reporting progress

Groovy

class ProgressReporter implements LTProgressReporter {
    final LTConnection ltCon

    ProgressReporter(LTConnection ltCon) { this.ltCon = ltCon }

    @Override
    void statusChanged(ProgressEvent event) {
        if (event.description?.trim()) {
            reportProgress(event.description)
        }
    }

    void reportProgress(String message) {
        ltCon.sendData([null, "gradle.progress",[msg: message]])
    }
}


  • statusChanges is called by gradle (LTProgressReporter extends the Gradle ProgressListener interface)
  • reportProgress sends the progress information to Light Table 

Light Table

(behavior ::on-gradle-progress
  :desc "Reporting of progress from gradle related tasks"
  :triggers #{:gradle.progress}
  :reaction (fn [this info]
              (notifos/msg* (str "Gradle progress: " (:msg info)) {:timeout 5000})))

The progress behaviour just prints a message to the Light Table status bar.


Executing Gradle Tasks 

There are two parts to this puzzle. One is to retrieve information about what tasks are actually available for the given project. The other is to actually invoke the task (tasks in the future).

Listing tasks Groovy/Server

 // Step 1: Retrieve generic Gradle model
def gradleProject = con.model(GradleProject)
  .addProgressListener(listener)
  .get()

// Step 2: Get list of available tasks
gradleProject.tasks.collect{
  [
    name: it.name,
    displayName: it.displayName,
    description: it.description,
    path: it.path
  ]
}

// Step 3: Send task list to client (omitted, you get the general idea by now !)

Listing tasks in Light Table

The list of tasks is actually retrieved by the Light Table plugin once you select to connect to a gradle project. Furthermore the list is cached in an atom.

(behavior ::on-gradle-projectinfo
  :desc "Gradle project model information"
  :triggers #{:gradle.projectinfo}
  :reaction (fn [this info]
              (object/merge! groovy {::gradle-project-info info})
              (object/assoc-in! cmd/manager [:commands :gradle.task.select :options] (add-selector))))

When the groovy server has finished retrieving the tasks (and other project info) the above behaviour is triggered in Light Table:

  • We store the project info in our Groovy object (an atom)
  • We also update the command for selecting tasks with the new list of tasks. See the section below for details.
(behavior ::set-selected
  :triggers #{:select}
  :reaction (fn [this v]
              (scmd/exec-active! v)))

(defn selector [opts]
  (doto (scmd/filter-list opts)
    (object/add-behavior! ::set-selected)))

(defn get-tasks []
  (->@groovy ::gradle-project-info :tasks))

(defn add-selector []
  (selector {:items (get-tasks)
             :key :name
             :transform #(str "<p>" (:name %4) "</p>"
                              "<p class='binding'>" (:description %4) "</p>")}))

(cmd/command {:command :gradle.task.select
              :desc "Groovy: Select Gradle task"
              :options (add-selector)
              :exec (fn [item]
                      (object/raise groovy :gradle.execute item))})


The above code adds a sub panel to the default sidebar command panel. When you select the command :gradle.task.select it will show a child panel listing the tasks from the get-tasks function.


;; Behavior to actually trigger execution of a selected task from the list above
(behavior ::on-gradle-execute
  :desc "Gradle execute task(s)"
  :triggers #{:gradle.execute}
  :reaction (fn [this task]
              (clients/send
                (clients/by-name "Groovy")
                :gradle.execute
                {:tasks [(:name task)]})))

Once you have selected a task the above behaviour is triggered. We get hold of an editor agnostic groovy client and send an execute task message with a list of task (currently always just one). The data we send will be extended in the future to support multiple tasks and build arguments.

Server side Task execution 

// Generic execute task function
def execute(Map params, Closure onComplete) {
    def resultHandler = [
        onComplete: {Object result ->
            onComplete status: "OK"
        },
        onFailure: {GradleConnectionException failure ->
            onComplete status: "ERROR", error: failure
        }
    ] as ResultHandler


    con.newBuild()
    .addProgressListener(listener)
    .forTasks(params.tasks as String[])
    .run(resultHandler)
}

Here we use the asynch features of the Gradle Tooling API. Executing a task may actually take a while so it certainly makes sense. Callers of the execute method will receive a callback (onComplete) once task execution is completed successfully (of failed).


projectConnection.execute(params) {Map result ->    
    ltConnection.sendData([
        null,
        result.status == "ERROR" ? "gradle.execute.err" : "gradle.execute.success",
        result
    ])
}

We invoke the execute method with a closure argument and return the results (success/failure) back to Light Table.


This brings us pretty much up to version 0.0.5

Summary

Well we covered a lot of ground here. We can now call any class that's in your Gradle project's class path from a groovy editor in Light Table. We've also started on providing Gradle features that are language agnostic. Starting with support for listing and executing tasks in your gradle project.
We've added decent progress reporting and performance seems to be pretty good too. Looks like we have something we can build further upon !

I have lots of ideas; Infinitesting,  single test with inline results, compile single file, grails integration ? etc etc. I also really want to show project dependencies in a graph. However before I can do any of those things I need to extend the tooling api with custom models ... and/or maybe I should see if I can contribute to the gradle project in extending the tooling-api with a richer generic project model. 

We'll have to wait and see. Next week I'm off to gr8conf.eu in Copenhagen. Really looking forward to meeting up with all the great Groovy dudes/dudettes.  And who knows maybe the hackergarten evening will result in something new and exciting ! 

mandag 12. mai 2014

A Groovy Light Table client - Step 3: Running out of quick wins

Background

This is the third post in my series "A Groovy Light Table client". A blog series about steps I take when trying to build a Groovy plugin for Light Table.

Running out of quick wins

After 0.0.2 of the plugin was released I was pretty happy. I had something that I could actually use as an alternative Groovy Console. However I was keen to keep up the flow so I figured I would try to implement an autocomplete feature. The task proved rather daunting. Not so much from the Light Table side of things, but rather from the Groovy side. First I tried to see if I could reuse anything from GroovySh, but that didn't look to promising. After that I tried to get my head around whether I could somehow reuse something from IntelliJ or Eclipse. I failed to see the light so I gave up that endeavour. Secondly I tried to see if there was an easy way to provide an inline documentation feature. Sadly I couldn't find much reusable and something with a small foot print here either. Someone should make a REST based doc search feature for Groovy Docs one day !

I turned my attention to a couple of other plugins that I thought would be useful for Light Table. I created a Buster.JS plugin InstaBuster for easily running JavaScript tests. I also created a snippets/code templates plugin lt-snippets and some snippet collections, among them a small collection of groovy snippets.

There is just no way I could ever compete with the mainstream IDE's, but then again that was never really the original intention. But even with limited capacity it should still be possible to provide useful groovy support and maybe even something fairly unique within the super hackable framework of Light Table.

A (small) step towards a Groovy REPL

After working with Light Table and the Clojure/ClojureScript REPL I have grown very fond of that exploratory nature of working. Is there anything I could do with the plugin to give a more REPL like feel ? Well a small but helpful step would be to be able to remember what I previously have evaluated ...

Groovy Bindings

A simple but still useful mechanism would be to cache bindings from script execution between evals.

// from the evalGroovy method
def evalResult = scriptExecutor.execute(data.code, clientSessions.get(currentClientId))
clientSessions.put(currentClientId, evalResult.bindings)

Each editor in Light Table gets it's own unique Id. So I just created a simple cache "ClientSessions" that hold a map of binding variables, mapped my that Id. When executing a script the current binding variables are applied to the script and after the script has executed the resulting binding variables are added/replaced in the cache. Dead simple really.

Clearing bindings - Light Table

I figured it would be handy to be able to clear any stored bindings. So a new command and behaviour was created in Light Table

;; Behavior for clearing bindings
(behavior ::on-clear-bindings
          :desc "Clear cached bindings for this editor"
          :triggers #{:on.clear.bindings}
          :reaction (fn [editor]
                      (let [info (:info @editor)
                            cl (eval/get-client! {:command :editor.clear.groovy
                                                    :origin editor
                                                    :info info
                                                    :create try-connect})]
                          (clients/send cl
                                        :editor.clear.groovy info
                                        :only editor))))

;; Command that allows a new keyboard bindable action for invoking the behaviour above
(cmd/command {:command :clear-bindings
              :desc "Groovy: Clear bindings for current editor"
              :exec (fn []
                      (when-let [ed (pool/last-active)]
                        (object/raise ed :on.clear.bindings)))})

The command retrieves the currently active editor and triggers the behaviour. The behaviour retrieves a client connection (or creates one if one shouldn't exist) and calls the server (groovy).


// Wiring up the behaviour in groovy.behaviors
:editor.groovy [:lt.plugins.groovy/on-eval
                     :lt.plugins.groovy/on-eval.one
                     :lt.plugins.groovy/on-clear-bindings
                     :lt.plugins.groovy/groovy-res
                     :lt.plugins.groovy/groovy-err
                     [:lt.object/add-tag :watchable]]

The final piece of the puzzle from the Light Table side is to attach the behavior to the :editor.groovy tag. This enables the behavior to be available from any editor that is tagged with this tag.

Clearing bindings - Groovy

// The command dispatch got a new command
case "editor.clear.groovy":
   clientSessions.clear(currentClientId)
   break;

The code above will just nuke any stored binding variables.

Conclusions

A tiny step that allows you to eval groovy expressions step by step. Anything that results in a binding is stored between evals. Obviously it's a bit limited in that you'll run into the familiar trap of trying to use def and be surprised(/annoyed) that it won't remember that or if you define a class it won't remember that either. It's probably possible to cater for some of these traps, but maybe not within the realms of a quick win. 

Anyways the end result is Version 0.0.3 !

Next steps

Firstly there is a Screencast brewing. After that I think a Light Table Gradle plugin is coming before the Groovy plugin gets any further updates. A pluggable Gradle plugin would enable the Groovy plugin to quite easily get the class path for you project. This would allow you to  explore your projects code in a REPL (-like) way. Exploratory testing FTW !




onsdag 7. mai 2014

Creating and Using Snippets in Light Table


An introduction on how to create and use simple and more elaborate snippets with the lt-snippets Light Table plugin. If you can't see the video above here's the link directly to youtube