Saturday, October 13, 2007

Java 5 features in Groovy

Groovy 1.1 RC-1 has been released yesterday and I thought this is a perfect time for giving a complete list of all the Java 5 features that are available in Groovy and how much of it. I will not explain the Java 5 features itself, I will just show what you can do and what not. So let us start right away:

Variable Arguments

Groovy is able to use this for a long time now. Since Groovy does not know exactly which method it will invoke at compile time we use a vargs (variable arguments) when selecting a method for invocation. but not only the vargs Java has...I mean these things with the funny little triple dotted type as last parameter, no Groovy just requires an array there. so if you want to use a vargs method from Groovy just do it like in Java. vargs methods defined in Groovy can be used as such from Groovy too. Currently Groovy does not add the special modifier for vargs to the array, so it won't be a vargs method for Java, just a method taking an array as last parameter. I just filled an issue for that and it will be fixed before the 1.1 release. Ah yes, before I forget... of course you can use the triple dot notation instead of using an array in Groovy too. The tripple dot notation will be exactly the same as an array as last parameter.

To use vargs no java5 vm is needed from the groovy side.

New For-Loop

Groovy had always a for-loop in that style, only we don't use only the ":", we use allow also the usage of the keyword"in" at that place. The Groovy for-loop asks for an iterator by calling the iterator() method, which is available on any object. Groovy adds a dynamic iterator() method to the MetaClass for every class as long as the class does not define it on its own.

To use the new for-loop no java5 vm is needed from the groovy side.

Covariant Return Types

It took a while before Groovy was able to use these (available since 1.1 RC-1). Originally we wanted to be able to overload a method if the return type differs, not override it. But it seemed to make no sense to differ here from Java. So Groovy follows here Java now and if you write a subclass, that has a method that matches a method in the parent class in name and argument types, but has a different return type, then covariants kick in. If the the new method has a return type derived from the old method, then the new method overrides the old method. If it does not derive from that, then you get a compile time error. As an example:

class A{
Object foo(){1}
class B extends A{
String foo() {2}
def b=new B()
To use covariant return types no java5 vm is needed from the groovy side.


Again these are available for some time now. The syntax is equal to Java. Even though you can define interfaces in Groovy you can not define an annotation in Groovy yet. But using Annotations in Groovy is basically the same as in Java.

If annotations are used Groovy needs to produce java5 bytecode.


Well... originally we didn't want to add them. They are complicated and look kind of surplus in a dynamic world. But some applications can make use of generics. for example List<book> books contains much more informations for a persistence layer like for example JPA. But maybe I should first explain that we do not support checks where type erasure is used. For example if you define a script like
List<String> list = ["a","b"]
list << 1
then Groovy won't complain. This is simply ignored. But if you use generics for a method, a field or in the class header (class X<A,B> extends Y<A> implents Z<B>), then Groovy will write the information in bytecode the same way as Java would do. That means if you use the Groovy class in Java, you have the same pain... ehm pleasure... what ever... as with a class defined in Java. With the 1.1 RC-1 release Groovy now also checks the header information, so you won't be able to write class X extends Map<String> anymore and get a compile time error now.

If generics are used Groovy needs to produce java5 bytecode. Groovy needs to be build with >java5 for this (only important if you build Groovy by yourself).

Generics+Covariant Return Types

I already covered these two, but I think the combination is worth an extra word. If you have a program like this
class A<T> {
T foo(T t) {1}
class B extends A<Long> {
String foo(Long l) {"2"}
then Groovy will do as Java and throw a compile time error. If you use raw types you will see that the following script for example works:
class A<T> {
T foo(T t) {1}
class B extends A {
String foo(Long l) {"2"}
def b = new B()
assert Object( )) == 1
assert 1) == "2"
If used with raw types the method A#foo returns Object and takes Object. Since B#foo takes a Long Groovy will not care about the covariant return types and compile it. So you end up with two version of foo in B, one derived from A returning 1 and one from B returning 2. If the usage is like in the first case above, then A<Long> "kind of" defines a foo method taking a Long and returning a Long. I say "kind of", because the class does not really define that method, it is just the view B gets of A. Anyway, since B defines also a foo(Long) the covariant return type function kicks in, but finds that the return types String and Long are incompatible. So it will give a compile time error. The following script would for example compile:
class A<T> {
T foo(T t) {1}
class B extends A<Llon>> {
Long foo(Long l) {2}
def b = new B()
assert 1) == 2
The foo defined in B will now override the foo defined in A.


Groovy now supports with 1.1-rc-1 also simple enums. With simple enums I mean you can declare them in Groovy, you can even declare them inside a nnormal class as long as you don't use references from the surrounding class. You can also define methods/fields/properties that will be in all enums. This should cover almost all of the features you need for enums. What we do not support is writing code that is special to one enum value. For example overwritng a method in the enum value, or addding a method/field/property. I have never seen such enums in real live, so I think it is not too important.... On the other hand I haven't seen much enums at all and my picture might be wrong. Anyway, Groovy is most possibly not going to support these enums with additional methods.
Groovy enums will differ a little from Java enums, they will implement GroovyObject and thus you can have a meta class per enum value. That relativates the missing feature with additional methods a bit I think.

If enums are used Groovy needs to produce java5 bytecode.

Asking The User!

I hope all the people out there, that like Groovy so much, will help us in finding bugs for these features. Some features are quite new, sometimes the tests lack fantasy (especially if written at 3 o'clock in then morning) and do not cover all needed cases. Sometimes something is simply misunderstood. I encourage all users to find the bugs and enter issues for them so we can deliver a pleasant 1.1 release to you.

Wednesday, July 18, 2007

About SwingBuilder

Disclamer: I will use the term "closure" quite often here and experts will say they are not closures. I still call them closure in the sense, that they are instances of groovy.lang.Closure. So If I say closure I don't mean that functional thing ;)

This time I thought I should write some things about Groovy SwingBuilder and assumptions people seem to make about it.


First thing you need to know is that SwingBuilder is a builder... that might be obvious, but it implies, that if I do a method call in the builder structure, then the builder will handle that call and map the method names to certain actions.

Now in Groovy we have this class BuilderSupport, that you can use to map structures in a builder. Personally I don't like that class much, because the logic looks more complicated than needed, but it fits very general cases. Anyway, the class tries to map method calls in the builder structure to calls of createNode in the builder class. There are several of them, each responsible for a certain case controlled by your method call. The most important fact here is that if your last argument is a closure, then this closure will not be part of the creatNode call, instead the closure will be used by the builder directly. I guess it is best to show examples:

def builder = new MyBuilder()
builder.start {
methodWithClosure {
methodWithNormalArgument("I am a argument I guess")
methodWithClosure a normal method call with one argument, that is the closure containing the method call with methodWithMap. methodWithClosure is now mapped to createNode(Object), the object there is the method name "methodWithClosure" as String. methodWithMap is mapped to createNode(Object,Map), where the first is again the method name and the map is our [foo:"bar"]. If you combine one normal argument and a map entries, then you get createNode(Object,Map,Object), where the last one contains your normal argument. And if there is no map and no closure, just a normal parameter, like with methodWithNormalArgument, then createNode(Object,Object) will be called. This logic supports only 1(!) normal argument, but that is enough in general.

After the createNode call of your choice is made the return value of that will be hold, I will call this currentNode. To connect the currentNode and its parent, which is done by setParent(currentNode,parentNode). now what is parentNode? Remember? we still have a closure to call. When we do, then our currentNode becomes parentNode and the new currentNode will have a parent. So the first time setParent is called we are not in a closure that belongs to the builder, which means the parentNode is null. If you would build a tree using this logic, then you would build the tree starting with the root and then adding node by node in I think it is called preorder traversal.

Architecture of SwingBuilder

SwingBuilder is making use of these methods in BuilderSupport. For each method call SwingBuilder creates a new instance of a bean we specified with the method call. So
frame(title:"I am a JFrame")
will create a new JFrame instance and set the property title. And as we just learned
frame(title:"I am a JFrame"){
label(text:"I am a JLabel")
will also create the JFrame, the property title will be set again, then the closure will be executed causing the label method to create a JLabel and the text property on that label is set. After that setParent is called with the first parameter being the JLabel and the second parameter being the JFrame.

The logic we stored in setParent will connect our frame with the label by frame.getContentPane().add(label). SwingBuilder#setParent knows several cases and handles adding a JMenuBar to a JFrame different from adding a JLable. This method and helper are around 100 lines, about 20% of SwingBuilder source.

So basically SwingBuilder is a builder that maps method calls to bean creation actions, using map arguments to init the beans and the closures to connect the created beans. It using a mapping method name -> bean class and contains itself nearly no methods you call when using SwingBuilder.

Names supported by SwingBuilder

If you are not sure if SwingBuilder supports a swing widget, just remove the J, keep the next letter in lower case and try it. For example JEditorPane becomes editorPane, JSplitPane becomes splitPane (both supported). But SwingBuilder does not only know widgets, it does also know layouts. there is usually no 'J', so just use the next letter in lower case, as in gridBagLayout, flowLayout or others. You can use the layout as normal method causing the layout property of the container to be set. I have often seen code like:
frame(layout:new FlowLayout()) {
but you can write that also as
frame() {
I like this version much better, because you do not need to import FlowLayout and can give the layout some options while keeping the frame call simple. Groovy supports all the normal layouts, even Box layout. Another special thing is maybe the method gbc, which is the same as the method gridBagCosntraints, which maps to GridBagConstraints. Maybe I should also mention TableLayout, which tries to implement the layout you know from the table tag in html. It needs tr and td calls to place the componentes... really just like in html. Take a look at alpahbetic widget list to get an ideas what you can do.

Another important link is extending SwingBuilder. It does not mention the possibility of simply subclassing the class SwingBuilder, but that should be obvious and was done for example by SwingXBuilder.

All in all it is a bit difficult to provide a documentation for SwingBuilder, because you still need to learn swing, SwingBuilder doesn't help you with that. And then it is just connecting instances of classes... For example when people ask how to attach an action to a JButton and I tell them to assign a closurey to the actionPerformed property, then I am not talking about a special property, actionPerformed is defined by the bean specification and assigning a closure to that property is a normal thing in Groovy.

New things in 1.1-beta2

We got complains that if I do
def frame = swing.frame(...) {
frame.visible = true
that the resulting gui will not be constructed in the EDT thread, but in the normal main thread. Now I am no Swing expert and I always assumed it makes no difference, but it seems that future changes in Java will need you to change in the EDT. And of course it is more clean that way too. So we eneded with adding two methods, the first is edt, which causes the attached closure to be executed while in EDT. the code looks then like
swing.edt {
def frame = frame(...) {
frame.visible = true
unlike many other methods available in SwingBuilder the edt method is a real method and no registered widget. the other method is static and called build. It will automatically create a new SwingBuilder instance and call the attached closure with that instance as parameter {
def frame = frame(...) {
frame.visible = true
build uses the edt method, so we build the GUI while in the EDT thread, just like before.

Future Plans

I think SwingBuilder is already a nice piece of work, but its evolution might not stop here. Currently I am thinking about integrating a Binding framework. That would some update logic to SwingBuilder, something you have to do all by yourself atm. For example imagine a label and a button and each time you press the button the label text should show a higher number. What do you do? You use a closure as actionPerformed for your button that increases a number and sets a new text for the label...
import groovy.swing.*
import groovy.swing.impl.*;
import javax.swing.border.EmptyBorder
import javax.swing.WindowConstants

def numClicks = 0
def state = {"Number of button clicks: $numClicks"}
def label {
def frame = frame (
title: "SwingBuilder Label Update",
defaultCloseOperation: WindowConstants.EXIT_ON_CLOSE
panel (border: new EmptyBorder(30, 30, 30, 30)) {
gridLayout(rows: 2, columns: 1, vgap: 10)
button (text: "I'm a button!",
mnemonic: "I",
actionPerformed: {numClicks ++; label.text = state()})
label = label (text: state())
frame.visible = true
note the closure state, that is called at different places? that's quite ugly I think. With a binding framework the code might become
import groovy.swing.*
import groovy.swing.impl.*;
import javax.swing.border.EmptyBorder
import javax.swing.WindowConstants

def model = new BindModel(numClicks:0) {
def frame = frame (
title: "Binding and SwingBuilder Test",
defaultCloseOperation: WindowConstants.EXIT_ON_CLOSE
panel (border: new EmptyBorder(30, 30, 30, 30)) {
gridLayout(rows: 2, columns: 1, vgap: 10)
button (text: "I'm a button!",
mnemonic: "I",
actionPerformed: {model.numClicks ++})
label (text: bind("Number of button clicks: $model.numClicks"))
frame.visible = true
as you see the need to keep an external closure vanished along with the need to keep a reference to the label. the way it will be used in the end is not yet sure, this is just a sketch based on a simple implementation I made.

While doing my "research" in this area I noticed that binding frameworks are not that well known. At last by the people I know. I myself didn't here about that before, but I am usually not doing much with swing. So I got a bit puzzled why people don't know these things if they can save so much code... but then I saw it. Some of these frameworks are producing rather cryptic code, that makes sense for the framework but is just plain hard to read. Your former models are now hidden in abstract constructs, but you still need to connect the things. So I guess SwingBuilder could find a more "natural" way by hiding all these things.

We will see.

Wednesday, July 04, 2007

Joint Compilation in Groovy

Note: This is not implemented since today, it is already some weeks old, but there was no information about it on the net... So I wrote this

When working with Languages like Groovy you naturally mix Groovy and Java all the time. But the problem then arises to compile the resulting monster. Especially the Eclipse plugin for Groovy makes you sometimes think your project will compile outside as nice as inside Eclipse. But it might not. Consider for example this case

class A {
B b;
class B extends A {}
And think A is written in Groovy, but B is written in Java, or the other way, A is written in Java, but B is written in Groovy. It is clear, that when you compile B, you need class A as well, because the class might be final or abstract or other things that need to be checked. But when you compile A, you will see that it refers B, which means you need to compile B as well. Now if this is one compiler we have no problem, but A and B are written in different languages with compilers not sharing their class information. That means we are stuck. And while this example looks a bit artificial, you get very fast into this situation in a larger project. Who would keep track of not referencing Groovy classes from the Java side to get the compilation done? It's disturbing.


Now the one solution would be to let the compiler share their class data. The Groovy compiler is able to do that, the eclipse compiler is able to do that, we might see a bright future for this in the future. But for example JavaC is not able to do this. At last I don't know how.

Another way is to create stubs for the Groovy classes, run the Java compiler with these stubs, then run the Groovy compiler and overwrite the generated stubs. Alex Tkachman was so free to show us Groovy people how to do this and provided a patch, we could us as base to get this version of the compilation running.

How it Works:

The details of the stub generation are not so important I think, they are created as Java files from the parse tree the Groovy compiler provides in a temporary directory and then feed to JavaC along with the normal Java files. You don't need to start the compiler yourself, the Groovy compiler will do this for you right after the parse phase and then continue with its normal compilation process.

Controlling JavaC:

Since we now have a combined compiler we of course want to use it in our build, but the problem is that javac would by default created a wrong bytecode version for us, we need 1.4 compatible bytecode, so source and target options are needed at last. I then decided to forward the options to the compiler from the command line of Groovy. So if you do
groovyc *.groovy *.java -j -Jsource=1.4 -Jtarget=1.4
You will get the java files compiled for 1.4. -j turns the joint compilation on , the -J parts are gving key-value pairs to the compiler. Using Options without value is also possible using the -F option, just without the equals part like -Fdeprecation. Anything the JavaC compiler supports can be dropped in there... Of course some special options like the VM memory size would not make sense since the VM is already created.

For the GroovyC Ant task the picture is a bit different. first I thought about a way to generically define attributes for the GroovyC task I can forward to JavaC... But me not being the Ant expert I gave this up and decided to do the following work around
<echo message="Groovyc of test code."/>
<java classname="org.codehaus.groovy.ant.Groovyc" fork="true" maxmemory="128M">
<pathelement path="${mainClassesDirectory}"/>
<pathelement path="${testClassesDirectory}"/>
<path refid="testPath"/>
<arg value="${testClassesDirectory}"/>
<arg value="${testSourceDirectory}"/>
<arg value="-j"/>
<arg value="-Jsource=1.4"/>
<arg value="-Jtarget=1.4"/>
Oh, that reminds me that the GroovyC task needs a fork ability. Anyway, that's when using the ant task from the command line. If you want to use it normally, then
srcdir="${mainSourceDirectory}" destdir="${mainClassesDirectory}"
jointCompilationOptions="-j -Jsource=1.4 -Jtarget=1.4"
can be used. Same game as on the command line.

What this solution can't do:

Yes, there is a downside. Ok, I think it is already a downside that we have to use a temporary directory, but another one is that we need to know all files we want to compiler before compilation. I don't think that is a problem when running a ant or maven based built, but for the typical usage on the command line, where you just compile your main class and the compiler will get a hold on all further classes will not work. To be more specific, it will not work when the Groovy compiler needs to get an additional Groovy class and the Java compiler would need that class too. That's because in this case no stub will be created and thus the java compiler will fail telling you it can't find a that class. On the other hand GroovyC works with the resulting class files, so if a Groovy class refers a Java class and JavaC did not compile it, then GroovyC won't be able to compile it either... well, ok, just give the compiler all needed files ;)

Future Work:

The current implementation uses JavaC directly a nice framework would be nice here to have more than just this compiler. And there is for example JCI, but JCI seems not to support options... well we need to take another look at it, maybe it supports enough. On the other hand I am thinking about integrating the JavaC task from ant. in that case we could maybe use the normal task as nested element (with some tweaks) and have all the abstraction to different compilers ant allows. Of course by directly using the Eclipse compiler (it is usable outside the IDE) we could let the compiler share class data and then compile files that are not part of the file list given at runtime.

But I think the ant task version will make it. the work around with the "-j -J -F" options might then vanish.

But none the less, have fun with the upcoming
Groovy 1.1 beta 2

Tuesday, March 27, 2007

Visitor Pattern in Groovy

Shame on me, such a long time without update. I was too busy it seems. I just wrote a small article about how to use the visitor pattern in Groovy. Enjoy the Visitor Pattern in Groovy and ignore the misspellings ;)

If you think I should mention other patterns as well, tell me please.

Saturday, January 06, 2007

AST macros and mixins

Hi all,

I know my last article was written some time ago, but I was busy with Groovy 1.0 bug times. Anyway. This is another part of my "beyond Groovy 1.0" series, only that we are now officially in that era and I think I no longer need to prefix the title with that.

So my next wild idea is about a macro mechanism for Groovy. But not the kind of macros you know from C or such, no I think of macros rewriting parts of the AST used by our compiler. So the idea is not really something new, it is something the compiler does already provide. The new thing is to let the compiler do the integration steps for you automatically. No need to add a PhaseOperation to the compiler or such.

No, I have not really thought of a syntax yet, let us think about this as a case study and not as a final draft or something. Ok, back to title. let us say, there is a statement like the import statement and let us call it macro. So when we want to use a macro we simply tell the compiler this by:

import macro Foo

or by
import macro Foo as bar

and let us assume the compiler can use them like a method:
bar {
// a block containing code

looks much like the dynamic features we already provide, but in fact the compiler recognizes "bar" as macro and does not produces a method call with a closure instance for this, no, instead the compiler takes the logic provided by our Foo class and applies it to this part of the code.

Why should we do that? One thing I always disliked in Groovy is the huge amount of keywords. Do they really have to be keywords? One example for this is synchronized. Usually you do something like:
synchronized (object) {
// code using object

But isn't that almost our macro "bar" from before? It is, not only almost. In fact it is really alike, the compiler would transform that into
synchronized (object, {  ...t })

just like the bar example
bar ({...})

it is no different. So that would mean we could remove the keywords "for", "while", "assert" and "synchronized". Ok, not much... but still it is a step forward. That would still mean that the do-while loop is not supported, but maybe there is a solution for this too.

How would Foo possibly look? Well I guess something like:
class Foo {
static phase = Phases.CONVERSION
def visit(ASTNode[] contextPath, SourceUnit source) {
// ... transformation code here

The compiler would take a look at the Foo class ensure that there is a default constructor and a static field named phase. Then the compiler would create a new instance of the class and add it to the phase operations it already provides, plus a filter to ensure only the correct classes are affected by this. So if the compiler compiles the file it goes along the AST until it finds the macro and then does execute it.

That's pretty easy to implement, really no big deal, but the impact might be big. I think such a construct would be the first step to unify the Groovy grammar parts and to remove special constructs. Well, the thing really needed for this is a way to exchange/remove statements in the AST. That is currently not possible. It is only possible to transform expressions, but not statements. But well, that is no problem we can't solve, is it?

The mentioned keywords earlier could be seen as macros themself. That doesn't mean that we can remove these statements from the AST, but it would mean a programmer could define additional keywords as he likes. For example a macro transforming a closure into a SQL statement. We already have this, but wouldn't it be exciting to have that at compile time? I mean with different checks and without runtime impact. the macro could use any expression, for example
  sql {
select row1,row2
from tx
where id==26

I am also thinking about SODA queries or queries for GORM. The above can't be interpreted using a normal closure, because I used 3 statements that are interpreted as method calls. There is for one select(row1,row2), then from(tx) and where(id==26). So to correctly interpret this we have to use the getClassNode method in MetaClass, but that relies on runtime analyzes of the source file atm. Yes, we are going to change that, but it means to store the source inside the closure in compressed form or else we wouldn't be able to ensure the correct interpretation. With these AST makros we are able to compile that directly into the goal construction. I mean even different SQL dialects could be covered by this.

But SQL is just a example. I used it only to illustrate the abilities a little. I know that this article does only give a little idea about how it might work and writing this doesn't mean it will go into any version of Groovy. But I think the idea is quite simple and yet powerful.

But this is not all... AST handling is complicated and not really something that would make your day. so I think about going a step further, in fact I am planining to adding this pretty soon. I am talking about AST level mixins.

"Ok, what's that again?" you may ask. I think of defining a chunk of code as normal groovy script and a defined way ofr a phase operation to use that code to modify the AST. This chunk of code does itself not contain any AST handling code, it will just be used as archetype for the compiler to produce the real code. This might be a simple thing as
class X {
long id

causing the compiler to add a field named id to the currently processed classNode, or something more complex. Well we have a problem when it comes down to bytecode instruction. For example if we want to replace a for loop we would have to define jumps. So, as long as there is no mechanism to represent this, there is no way this transformation by example will work for all kinds of macros too. But well, this is like doing research. At the beginning you have only a vague idea of how things might work. you do experiments to see if your expectations are correct and you develop new ideas if they do not or if the experiments are uncovering no facts.

But still, the idea is exciting somehow. I am sure another language does already have a construct like that... well I am thinking about LISP here. I wouldn't dare to compare this to the macros in LISP (I am talking about Common LISP), but possibly they will become alike in the future after doing same practical work. LISP does have a more easy part here, because LISP does have a much simpler syntax than Groovy. Doing structural work in such a language is naturally more enjoyable than in a complex language like Groovy. But I still think it can be done.

Btw, this is no replacement for DSLs looking like natural language, but that is more a syntax issue, than an issue of AST macros.

Wednesday, January 03, 2007

Groovy 1.0 is here!

16 hours ago Groovy 1.0 was released. The work of 3,5 years. So many discussions, so many code was produced, removed and reworked.

In my eyes this is not the final version of Groovy, nor is it perfect. But it is important to give a stable version to the world out there. I hope many people will like that Groovy. But that doesn't mean Groovy wouldn't evolve anymore. There are many ideas we can try out now since we do not any longer have to focus on the release. In the release notes you will find not much this time, we fixed a number of bugs, but these are more like minor adjustments. But Groovy In Action and do show increasing interest.

The next months will be spent to improve the documentation and to really start the JSR process. Then Groovy will be really like a standard and get a good documentation.

New Layout

Since the changes on my blog wasn't working properly, so I decided it is time to get a new layout. Making custom layouts here is not the most nice thing to do, because I always have problems with layout support pictures. But putting them in a post and reusing the link to the image allows me to have layout pictures as the two on top, that are nearly invisible. The layout is not perfect and my javascript for keywords is no longer working, but I think it is a good start. A multicolumn layout made in css can really be difficult to do. This page currently consists of about 4 columns, 2 for images, 1 for the sidebar and 1 for the content itself.