AST macros and mixins

Hi all,

I know my last article was written some time ago, but I was busy with Groovy 1.0 bug times. Anyway. This is another part of my "beyond Groovy 1.0" series, only that we are now officially in that era and I think I no longer need to prefix the title with that.

So my next wild idea is about a macro mechanism for Groovy. But not the kind of macros you know from C or such, no I think of macros rewriting parts of the AST used by our compiler. So the idea is not really something new, it is something the compiler does already provide. The new thing is to let the compiler do the integration steps for you automatically. No need to add a PhaseOperation to the compiler or such.

No, I have not really thought of a syntax yet, let us think about this as a case study and not as a final draft or something. Ok, back to title. let us say, there is a statement like the import statement and let us call it macro. So when we want to use a macro we simply tell the compiler this by:

import macro Foo

or by

import macro Foo as bar

and let us assume the compiler can use them like a method:

bar {
  // a block containing code
}

looks much like the dynamic features we already provide, but in fact the compiler recognizes "bar" as macro and does not produces a method call with a closure instance for this, no, instead the compiler takes the logic provided by our Foo class and applies it to this part of the code.

Why should we do that? One thing I always disliked in Groovy is the huge amount of keywords. Do they really have to be keywords? One example for this is synchronized. Usually you do something like:

synchronized (object) {
  // code using object
}

But isn't that almost our macro "bar" from before? It is, not only almost. In fact it is really alike, the compiler would transform that into

synchronized (object, {  ...t })

just like the bar example

bar ({...})

it is no different. So that would mean we could remove the keywords "for", "while", "assert" and "synchronized". Ok, not much... but still it is a step forward. That would still mean that the do-while loop is not supported, but maybe there is a solution for this too.

How would Foo possibly look? Well I guess something like:

class Foo {
  static phase = Phases.CONVERSION
  def visit(ASTNode[] contextPath, SourceUnit source)  {
    // ... transformation code here
  }
}

The compiler would take a look at the Foo class ensure that there is a default constructor and a static field named phase. Then the compiler would create a new instance of the class and add it to the phase operations it already provides, plus a filter to ensure only the correct classes are affected by this. So if the compiler compiles the file it goes along the AST until it finds the macro and then does execute it.

That's pretty easy to implement, really no big deal, but the impact might be big. I think such a construct would be the first step to unify the Groovy grammar parts and to remove special constructs. Well, the thing really needed for this is a way to exchange/remove statements in the AST. That is currently not possible. It is only possible to transform expressions, but not statements. But well, that is no problem we can't solve, is it?

The mentioned keywords earlier could be seen as macros themself. That doesn't mean that we can remove these statements from the AST, but it would mean a programmer could define additional keywords as he likes. For example a macro transforming a closure into a SQL statement. We already have this, but wouldn't it be exciting to have that at compile time? I mean with different checks and without runtime impact. the macro could use any expression, for example

  sql {
    select row1,row2
    from tx
    where id==26
  }

I am also thinking about SODA queries or queries for GORM. The above can't be interpreted using a normal closure, because I used 3 statements that are interpreted as method calls. There is for one select(row1,row2), then from(tx) and where(id==26). So to correctly interpret this we have to use the getClassNode method in MetaClass, but that relies on runtime analyzes of the source file atm. Yes, we are going to change that, but it means to store the source inside the closure in compressed form or else we wouldn't be able to ensure the correct interpretation. With these AST makros we are able to compile that directly into the goal construction. I mean even different SQL dialects could be covered by this.

But SQL is just a example. I used it only to illustrate the abilities a little. I know that this article does only give a little idea about how it might work and writing this doesn't mean it will go into any version of Groovy. But I think the idea is quite simple and yet powerful.

But this is not all... AST handling is complicated and not really something that would make your day. so I think about going a step further, in fact I am planining to adding this pretty soon. I am talking about AST level mixins.

"Ok, what's that again?" you may ask. I think of defining a chunk of code as normal groovy script and a defined way ofr a phase operation to use that code to modify the AST. This chunk of code does itself not contain any AST handling code, it will just be used as archetype for the compiler to produce the real code. This might be a simple thing as

class X {
  long id
}

causing the compiler to add a field named id to the currently processed classNode, or something more complex. Well we have a problem when it comes down to bytecode instruction. For example if we want to replace a for loop we would have to define jumps. So, as long as there is no mechanism to represent this, there is no way this transformation by example will work for all kinds of macros too. But well, this is like doing research. At the beginning you have only a vague idea of how things might work. you do experiments to see if your expectations are correct and you develop new ideas if they do not or if the experiments are uncovering no facts.

But still, the idea is exciting somehow. I am sure another language does already have a construct like that... well I am thinking about LISP here. I wouldn't dare to compare this to the macros in LISP (I am talking about Common LISP), but possibly they will become alike in the future after doing same practical work. LISP does have a more easy part here, because LISP does have a much simpler syntax than Groovy. Doing structural work in such a language is naturally more enjoyable than in a complex language like Groovy. But I still think it can be done.

Btw, this is no replacement for DSLs looking like natural language, but that is more a syntax issue, than an issue of AST macros.

Groovy 1.0 is here!

16 hours ago Groovy 1.0 was released. The work of 3,5 years. So many discussions, so many code was produced, removed and reworked.

In my eyes this is not the final version of Groovy, nor is it perfect. But it is important to give a stable version to the world out there. I hope many people will like that Groovy. But that doesn't mean Groovy wouldn't evolve anymore. There are many ideas we can try out now since we do not any longer have to focus on the release. In the release notes you will find not much this time, we fixed a number of bugs, but these are more like minor adjustments. But Groovy In Action and GroovyGrail.com do show increasing interest.

The next months will be spent to improve the documentation and to really start the JSR process. Then Groovy will be really like a standard and get a good documentation.

New Layout

Since the changes on blogger.com my blog wasn't working properly, so I decided it is time to get a new layout. Making custom layouts here is not the most nice thing to do, because I always have problems with layout support pictures. But putting them in a post and reusing the link to the image allows me to have layout pictures as the two on top, that are nearly invisible. The layout is not perfect and my javascript for keywords is no longer working, but I think it is a good start. A multicolumn layout made in css can really be difficult to do. This page currently consists of about 4 columns, 2 for images, 1 for the sidebar and 1 for the content itself.

ChitChat with the Groovy Compiler

Ever wished to be able to influence the compiling process in Java?

To do this you usually don't influence the compiler directly, you use AOP tools with strange syntax rules and extra files for configuration or you are working on the bytecode level and sue the instrumentation API. But these technics have their limits. To be able to instrument a class you need a ready compiled, the verifier passing class. What if you can't produce such a class?

But ok, let us go some steps back. Programming by convention is in all mouths, so let us look at adding a method depending on the class name. I guess people would use a inter-type declaration in AspectJ to add methods to a certain class. The problem here is, we don't know the class before. We react to the name of the class and only if the name fits, we add the method.

Another thing is adding imports. There is no point in adding imports in a class file, all class names in a class file are already resolved against the imports. Adding another import would be pointless. But if you want to let people write a small peace of code where they can forget about importing every class they use, then we are talking about adding something to what the compiler gets as input. The most easy solution would be to use some kind of macro system. Expand the macros and give the result in the compiler. But then, what is about the line numbering? Exceptions will show useless information about the position in the source file, compiler errors will point to invalid places. That's a bit bad.

Compiler are normally working on an AST, an abstract syntax tree. Such a tree represents the source as a tree, each node is a part logic created by the source. For example, the expression "foo.bar()" is in Groovy a method call expression with foo as the object the call is made on, bar the name of the method and an empty number of arguments. Such a method call expression is a node in the AST. I am afraid that explaining all the expression types and statements in Groovy would be too much for this small article. I suggest to look into the packages org.codehaus.groovy.ast.expr, org.codehaus.groovy.ast.stmt and org.codehaus.groovy.ast. Of course having around 35 expressions and 19 statements doesn't make this task very nice, but it is needed. Groovy uses the visitor pattern to visit the classes. That makes it a bit problematic for people how are not used to that. but it really is nice to have that.

Each phase of the compilation process after the AST is created consists of a visitor travaling along the AST replacing or modifying nodes. Btw, the Groovy AST is for example used in Eclipse for the outline.

Now imagine you could change the AST by yourself. Rename methods, add code, add methods... yes, right, we wanted to add a method if the class name fullfills a certain convention.

Now the Groovy compiler offers here some hooks you can use to change what the compiler will produce. What I will explain now counts only for the case that a file is compiled using the GroovyClassLoader, for example because a script is found in the classpath or because we use the GroovyShell to execute a script. The basic process to creat a compilation unit, that will hold the AST and other information, add a source to the compilation unit and then start the compilation. By subclassing GroovyClassLoader you can overwrite createCompilationUnit(CompilerConfiguration, CodeSource), the very first step. The CompilationUnit is really our compiler. It holds all the logic. So if you return a subclass of CompilationUnit here, you would return a custom compiler. But I don't want to go this far here.

I talked about hooks, but subclassing CompilationUnit would be a bit big possibly. I talked also about the phases a compiler goes through and that's where CompilationUnit offers hooks. The addPhaseOperation methods do allow you to give the compiler additional logic for a phase of your choice. In Groovy these phses are:

• initialization: open files setting up the environment and such
• parsing: use the grammar to produce tree of tokens repersenting the source
  
• conversion: make a real AST out of these token trees
• semantic analysis: check for all things the grammar can't check for, resolv classes and other things
• canonicalization: complete the AST
• instruction selection: choose isntruction set, for example java5 or pre java5
• class generation: create the binaries in memory
• output: write the binaries to the file system
• finalization: cleanup
  

not all these phases are really filled, the canonicalization step is currently empty, but that is a implementation detail. If we want to operate on the the AST then we need one, thus the conversion phase is the most early phase to do that. You have to decide if your operation is about changing a certain source file or a certain class. Each file may contain multiple classes, so this is not equal. For example if we want to add an method, then it makes not much sense to work on the source (SourceUnit) but on the ClassNodes instead.

In our example we would call the addPhaseOperation(PrimaryClassNodeOperation, int) method. You may wonder about the name primary class node operation. A class node represents a class, but in the compilation process there are two kinds of classes: precompiled classes and classes we want to compile. These primary class node are classes we want to compile.

If you know the AST then the next steps are quite easy. For example:

class MyOperation extends PrimaryClassNodeOperation {
        public void call(SourceUnit source, GeneratorContext context, ClassNode classNode) throws CompilationFailedException {
               classNode.addMethod("foo", OpCodes.PUBLIC, null, null, null,code);
        }
}

This would add a method named foo to every class we compile, foo takes no arguments, is public and has the return type Object. The contents of the method are stored in "code". I haven't specified that here to keep the article not totally complicated. Ok, again let us think back to the orignal task of adding a method if the class name fullfills a certain convention. For example if a class is XyzFoo, then add a method xyz, if the class is AbcFoo, then abc. Ok, that is a bit crazy, but well, just for the demonstration of the flexibility:

import org.codehaus.groovy.ast.ClassNode;
import org.codehaus.groovy.ast.stmt.BlockStatement;
import org.codehaus.groovy.classgen.GeneratorContext;
import org.codehaus.groovy.control.CompilationFailedException;
import org.codehaus.groovy.control.CompilationUnit;
import org.codehaus.groovy.control.CompilerConfiguration;
import org.codehaus.groovy.control.Phases;
import org.codehaus.groovy.control.SourceUnit;
import org.codehaus.groovy.control.CompilationUnit.PrimaryClassNodeOperation;
import org.objectweb.asm.Opcodes;

import groovy.lang.GroovyClassLoader;

class MyGroovy extends GroovyClassLoader {
    protected CompilationUnit createCompilationUnit(CompilerConfiguration config, CodeSource source) {
        CompilationUnit cu = super.createCompilationUnit(config, source);
        cu.addPhaseOperation(new PrimaryClassNodeOperation() {
            public void call(SourceUnit source, GeneratorContext context, ClassNode classNode) throws CompilationFailedException {
                String name = classNode.getName();
                if (name.endsWith("Foo") && name.length()>3) {
                    name = name.substring(0,name.length()-3);
                    BlockStatement code = new BlockStatement();
                    classNode.addMethod(name,Opcodes.ACC_PUBLIC,null,null,null,code);
                }
            }
        },Phases.CONVERSION);
        return cu;
    }
}

Grails does use this mecahnism to add the toString and hashcode methods as well as the id field in domain classes.

Another easy example is adding a import. But since we don't add an import to a specific class, but to all classes in a file, we are operation on the SourceUnit (representation of the source file) so we use a SourceUnitOperation.

        cu.addPhaseOperation(new SourceUnitOperation() {
            public void call(SourceUnit source) throws CompilationFailedException {
                source.getAST().addImport("User",ClassHelper.make("my.company.UserDAO"));
            }
        },Phases.CONVERSION);

here we add an import for UserDAO and make the class known as User inside the file we compile. That is comparable to "import my.company.UserDAO as User" in Groovy.

As you may see this is much more than aspect oriented programming could do. You can place arbitary logic inside the compiler and modify the ASTs. If you want to do more complex operations or more local ones, then you might have to really go into the AST. I suggest to use the LabelVerifier as entry point for this. A little debugging helps you to understand how the AST looks like and how the visting works. I plan to write a big documentation about these things, but this will be on the Groovy wiki then. The purpose of this article is just to give you a insight in the internals of the compilation and how it can be influenced from the outside. Without knowing the AST there is not really much you can do. It is not that the AST is very complicated or bad documented, but learning about over 60 classes isn't done in a short form. The biggest exmaple of AST traversion is AsmClassGenerator, the part in the compiler creating the binaries. But it is a big class, not too good to learn.

My plans for the future are to add an mechansim that allows to write some text that gets mixed into the classes the compiler produces. In the import example above you would maybe write something like

cu.mixinPerSource("import my.company.UserDAO as User")

. Or in the method example:

cu.mixinPerClass {"def ${it[0..-3]}(){}"}

. But it looks like this will ahve to wait for post 1.0

You could also use the AST to transform any DSL containing binary operations such as "&" and "<" into a builder or a method based version. You could transform it into a database request or whatever. There are many possible usages of this. And in fact something like that is already used in GroovySQL - but well hidden ;)

So might I ask again: Ever wished to be able to influence the compiling process in Java?

Concurrent Loop Excetion - Comparing Java and Groovy

Neal showed in his blog a new example of how closures in java could look like, in real world usage.

If I would program this in Groovy (which runs on the JVM btw), I would maybe write this:

def getAttendees() {
  return getResponses()
         .findAll{it.mayAttend()}.collectAsync(threadPool) {it.attendee}
}

Right, that's nearly a one-liner. Of course that code is not fully equal to Neal's program, because mayAttend() is executed synchronous, and Groovy doesn't have collectAsync. But we could write such a method and use it in a category. Maybe the method would look like:

def collectAsync(threadPool, Collection col, Closure action) {
  def result = []
  def tasks = col.collect {
    try{
      synchronized (result) {
        result << action.call(it)
      } catch (Throwable ex) {
        LOGGER.log(Level.SEVERE, "Uncaught Exception", ex);
      }
  }
  threadPool.invokeAll(tasks)
  if (!result) return null
  return result
}

For the non Groovy people some comments.. Groovy has an dynamic typing system, native literals for lists (see result), operator overloading (<<), closures and of course runs on the jvm with very good integration. Anyway, I am wrapping the original closure in another closure for the logging. This removes some code from the usage. If we where funny, we could use an additonal closure for the logging. "it" is a shortcut for the current value given to the closure. "it.attendee" is equal to "it.getAttendee()", Groovy does have logic for beans.

So how does it work? I use findAll to filter out all EventResponses that don't attend. findAll returns a new list containing only the important responses. collectAsync creates a list of closures, each closure representing a task, and gives that to my threadpool as suggested in the lower part of the article - of course it wasn't with closures there. Anyway, during execution each task calls the action closure and writes in "result". The nice thing is, that this function could be a library function used somewhere. There is no reference to something special besides the logger and threadPool. In fact the code is short only if the method is a library function. It is no use to tell people how well closures might look, if I compare library functions and their usage with raw code. The Groovy version is short because of the lack of typing we are using and much other factors. If I would have to add types then it would become much larger. For example even if there where a findAll method working on a collection and able to take such a closure, I still would have to define the list element type... possibly findAll<EventResponse>{it.mayAttend()} Then, "it" is not typed. Ok then it might look like: findAll{EventResponse it -> it.mayAttend()}. the Groovy syntax for defining parameters is a little different form the closure proposal, don't care to much about that. Next is collectAsync.. collectAsync<Principal>(threadPool){EventResponse it -> it.attendee} That makes:

getResponses().findAll{
  EventResponse it -> it.mayAttend()
}.collectAsync<Principal>(threadPool){
  EventResponse it -> it.attendee
}

Could still be seen as a one-liner, but it is a bit too big for my taste. Of course collectAsync would have to be changed too:

  Collection<R> collectAsync(threadPool, Collection<T> col,
                                Closure<T,T> action) {
  List<R> result = []
  List<closure<Void>> tasks = col.collect<> {
    try{
      synchronized (result) {
        result << action.call(it)
      }
    } catch (Throwable ex) {
     LOGGER.log(Level.SEVERE, "Uncaught Exception", ex)
    }
  }   
  threadPool.invokeAll(tasks)   
  if (!result) return null
  return result 
}

And this version is just a wish... I mean [] must be replaced by new ArrayList<R>(), !result with isEmpty(). Then we still are not in java, because we need to redefine our generic closure type to Closure1 for the closure taking one argument and Closure0 for closure taking no arguments. Next downside is that findAll and collectAsnyc might not be defined on Collection to keep compatibility (that is no problem in Groovy, as Groovy does have a special mechanism to handle this). So findAll might be a static import, and collectAsync is possibly defined in whatever threadPool is. Then the code may look like:

return threadPool.collectAsync<Principal>(
  findAll(getResponses()) {
    EventResponse it -> it.mayAttend()
  }), {
    EventResponse it -> it.attendee
  })

which is much worse to read due to the new ordering...

List<EventResponse> filtered = findAll(getResponses()) { 
  EventResponse it -> it.mayAttend()
}
return threadPool.collectAsync<Principal>(filtered){
  EventResponse it -> it.attendee
}

looks much better again.. Of course we are not using the syntax suggested in the closure proposal.. then it would look like:

List<EventResponse> filtered = findAll(getResponses(),
          EventResponse(EventResponse it) {it.mayAttend()});
return threadPool.collectAsync<Principal>(filtered, 
          Principal(EventResponse it) {it.getAttendee()})

And a version using inner classes would like:

List<EventResponse> filtered = findAll(getResponses(),
   new Closure1<EventResponse,EventResponse) {
     EventResponse call(Eventresponse it) {it.mayAttend()}
   });
return threadPool.collectAsync<Principal>(filtered, 
  new Closure1<Principal,EventResponse) {
    Principal call(EventResponse){it.getAttendee()}
  });

So we really gain something from the closure proposal - if compared to the current usage of inner classes, but then if I compare that to the very short Groovy version... Java could be shorter here if more type inference was used. It is not the shortness of the Groovy version I like so much, it is the clear code.

But another thing should be made clear with this: the programming style in a language like Groovy is different from the style normally used in Java. I mean compare my endresult with the version without closures at the end of the article. I think I would prefere the other version. Even if I "beautify" the code by using proper identation and such things, I am not gaining much. Strange, isn't it? Java prevents us from defining usefull library functions because of its syntax bloat. Even though there is neraly no logic left in the method itself and all is done by library functions, I have to write so much code for the little compiler... annoying would I say.

I really hope they decide to add more type inheritance. Of course I know that type inference is no easy task for the compiler builder, but it should be doable. Java has way to go here I guess.

Annotations in Groovy

Annotations are thought for providing MetaData.

Such MetaData can be used by the compiler or other processing tools by using the source, the bytecode or at runtime. Annotations are not thought to have an semantic effect.... I always have to tell me this... Somehow I really have a problem with that. Anyway, Groovy doesn't have them. And Groovy won't get Annoatations in 1.0. But as soon as 1.0 is out we want to add them. Tools working with annoatations will then most likely work for Groovy too.

The new JUnit is an example of why Groovy should have annotations. It is not sure yet what set of default annotations Groovy will support, but people will be able to write annotations in Groovy. How much of the enhancements though Groovy can be used in annotations is open, we need to look at what the bytecode allows and if we can go around it... and if we want to do that.

And there is Hibernate... avoiding to persist the metaClass field is sometimes very important. We had several problems with serialization of that property, because the MetaClass is not serializeable and so the default serialization might fail. Using bean serialization might make things even worse. And Hibernate...

@Entity
class Customer implements Serializable {

    @Transient
    MetaClass metaClass
    
    @Id
    Long id

    String firstName
    String lastName
    Date birthday

    @Transient
    Integer age

    @Embedded
    private Address homeAddress

    @OneToMany(cascade=CascadeType.ALL)
    @JoinColumn(name="CUSTOMER_ID")
    Set orders
}

I "stole" that example form the Hibernate page and added the transient property for metaClass. Of course you don't have to add the getter and setter methods, as they are already generated by Groovy. Just add buisness code and be lucky ;)

And of course there are more tools using annoatations and I am sure there will be much more of them in the future. I am very curious if other dynamic languages on the JVM will go that way to. But as most scripting languages on the jvm do not generate compatible class objects it might be out of scope for many of them.

Groovy on Speed - a "fast mode" for Groovy

fast mode? What's that?

The Problem:
Did you ever try to meassure the performance of groovy? It is a quite difficult task. Because of the high integration of Groovy and Java, you are contantly calling Java and Groovy. For example think of some time consuming operations in a Java library. The Groovy part would probably call the library and then return the result. The overhead through Groovy is minimal.

Calling a Groovy method doing some calculations on the other hand might result in bad performance. The solution is quite simple. Identify these parts and write them in Java, by delegating to a Java library or subclass the Groovy class in Java and overwrite the methods in Java.

Of course this is a possibility. But it means you have to use Java for these parts. It means you can't use Groovy syntax for the operations. That sure is no problem for people seeing Groovy as frontend... somewhere... occasionally used, but no essential part. It does indeed become a problem for groovy evangelists.

What exactly is the Problem?
What Goovy makes slower than Java is it's ability to add/intercept methods at runtime and it's slightly different type conversion model. A method invocation in Groovy consists usually of several normal method calls, where the arguments are stored in a array, the classes of the arguments must be retrieved, a key is generated out of them, a hashmap is used to lookup the method and if that fails, then we have to test the available methods for compatible methods, select one of the methods based on the runtime type, create a key for the hasmap and then in the end, do a reflection like call on the method.

Benchmarks usually use the same typs of arguments, so the hasmap kicks in and allows a "fast" method selection. Only counting the method calls, you get around 4 calls before you are able to look in the Hashmap. Then you create the key, do the lookup in the map... given all that I think it is quite a wonder Groovy has such a good performance!

How to make it even faster?
The way to make something faster is either to find an algorithm with a better runtime complexity, or to reduce the constants. If we would not have to create the Object[], we might be able to reduce the costs very much.. I made some meassurments and it seems that invoking a method taking 3 arguments is 3 times slower than a method taking no arguments. That's because we not only have to create the Object[] for the 3 arguments, but we also have three objects which types we need to get, the key becomes longer, and as the key is a String there is a coherence of the length of the string and the number of arguments.

Java does avoid that. Java does select the method at compile time, so there is no Object[], no type extraction and such. Of course the VM checks the types of the parameters, but in case of Groovy the VM does have to do that more than once. The end of the call in Groovy is the same as in Java, but there has much action taken place before. So if we were able to select a method at compile time in groovy, we could avoid that problem completly.

What does it mean?
It means Groovy would be as fast as Java! But that is no general solution. Dynamic typing doesn't allow you to choose the methods at runtime. And the MetaClass facilities won't help here too. so you are changing the semantics of a method call when chanign the typing from dynamic to static.

So what is the Fast Mode then?
The idea is not do dynamic typing every where. In fact there was already a proposal going into that direction. It was even in the codebase, but it was a global switch. I think that is the wrong solution. My solution is to let the programer decide where he wants to have that. For simplicity I will talk about an annotation here, but there was no decision on what to use. Anyway, I would use this annotation for a class, for a method, for a block of code or a single statement. Each so marked part would loose dynamic typing and letting the compiler use static typing to choose the method. This means that the MetaClass is ignored. It does not mean, that we loose any of the addons to the jdk we made. It will still be possible to use "each" or native syntax for BigDecimals, lists and maps. It doesn't mean that you get back checked exceptions or such. I am not sure yet if the compiler should forbid the mixed usage of static typed and dynamic typed arguments, or if it simply should fall back to the normal invocation mode.. Anyway, so instead of asking the MetaClass to tranform the type and make the call the compiler, we will add code to make the type transformation and the method call based on the static types. We loose multimethods then, and we loose the ability to catch unknown methods and properties! So we loose one of the most appealing feature of Groovy. And given the fact, that a builder depends on them we loose them then too. But as you can limit the usage of the fast mode very much it is no problem at all.

Of course this adds much to the complexity of the language.

What about ivokedynamic?
invokedynamic is the great hope of all dynamic languages on the JVM. If it helps us to not to create the Object[] and if it helps us to avoid to create the hasmap key and if it helps us to avoid the type tranformations, then it might be a much better solution than my fast mode. But even if it does all this, it won't be in Java6. It is Java7 at best. And it is not yet defined. The work for this is still in the early stages and while I hope the result will be very good, it is no solution for now.

The Goal
The idea of the "fast mode" is to provide a way to give the maximum performance at certain places. Then you can do calculations in Groovy. I am no fan of using one language for all uasages, but on the other hand I think that such a mode would be very good for Groovy. And it would tell all those benchmark writters out there not to meassure a language with calculations. That is a very stupid thing to do I would say.

The implementation is surely not done in a few minutes and this "fast mode" will not go into Groovy 1.0. But 2.0 or even 1.1 may have that.

Other ideas?
There were other ideas, yes. For exmaple not to simply make the call through the Metaclass, but to let the compiler guess a method and check with the MetaClass if the guess is right. Sadly that doesn't help, because the method selection has still to be done and is the cost intensive factor here. Writing a hotspot engine for Groovy might be a solution. Java is making HotSpot open, so it is theoretically an option. And possible a much better option than anything else. But there are many unclear details.... none has the knowlefge to write something like that, noone knows the API, how is the HotSpot engine added to the normal Java SDK?

We always tried to avoid the need of a special Groovy JVM, even it is jsut the JVM with additional bytecodes. if the HotSpot engine can't be added to any normal JDK then I don't see why this should be any different.

Reality!
I think the space for doing optimizations is big. For example in the "fast mode" we probably don't need to always create a closure. Instead we can inline the closure handling method and the closure itself. But I think only a concrete inplementation of all the possibiilites would be a fair way to tell if something is better. But even if the normal method invocation in Groovy gets a boost fomr a better HotSpot engine, or new bytecode, it only means that the "fast mode" is no longer needed. I would always recommend to use the normal Groovy method invocation and identify the cost intensive parts. Neither will the "fast mode" solve all problems nor will it free you from the usage of profilers.