Abstract Syntax Tree (AST)

April 07, 2015 in Java Articles

Written by Ken Kousen


Look, I get it. Learning new things is hard, and it's scary. I know. I teach technical training classes for a living, so I see it all the time. I also am a writer, a speaker, and a software developer, which means I have to learn new things all the time, and it's hard.

But seriously, if you already know Java (and I'm talking the standard stuff, not that newfangled lambda weirdness which we'll save for other blog posts), you can add Groovy with minimal effort and enormous gain. You don't have to dig into every little facet of the language. Just a few additions to your coding tool kit can make a huge difference in your job.

I've been recommending Groovy for Java deveopers for years?—?even writing a popular and well-reviewed book on it (Making Java Groovy, available here at Manning, or at here at Amazon or wherever better development books are sold or wherever I can convince them to sell it), and it seems I'm beginning to develop a bit of an attitude. I get that. I'll try to keep it in check, at least for the rest of this article.

Earlier articles I've written here have tried to demonstrate how adding Groovy can help you. Heck, I spent an entire blog post just on POGOs, if you can believe it. And you're still not convinced? Seriously? What does it take to get through to you?

wow-that-escalated-quickly

In this article, I'm going to show a great feature from Groovy known as Abstract Syntax Tree (AST) transformations. Hopefully they will both dazzle and amaze you, and maybe even convince you to give Groovy a try.

Basic POGO annotations

AST transformations are triggered by an annotation and an implementation class. The annotation tells Groovy that during the compilation process, after it has built an abstract syntax tree from the source code, it should then execute the implementation, which generates new code in a controlled way.

Here's a trivial example. Below is a simple Plain Old Groovy Object (POGO):

class Person {
  String first
  String last
  final LocalDate dateOfBirth
}

This is Groovy, so:

  1. The attributes first, last, and dateOfBirth are all private
  2. The class Person is public
  3. Groovy autogenerates getter and setter methods getFirst, setFirst, getLast, and setLast
  4. Because dateOfBirth is marked final, Groovy only generates a getter method for it, getDateOfBirth

Not bad for five lines of code. But it gets better. If I add the annotation @ToString to it, as in:

  import groovy.transform.*
  import java.time.*
  
  @ToString
  class Person {
    String first
    String last
    final LocalDate dateOfBirth
  }

Now the class has a generated toString method (i.e., an override of public String toString() from Object) that returns the fully-qualified class name followed by the values of the attributes in order, top down:

  Person p = new Person(first: 'Steve', last: 'Heckler',
    dateOfBirth: LocalDate.of(1976, Month.FEBRUARY, 14))
  assert p.toString() == 'Person(Steve, Heckler, 1976-02-14)'

Btw, here I'm using Java 8's LocalDate class from the new java.time package, because why not?

If the compiler can generate a toString method, what about overrides for equals and hashCode?

Funny you should ask:

  import groovy.transform.*
  import java.time.*
  
  @ToString @EqualsAndHashCode
  class Person {
    String first
    String last
    final LocalDate dateOfBirth
  }

  Person p1 = new Person(first: 'Steve', last: 'Heckler',
    dateOfBirth: LocalDate.of(1976, Month.FEBRUARY, 14))
  Person p2 = new Person(first: 'Steve', last: 'Heckler',
    dateOfBirth: LocalDate.of(1976, Month.FEBRUARY, 14))
  
  assert p1 == p2
  assert p1.hashCode() == p2.hashCode()

With Java, we use the equals method rather than the == operator, but in Groovy all operators actually invoke methods. The == operator here calls the generated equals method, so we're good. The @EqualsAndHashCode AST transformation generates both equals and hashCode methods according to the prescription laid down in Josh Bloch's Effective Java book long ago.

The philosophy is, hey, if an IDE can generate that, why can't the compiler? Here it can, and does.

I can do more, though. How about generating a tuple constructor as well? “What's a tuple constructor,” you ask? It's a constructor that takes the attributes as arguments, top down. To wit:

  import groovy.transform.*
  import java.time.*
  
  @ToString @EqualsAndHashCode>
  @TupleConstructor
  class Person {
    String first>
    String last
    final LocalDate dateOfBirth
  }

  Person p1 = new Person(first: 'Steve', last: 'Heckler',
    dateOfBirth: LocalDate.of(1976, Month.FEBRUARY, 14))
  Person p2 = new Person('Steve', 'Heckler', LocalDate.of(1976, Month.FEBRUARY, 14))

  assert p1 == p2
  assert p1.hashCode() == p2.hashCode()

That works, too. Sweet. The combination of @ToString, @EqualsAndHashCode, and @TupleConstructor is so popular that there's a short cut for it. It's called @Canonical, and does the same thing.

  import groovy.transform.*
  import java.time.*
  
  @Canonical
  class Person {
    String first
    String last
    final LocalDate dateOfBirth
  }

  Person p1 = new Person(first: 'Steve', last: 'Heckler',
    dateOfBirth: LocalDate.of(1976, Month.FEBRUARY, 14))
  Person p2 = new Person('Steve', 'Heckler', LocalDate.of(1976, Month.FEBRUARY, 14))

  assert p1 == p2
  assert p1.hashCode() == p2.hashCode()

The result is that our little POGO here now has, in additon to the private attributes and public getters and setters listed earlier:

  • A map-based constructor so I can set all the properties (including the one marked final)
  • A tuple constructor
  • An equals method that checks the values of the attributes
  • A hashCode method that generates an int based on those attributes
  • A toString method that returns the fully-qualified class name and the values of the attributes in order

That's all in six lines of code (plus the import statement).

are-you

@Delegate

This is one of my favorites. Say you have a composition relationship, where a whole is made up of various parts. Then you would like the methods in the contained objects exposed through the container.

Here's an example. A SmartPhone is composed of a Phone, a Camera, and more. Here's a start:

  class Phone {
    String dial(String num) { "Dialing $num..." }
  }
  
  class Camera {>
    String takePicture() { "Taking picture..." }
  }
  
  class SmartPhone {
    Phone phone = new Phone()
    Camera camera = new Camera()
  }

So far, so good. The Phone class has a dial method that returns a String, and the Camera class has a takePicture method that does the same thing. Both are included in the SmartPhone class.

I would like to be able to invoke dial and takePicture on the SmartPhone directly, and have it call the same methods on the internal objects and return the results. I could write all those delegation methods, or use the @Delegate AST transformation.

  import groovy.transform.*
  
  class SmartPhone {
    @Delegate Phone phone = new Phone()
    @Delegate Camera camera = new Camera()
  }

  SmartPhone sp = new SmartPhone()
  assert sp.dial("Accelebrate's number") == "Dialing Accelebrate's number..."
  assert sp.takePicture() == 'Taking picture...'

Now all public methods is both Phone and Camera are exposed through the SmartPhone class. You don't have to write any of the relevant methods?—?the compiler generates them for you.

I know what you're thinking[1]. You're wondering what would happen if both Phone and Camera shared a method in common. What would happen to SmartPhone then?

Here's a variation of the last example:

  class Phone {
    String manufacturer
    String dial(String num) { "Dialing $num..." }
  }
  
  class Camera {
    String manufacturer
    String takePicture() { "Taking picture..." }
  }
  
  class SmartPhone {
    @Delegate Phone phone = new Phone(manufacturer: 'Samsung')
    @Delegate Camera camera = new Camera(manufacturer: 'Nikon')
  }
  
  SmartPhone sp = new SmartPhone()
  assert sp.manufacturer == ???

Note again that the manufacturer property provides a getManufacturer method, which is what I'm calling on the SmartPhone.

There's an answer, and then there's an explanation, which shows how to handle this situation in reality.

The answer is: Phone wins, because it's first in the class (read top down).

assert sp.manufacturer == 'Samsung'

Why does it work that way? Because the @Delegate annotation works by adding “missing” methods. When the compiler starts processing the SmartPhone class and sees the annotation on Phone, the getManufacturer method does not yet exist on SmartPhone, so the compiler adds it. By the time it hits the Camera class, the method isn't missing anymore.

That's the key to the right way to handle this problem. If you don't want to rely on the default, or you have an alternative implementation in mind, just add it. Groovy won't replace something that's already there.

  class SmartPhone {
    @Delegate Phone phone = new Phone(manufacturer: 'Samsung')
    @Delegate Camera camera = new Camera(manufacturer: 'Nikon')

    String getManufacturer() {
      "Phone: ${phone.manufacturer}, Camera: ${camera.manufacturer}"
    }
  }
  
  SmartPhone sp = new SmartPhone()
  assert sp.manufacturer == 'Phone: Samsung, Camera: Nikon'

How cool is that?

@Immutable

All the cool[2] kids these days are learning about concurrency, with all those cores and clouds and whatnot. Concurrency is really hard if you have, as the saying goes, “shared mutable state”.

shared-mutable-state

If your objects are immutable, that problem goes away. Some languages, like Clojure, make everything immutable. Some, like Scala, have both mutable and immutable objects. Some, like Java, make immutability very hard to achieve. To make a class produce only immutable objects, you have to:

  • Remove all the setter methods
  • Make all the attributes final, with getter methods as needed
  • Wrap any collections and maps with their unmodifiable counterparts
  • Make the class final (amazing how many people forget that)
  • Provide a constructor for supplying the attributes
  • Defensively copy any mutable components

Even that's not enough. You could try to do all that, or you can use the @Immutable annotation from Groovy, which does all of that for you.

  import groovy.transform.*
  @Immutable
  class Contract {
    String company
    String workerBee
    BigDecimal amount
    Date from, to
  }

  Date start = new Date()
  Date end = start + 7

  Contract c = new Contract(company: 'Accelebrate',
    workerBee: 'Me', amount: 500000,
    from: start, to: end)

This Contract class has a default and tuple constructor, along with the normal map-based constructor. Dates and strings are defensively copied. The attributes are marked final, and any attempt to change them results in a ReadOnlyPropertyException. There are no getter methods.

// Steve: Whoa! That amount was a typo. I'll just fix it...
c.setAmount(5000) // throws MissingMethodException, making me happy
// Me: Nope! I get ALL the cash! It's freakin' IMMUTABLE, sucka!

// Me: Um, but there's no way I'll be done in a week...
c.to = now + 50 // throws ReadOnlyPropertyException, making Steve happy
// Me: Aw, nutbunnies.

All currency denominations are in quatloos[3]. Good luck cashing that.

The class also provides a toString, an equals, and a hashCode method. Again, all of that from about half a dozen lines of code.

Would you rather write the corresponding class in Java, which would take well over 50 lines? I think not.

@Sortable

The @Sortable AST transformation adds code to make the class implement Comparable based on its attributes. Here is a class to represent golfers:

  import groovy.transform.*
  @Sortable
  class Golfer {
    String first
    String last
    int score

    String toString() { "$score: $last, $first" }
  }

Without the annotation, you'd have to write a sorting algorithm yourself. Wtih the annotation, the golfers are sorted by first name, then equal first names are sorted by last name, and equal first and last names are sorted by score.

  def golfers = [
    new Golfer(score: 68, last: 'Nicklaus', first: 'Jack'),
    new Golfer(score: 70, last: 'Woods', first: 'Tiger'),
    new Golfer(score: 70, last: 'Watson', first: 'Tom'),
    new Golfer(score: 68, last: 'Webb', first: 'Ty'),
    new Golfer(score: 70, last: 'Watson', first: 'Bubba')]

  golfers.sort().each { println it }

which outputs

  70: Watson, Bubba
  68: Nicklaus, Jack
  70: Woods, Tiger
  70: Watson, Tom
  68: Webb, Ty

Of course, that's not how you sort golfers. Fortunately, the annotation takes an includes argument which can list the proper ordering:

  import groovy.transform.*
  @Sortable(includes = ['score', 'last', 'first'])
  class Golfer {
    String first
    String last
    int score

    String toString() { "$score: $last, $first" }
  }

Now the sorted list is:

  68: Nicklaus, Jack
  68: Webb, Ty
  70: Watson, Bubba
  70: Watson, Tom
  70: Woods, Tiger

Much better. Note how Webb comes before both Watsons, because of his score, and Bubba comes before Tom because of his first name.

Fans of CaddyShack[4] will recall however, that Ty Webb didn't sort golfers that way.

compare_by_height

Judge Smails
   Ty, what did you shoot today?

Ty Webb
   Oh, Judge, I don't keep score.

Judge Smails
   Then how do you measure yourself with other golfers?

Ty Webb
   By height.

With that in mind, here's the new class:

  import groovy.transform.*
  @Sortable(includes = ['height', 'score', 'last', 'first'])
  class Golfer {
    String first
    String last
    int score
    int height

    void setHeight(int height) { this.height = -height }
    int getHeight() { height.abs() }

    String toString() { "$score: $last, $first (${height.abs()})" }
  }
  
  def golfers = [
    new Golfer(height: 70, score: 68, last: 'Nicklaus', first: 'Jack'),
    new Golfer(height: 73, score: 70, last: 'Woods', first: 'Tiger'),
    new Golfer(height: 69, score: 70, last: 'Watson', first: 'Tom'),
    new Golfer(height: 76, score: 68, last: 'Webb', first: 'Ty'),
    new Golfer(height: 75, score: 70, last: 'Watson', first: 'Bubba')]

  golfers.sort().each { println it }

The new result is now:

  68: Webb, Ty (76)
  70: Watson, Bubba (75)
  70: Woods, Tiger (73)
  68: Nicklaus, Jack (70)
  70: Watson, Tom (69)

This does include a bit of a kludge. The sort is in ascending order, so to make it go the other way I stored the negative of the height. The client doesn't have to know that, though, because I took the absolute value when returning it, or when printing it in the leaderboard.

Finally, did you notice how I used a value.abs() method on the property rather than the Java Math.abs(value)? That's one of the cool things about the Groovy JDK. It often takes static methods from one class and makes them easier to use instance methods of another. The Groovy JDK is full of great stuff, but that will be the subject of a future blog post.

Conclusions

Groovy provides Abstract Syntax Tree transformations, which modify the code during the compilation process, known as compile-time metaprogramming. Writing them is a bit of tedious process, but using them is trivially easy. In this post I reviewed @ToString, @EqualsAndHashCode, @TupleConstructor, @Canonical, @Delegate, @Immutable, and @Sortable.

The Groovy library includes several more, like @Singleton, @Lazy, @TypeChecked, and even @CompileStatic.

AST transforms save you so much work, it's worth asking why you aren't already using them. So, why not? Don't you want to go home at night? Wouldn't you like to avoid working on the weekends?

It's time to go for it. Add Groovy to your Java projects today.

kid

 

  1. And you should be ashamed of yourself. Just kidding. I don't really know what you're thinking, though I have my suspicions. More to the point, I'm often very grateful that nobody really knows what I'm thinking.
  2. Like how I just connected this section to the previous one by repeating the word “cool”? That's the sort of thing you learn to do if you want to be a semi-professional writer like me.
  3. Monetary unit used by the Providers to bet on drill thrall competitions in the Star Trek Original Series episode The Gamesters of Triskelion, but if you're the sort of person to read this article you probably already knew that.
  4. And if you're not a fan of CaddyShack, I honestly don't know what to do with you.

Respectfully submitted by Ken Kousen <[email protected]>, who teaches this stuff.,


Written by Ken Kousen

Ken Kousen

Ken Kousen is an independent consultant and trainer specializing in Spring, Hibernate, Groovy, and Grails. He is a regular speaker at conferences and the Author of "Modern Java Recipes" (O'Reilly Media), "Gradle Recipes for Android" (O'Reilly Media), and "Making Java Groovy" (Manning).
  


Learn faster

Our live, instructor-led lectures are far more effective than pre-recorded classes

Satisfaction guarantee

If your team is not 100% satisfied with your training, we do what's necessary to make it right

Learn online from anywhere

Whether you are at home or in the office, we make learning interactive and engaging

Multiple Payment Options

We accept check, ACH/EFT, major credit cards, and most purchase orders



Recent Training Locations

Alabama

Birmingham

Huntsville

Montgomery

Alaska

Anchorage

Arizona

Phoenix

Tucson

Arkansas

Fayetteville

Little Rock

California

Los Angeles

Oakland

Orange County

Sacramento

San Diego

San Francisco

San Jose

Colorado

Boulder

Colorado Springs

Denver

Connecticut

Hartford

DC

Washington

Florida

Fort Lauderdale

Jacksonville

Miami

Orlando

Tampa

Georgia

Atlanta

Augusta

Savannah

Hawaii

Honolulu

Idaho

Boise

Illinois

Chicago

Indiana

Indianapolis

Iowa

Cedar Rapids

Des Moines

Kansas

Wichita

Kentucky

Lexington

Louisville

Louisiana

New Orleans

Maine

Portland

Maryland

Annapolis

Baltimore

Frederick

Hagerstown

Massachusetts

Boston

Cambridge

Springfield

Michigan

Ann Arbor

Detroit

Grand Rapids

Minnesota

Minneapolis

Saint Paul

Mississippi

Jackson

Missouri

Kansas City

St. Louis

Nebraska

Lincoln

Omaha

Nevada

Las Vegas

Reno

New Jersey

Princeton

New Mexico

Albuquerque

New York

Albany

Buffalo

New York City

White Plains

North Carolina

Charlotte

Durham

Raleigh

Ohio

Akron

Canton

Cincinnati

Cleveland

Columbus

Dayton

Oklahoma

Oklahoma City

Tulsa

Oregon

Portland

Pennsylvania

Philadelphia

Pittsburgh

Rhode Island

Providence

South Carolina

Charleston

Columbia

Greenville

Tennessee

Knoxville

Memphis

Nashville

Texas

Austin

Dallas

El Paso

Houston

San Antonio

Utah

Salt Lake City

Virginia

Alexandria

Arlington

Norfolk

Richmond

Washington

Seattle

Tacoma

West Virginia

Charleston

Wisconsin

Madison

Milwaukee

Alberta

Calgary

Edmonton

British Columbia

Vancouver

Manitoba

Winnipeg

Nova Scotia

Halifax

Ontario

Ottawa

Toronto

Quebec

Montreal

Puerto Rico

San Juan