Accessing information from the pure ParseTrees

[ghbakir]

Hi there.
I am evaluating JSQLParser for a project where I would like to have access to the actual parsetrees.

A simple test is to parse the two queries
String s_and = "SELECT director FROM movie WHERE YEAR = 1999 AND YEAR = 2000";
String s_or = "SELECT director FROM movie WHERE YEAR = 1999 OR YEAR = 2000";

and see the difference in the data structures.

I am dumping out ID + value if present as follows

  void MyDump(SimpleNode n, int level) {
    for ( int i = 0; i < level; ++i)
      System.out.print(' ');

    if (n.jjtGetValue() != null)
      System.out.println(n.getId() +  n.jjtGetValue().toString()); 
    else 
      System.out.println();

    for (int i = 0; i < n.jjtGetNumChildren(); ++i) {
      MyDump((SimpleNode)n.jjtGetChild(i), level + 1);
    }
  }

My problem:
Except for the first line there is no difference in the inspectable parse trees:

 6 SELECT director FROM movie WHERE YEAR = 1999 OR YEAR = 2000
   9 director
    19 
     4 director
   5 movie
   13 
    14 YEAR = 1999
     19 
      4 YEAR
     19 
    14 YEAR = 2000
     19 
      4 YEAR
     19 
(Side question: Are these IDs defined in CCJQLParserConstants?)

From the tree alone it is not possible to distinguish what kind of expression is performed.
(AND/OR)

Is it possible that this type of information is in the actual instances uses to construct the expression
(meaning there are some dynamic instance of expression) but not present in the traversable tree?

Ideally the data that one can traverse in the AST tree would differentiate between both cases.
Any suggestions how to figure out this info?

Thanks.

[manticore-projects]

Greetings.

1. the information about the Statements and Expressions is stored in the Statement Object, which the Parser or ParserUtil will return:

    Statement statement = CCJSqlParserUtil.parse(sql);
    if (statement instanceof Select) {
      Select stmt = (Select) statement;
      if (stmt.getSelectBody() instanceof PlainSelect) {
        PlainSelect ps = (PlainSelect) stmt.getSelectBody();
        OracleHint hint = ps.getOracleHint();
        assertNotNull(hint);
        assertEquals(hints[0], hint.getValue());
      } else
        if (stmt.getSelectBody() instanceof SetOperationList) {
          SetOperationList setop = (SetOperationList) stmt.getSelectBody();
          for (int i = 0; i < setop.getSelects().size(); i++) {
            PlainSelect pselect = (PlainSelect) setop.getSelects().get(i);
            OracleHint hint = pselect.getOracleHint();
            if (hints[i] == null) {
              Assert.assertNull(hint);
            } else {
              assertNotNull(hint);
              assertEquals(hints[i], hint.getValue());
            }
          }
        }
    }

2. Best way to traverse through the AST is the Visitor Pattern. Please refer to the Deparser Implementation which uses this pattern.

[ghbakir]

Thanks for the answer.
The Deparser is indeed very instructive.

[wumpz]

As @manticore-projects mentioned, the access to the parse tree is done using the visitor pattern. @ghbakir Your test uses the more basic AST tree of the parser. Maybe there is a bit of a confusion (https://stackoverflow.com/questions/5026517/whats-the-difference-between-parse-trees-and-abstract-syntax-trees-asts).

However, since the nodes returned for this AST are not complete due to performance issues you see no difference in your output. For instance, the AndExpression has no representing node in the returned AST tree, while it has a representation in the parse tree as an AndExpression object.

[ghbakir]

Thanks! Indeed I was not aware of the actual difference between the AST tree of the parser and the parse tree as represented by the class hierarchy. I did expect it to be a lossless mapping of each. The AST tree was very convenient to traverse while the parse tree nodes need to have a type per node to know what kind of fields to access.

Thanks for edification.