Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Playing with the JVM Reciever #10

Open
mattys101 opened this issue Oct 2, 2017 · 28 comments
Open

Playing with the JVM Reciever #10

mattys101 opened this issue Oct 2, 2017 · 28 comments

Comments

@mattys101
Copy link
Contributor

I've been playing with the concept of the JVM receiver a little
JVMReceiver-Example.zip

You might not want to look at the code files, they are a bit hacky and woefully incomplete, but the smalltalk file might give you an idea of what I am thinking.

| tempVar |
tempVar := 3.
JVM invoke: 'println' on: 'java.lang.System.out' with: 'TestingJVM'.
JVM invoke: 'print' on: 'java.lang.System.out' with: 1.
JVM invoke: 'println' on: 'java.lang.System.out'.
JVM invoke: 'println' on: 'java.lang.System.out' with: 2 ofType: 'int'.
JVM invoke: 'println' on: 'java.lang.System.out' with: tempVar ofType: 'int'.

The core of it is to the JVM pseudo variable a suite of 'methods' like

invoke:on:
invoke:on:with:
invoke:on:with:ofType:

that can be used in different situations and have varying degrees of inference depending on whether the values given are literals or variables. That sort of thing.

Basically, I think if we can give the JVM reciever a nicer interface and some smarts, we can make it a little more natural to mix adaptor code with Smalltalk code. For example, we could perform calculations, call other methods, etc. using standard Smalltalk and use the result in the code output by the JVM receiver. I think your original one was much more limited, unless I am mistaken, and basically required you to write methods as if you were writing the byte code directly?

The other thing about giving it some smarts is that you can potentially detect some issues at (smalltalk) compile time, not when the code is executed, which I think was a bit of a problem before?

What do you think?

Matt
@jamesladd
Copy link
Owner

Hi Matt,

Thank you for taking the time to look at this. Really appreciated it.
I like where you are going, and would like to give you some background that I think may help
with your design / implementation.

The special reserved word JVM was there to allow JVM bytecode to be emitted into the bytecode stream at compile time. So while it would look like a Runtime construct it would actually be a compilation time action.

The messages that JVM can respond to were specifically targeted as being close to the JVM bytecode instruction. There are two reasons for this. 1. This allows the author to output any bytecode that the JVM can understand which is important because you can do more in JVM bytecode than the Java language allows, and 2. There is a tool called AsmWeb that will take a java class and dump the bytecode in a format that AsmWeb can understand when using its bytecode manipulation library. An cut down version of this dump of the Stic main class follows:

    cw.visit(52, ACC_PUBLIC + ACC_SUPER, "st/redline/Stic", null, "java/lang/Object", null);
    cw.visitSource("Stic.java", null);
    {
        fv = cw.visitField(ACC_PRIVATE + ACC_FINAL, "args", "[Ljava/lang/String;", null, null);
        fv.visitEnd();
    }
    {
        mv = cw.visitMethod(ACC_PUBLIC + ACC_STATIC, "main", "([Ljava/lang/String;)V", null, new String[]{"java/lang/Exception"});
        mv.visitCode();
        Label l0 = new Label();
        mv.visitLabel(l0);
        mv.visitLineNumber(14, l0);
        mv.visitTypeInsn(NEW, "st/redline/Stic");
        mv.visitInsn(DUP);
        mv.visitVarInsn(ALOAD, 0);
        mv.visitMethodInsn(INVOKESPECIAL, "st/redline/Stic", "<init>", "([Ljava/lang/String;)V", false);
        mv.visitMethodInsn(INVOKESPECIAL, "st/redline/Stic", "run", "()V", false);
        Label l1 = new Label();
        mv.visitLabel(l1);
        mv.visitLineNumber(15, l1);
        mv.visitInsn(RETURN);
        Label l2 = new Label();
        mv.visitLabel(l2);
        mv.visitLocalVariable("args", "[Ljava/lang/String;", null, l0, l2, 0);
        mv.visitMaxs(3, 1);
        mv.visitEnd();
    }

The idea here is that an Author could write Java, dump the bytecode and then implement the equivalent with messages to JVM. So keeping the set of message JVM responds to close to this is a key.

What I used to have in an early version of Redline Smalltalk written by Chad Nash was an adaptor that would automatically wrap a Java class and enable it to be treated as a Smalltalk class. For example you could wrap System and then say:

System out println: 'hello world'.

What I think Redline needs is both a JVM bytecode emitter as described above and a Java class in Smalltalk that uses the JVM emitter to wrap a Java class so it can be treated like a Smalltalk class.
So not a full blown automatic Adaptor but a manual one, where an Author can say:

| j |
j := Java new: 'MyJavaClass'.
j call: 'someMethod'.
j call: 'someMethodWithArg' with: arg1 and: arg2.

The implementation of the Java class would in turn use the JVM class to emit the bytecode needed to function.

So there would be a JVM special class to emit any bytecode and have methods similar to the Asm dump to make it super easy for an Author to covert the Asm output to JVM calls.

And a class in Smalltalk called Java that would provide a higher level abstraction.

This means you approach is worth doing but would be adjusted slightly to be the Smalltalk Java class but we would still need a JVM class.

Does this make sense? Happy to discuss further.

@jamesladd
Copy link
Owner

jamesladd commented Oct 2, 2017 •
edited
Loading

One challenge I'm trying to save you from is doing a lot of work and then finding you can't handle a situation where a Java class throws a specific exception and you need to catch it.

BTW - There is (or will be) a special field in every ProtoObject called javaValue which is there to hold an instance of a Java class. This is to support wrapping in a easy way.
ProtoObject javaValue. <- Answer contained Java object.
ProtoObject javaValue: object. <- Hold contained Java object and return receiver.

ProtoObject will be the superclass of all classes. ie: Object is a subclass of ProtoObject.

@mattys101
Copy link
Contributor Author

Good points and thanks for the background. I forgot that there were things the JVM could do that Java doesn't allow. My approach would have blocked that out. I also didn't even consider exception handling---who does that ;-)
Although, would it be possible to make the Smalltalk #on:do: exception handling work with the standard Java exceptions?
I've only been looking at this stuff casually, so I'm not 100% sure what's possible and what's not.

I like the idea of separating out the basic, basic JVM bytecode emitter and a Java wrapper. I may have been trying to do too much with the JVM special class. My aim was to make using it a little more natural to Smalltalk, while outputting somewhat checked bytecode at compile time. One issue I see with the low level interface is there is no real way to check that you are generating correct bytecode until it runs---if you rely on the AsmWeb tool then you are potentially restricting yourself to the JVM subset supported by Java. But if you have another class that provides the higher level of abstraction then it's all good.

How much of the old JavaAdapter are you planning to resurrect? I had a look at it once before with the intention to port it from one of your earlier versions into one of the earlier rewrites. I discovered it was pretty incomplete and hacked up---not judging :-P ---for example, I discovered code in there specific to getting the ArrayList example to work. And that was basically the only example I could get to work.

For the Java wrapper class, would it be useful to look at what JNIPort does to generate its wrapper classes or is it too different to be worthwhile?
Out of curiosity, have you looked at how JRuby does its wrapping of Java objects? If so, what do you like/not like about it? (It's been something I've been meaning to look at, but never seem to be able to find the time)

@jamesladd
Copy link
Owner

Thank you for your time looking into this.

In brief I think we need both the JVM and Java (Wrapper) classes.
I take your point on the JVM bytecode not necessarily being correct until you run the code but with proper testing it would all come out in the wash. I'm keen not to have the compiler double check the code so as to make the compiler lite and fast. The code will be tested anyway.

With AsmWeb it will dump the bytecode generated by the JDK and in some places this actually shows some good techniques, for example how method handles are used to cache lambdas - something I will be using for Smalltalk blocks.

I'm not planning to resurrect the old Java Adaptor as it was not robust. I'm hoping your Java Wrapper will be much better. The Java Adaptor was inspired by the JRuby Java Adaptor and while it would be convenient to automagically wrap Java I think it would be better in the first instance to use a wrapper.
Less of the stop the world while I generate a hundred classes to expose the Java when I only wanted to call System.out.println.

Please consider doing the Java wrapper.

BTW - The approach I am thinking of using for the SmalltalkGeneratingVisitor and Emitter is to build a Message object (I'll do this soon) which is passed to the emitter. ie: emit(message).
As almost 99% of the work is generating a message send and putting the parts of that (receiver, selecter, args) into an object and passing it to the Emitter makes sense to me and from experience will tend to be cleaner - especially if the Visitor can push a message onto a stack and then visit children that will pop it and add elements to it.

  • J

@jamesladd
Copy link
Owner

If you look at the branch jcl/introduce-message-emitter you can see where I am going with the Emitter and building up objects to emit.

@mattys101
Copy link
Contributor Author

The emitter stuff looks much neater than your previous version.

Thinking about the Java wrapper class. I may not have fully understood what you were thinking and am struggling to determine where the wrapper class and the old Adapter depart in their approach.

So with what you sketched above:

| j |
j := Java new: 'MyJavaClass'.
j call: 'someMethod'.
j call: 'someMethodWithArg' with: arg1 and: arg2.

my thinking is that you would get an instance of the 'Java' class that wraps an object of the specified class. The Java class itself does not necessarily have many methods other than say call:, call:with, call:withArgs: and similar, maybe some to retrieve (non-private) fields. These methods would be dynamic and search for the appropriate methods of the underlying java object and call them, most likely with caching so it doesn't have to search every time. But it wouldn't generate an in memory class like the old Java Adapter did. Is that right?

A possible issue might be dealing with static methods since something like:

Java new: 'java.lang.System'.

would create an instance of 'java.lang.System' wrapped by an instance of Java. One possibility is to have another wrapper class JavaStatic that gives access to the static members of the class. JNIPort does something similar to this IIRC. Then you could do something like:

| j |
j := JavaStatic new: 'java.lang.System'.
j call: 'exit' with: 1.

But accessing fields might be problematic/messy, for example:

out := j field: 'out'. "if we extend the metaphor used for the call: method"
out call: 'println' with: 'this isn''t too bad, except you have to store it in a variable'.
(j field: 'out') call: 'println' with: 'this is starting to look messy though'.
j out call: 'println' with: 'this is a little better'.
j out println: 'this is the nicest way though'.

The problem is I think the last two bring us back into Adapter territory, unless we did it all dynamically based on an implementation of #messageNotUnderstood. If that were the case, we could allow adapter classes to be created manually (or semi-automatically) that subclass the classes Java or JavaStatic as appropriate, where methods can be implemented if desired. This way the dynamic behaviour can be inherited if you are lazy ;-) or you could implement the adapter methods yourself if you wanted. You could even ship some pre-made adapters for commonly used classes if you wanted. (Maybe some of the core classes String, List, Set, etc. may even be (special) adapter classes? Or maybe I am just over thinking things now)

Is this the type of thing you were thinking? If I can get on the same page (or at least pick up the same book) I might be able to start working on the Java and JavaStatic classes.

BTW: I think such an approach would be very JNIPort-ish, which is not necessarily a bad thing in that anyone working with JNIPort in their current Smalltalk environment wouldn't really need to change the way they interact with Java. Not sure if that is your intention though or if it is the best approach.

What do you reckon?

@jamesladd
Copy link
Owner

For me the goal is a quick win as getting Redline out is key and we can always improve later.
With that in mind I would suggest a Java class like you have suggested is a good start.
Ie: Java new: 'java.lang.String' etc
Where the Java Smalltalk class simply wraps the Java Java class and allows people to make calls to it.
ie: call: 'reverse' or call: 'method' with: arg1 and: arg2.
And the Java Smalltalk class can use reflection in the first instance to on each call to make things happen.
This get the ability done and into Redline.
Later you might add caching of the reflected methods and even later still it might do a full reflection on the Java Java class when a new instance is created.

In regards to fields, I would suggest we don't bother. Why access a field rather than a method?
I know there may be one or two Java classes that don't have methods to act on the values in fields but these are not very OO and probably should be avoided or wrapped in a better Java class to begin with ;)

What do you think of this simple approach to begin with?

@mattys101
Copy link
Contributor Author

Sounds like a plan. I was thinking too much about how it might be in the end, not what can be done short term to get something going. Would probably do the pair of classes, though, i.e., the Java and JavaStatic classes.

I think we need to work out a better convention to talk about the classes rather than the Java Smalltalk class and the Java Java class though ;-)

WRT fields, I don't see why we couldn't include them though. I think they are almost a must for statics, in which case we'd pretty much have the code anyway. I might have to think about it a little more though.

@jamesladd
Copy link
Owner

jamesladd commented Oct 6, 2017 via email

@mattys101
Copy link
Contributor Author

That way you do not have to have an instance of the Java class (as opposed to the Java class, which is the one we will implement) just to call static methods, which might be important for some classes with private/protected constructors and factory methods. It would also allow a parallel situation with Smalltalk objects and their classes, e.g.

| javaString aString |
javaString := JavaStatic new: 'java.lang.String'.
javaString call: 'valueOf' with: 1. "static method call"
aString := javaString new. "equivalent to Java new: 'java.lang.String'"
aString call: 'reverse'. "instance method"
aString javaClass call: 'valueOf' with: true. "static methods call from the instance"

Or am I over thinking things again?

The other nice thing about it is that it is similar to JNIPort, so that if you currently use JNIPort (like I do) porting the code into Redline should be relatively straightforward ;-)

Considering a lot of code would be shared between the Java and JavaStatic classes (I think), I don't think it would make it considerably more difficult or time consuming to have them both rather than 1 that does it all.

What do you think?

@jamesladd
Copy link
Owner

jamesladd commented Oct 8, 2017 via email

@mattys101
Copy link
Contributor Author

Depends on what you start off with. If you already have an instance of Thingo, it could just be:

aThingo javaClass call: 'myStaticMethod' with: 'someValue'

If you don't have either yet you could do:

jThingo := JavaStatic new: 'Thingo'.
jThingo call: 'myStaticMethod' with: 'someValue'.
aThingo := jThingo new.
aThingo2 := Java new: 'Thingo'. "This is equivalent to the line above"
aThingo call: 'nonStaticMethod' with: 'someValue'.

We could add a callStatic: method if we wanted, but it would just be equivalent to:

- callStatic: aString with: anObject
       ^self javaClass call: aString with: anObject

Maybe it is the wrong approach, but I like it due to the parallels to standard Smalltalk, i.e.:

Thingo myStaticMethod: 'someValue'.
aThingo := Thingo new.
aThingo class myStaticMethod: 'someValue'.
aThingo nonStaticMethod: 'someValue'.

and of course:

"Given the instance method wrapping the class method"
- myStaticMethod: aValue
       ^self class myStaticMethod: aValue
"We can call"
aThingo myStaticMethod: aValue

Does that seem like a reasonable pattern of usage to you?

The main problem I see with having only a single Java class is if you only call a static method, e.g.:

(Java new: 'Thingo') callStatic: 'myStaticMethod' with: 'someValue'. "This would create an instance of Thingo"
"Alternatively... a method that does not instantiate the class"
(Java newStatic: 'Thingo') callStatic: 'myStaticMethod' with: 'someValue'.
"Or should it just be?"
(Java newStatic: 'Thingo') call: 'myStaticMethod' with: 'someValue'.

So you end up with both sets of methods anyway, with the possible confusion of it all being on the same class.

If you prefer I can implement it as a single Java class (it is your baby after all), but I think the two classes would be cleaner.

@jamesladd
Copy link
Owner

jamesladd commented Oct 10, 2017 via email

@jamesladd
Copy link
Owner

Mind you, when you say:

aThingo javaClass call: 'myStaticMethod' with: 'someValue'

This is nice as it is more Smalltalk. ;)

@mattys101
Copy link
Contributor Author

The idea is that new: would actually create an instance of the underlying Java class while newStatic: would not. At the risk of mixing Smalltalk and Java code, they could be defined like so:

Java (class)>>new: aClassName
    aClass := self findClass: aClassName. "retrieves the java.lang.Class object"
    anInstance := aClass.newInstance().
    ^super new 
          javaValue: anInstance; 
          javaClass: (Java newStatic: aClassName); "so we can access the class from the instance"
          yourself

Java (class)>>newStatic: aClassName
    aClass := self findClass: aClassName.
    ^super new
          javaValue: aClass;
          isStatic: true; "otherwise we can't tell if we want to execute methods of the java.lang.Class object or the static methods of the class itself"
          javaClass: (Java new: aClass); "if we wanted to get really meta ;-)"
          yourself

"Using them"
anInstance := Java new: 'Thingo'.
anInstance call: 'myInstanceMethod'.
theClass := Java newStatic: 'Thingo'.
theClass call: 'myClassMethod'.
anInstance callStatic: 'myClassMethod'. "convenience method"
anInstance javaClass call: 'myClassMethod'. "same as above"

So in the end you have two objects, one representing the Thingo class itself, and one representing an instance of Thingo. This way the actual method call is the same when operating directly on the appropriate object.
The behaviour I am aiming for is to call methods on Java objects; and classes are objects too ;-)

Without this distinction you would have a problem with the following I think.

public class Thingo {
    // Private constructor with factory method
    private Thingo() {}
    public static Thingo factoryMethod() {
        return new Thingo();
    }
}

anInstance := Java new: 'Thingo'. "Error. Can't call the constructor"
"Need to use"
anInstance := (Java newStatic: 'Thingo') call: 'factoryMethod'.

Since the standard instance creation would error, you wouldn't end up with an object to be able to call the factory method even---unless you hid the error, in which case you might think you had a valid instance when you didn't. So I think you need a way to have an object that represents the class, regardless of any instances.

ie: I have an instance of X and I want to call a method on it. I don't really care if it is static or not, that was an implementation issue but not my issue.

I think you have to have the distinction somewhere though, e.g. call: vs. callStatic: unless you wanted to make call: try an instance method first, if that is not valid call the class/static method. We could do that, but it might hide some class methods.

When writing (OO) code you decide if you want to implement a method as a class method or an instance method. At the end of the day, if someone is calling one of those methods (be it on a Java object or Smalltalk object), they are still going to be referencing the API and will know if it is a class method or not. I believe hiding the distinction behind a single call: method that does both for Java objects only would just confuse the issue and probably have a lot of edge cases that will break it.

I hope that makes my thinking clearer.

@jamesladd
Copy link
Owner

jamesladd commented Oct 11, 2017 via email

@mattys101
Copy link
Contributor Author

Yeah, exactly. Sorry about the confusion, no doubt further proliferated by my use of newStatic: ;-)

is JavaStatic should really be named JavaClass ;)

I thought about this when I was writing it up the first time. I was thinking JavaStatic because I had been looking at the JVM stuff, with invokeStatic and that sort of thing. Then I had a look at what JNIPort referred to it as, and it called it JavaStatic, so it validated my thinking :-O

So if you're happy with the two classes: Java and JavaClass that's the approach I'll take :-)

@jamesladd
Copy link
Owner

jamesladd commented Oct 11, 2017 via email

@mattys101
Copy link
Contributor Author

I've put what I'm doing over here
Wasn't sure if you wanted it all together what. Can always copy it later if/when you merge it in.

@jamesladd
Copy link
Owner

jamesladd commented Oct 16, 2017 via email

@mattys101
Copy link
Contributor Author

I'm trying to keep the implementation compatible with Smalltalk, using PrimObject everywhere. Then the idea is that to get the intial Smalltalk version working we will just be able to compile some Smalltalk methods into Java and JavaClass that use the special JVM receiver to output the bytecode that calls the methods implemented in Java.

Something like:

Smalltalk package: 'st.redline.kernel'.

Java methodAt: #call: put: [:arg |
    "I can't remember the exact details"
    JVM loadTemp: #arg.
    JVM invokeVirtual: 'call' class: 'st/redline/kernel/Java' signature: '(PrimObject)PrimObject'.
    JVM return.
]

What do you think?

@jamesladd
Copy link
Owner

jamesladd commented Oct 16, 2017 via email

@mattys101
Copy link
Contributor Author

I've been thinking about how to nicely map Smalltalk objects to Java objects---at the moment I have just been assuming everything has a javaValue---and I was thinking it might be good to have an 'implicit' conversion using an asJava method or similar that answers an instance of Java. This way people could just implement asJava for their class in Smalltalk and passing that object to a Java call: ... would work as expected. Classes that (will) already wrap a Java object (like String, Integer, and Java) could have a simple definition of asJava that just answers self.

I think this would help keep the wrapper code a bit cleaner, as it would basically be (in Smalltalk for brevity):

(anObject respondsTo: #asJava) ifTrue: [anObject asJava javaValue]

There are just two issues I think.

The first is how to handle objects that don't have an asJava method:

  1. Could just call javaValue(), which is the easiest but most likely lead to null pointer errors somewhere in the Java code. Which might be painful to debug. The upside of this is it might be more efficient for the primitive classes that already wrap Java objects if you get rid of the asJava implementation that returns self.
  2. Could have a default implementation of asJava that answers the PrimObject wrapped by a Java instance. This might be OK in some cases, but would most likely lead to errors in the Java code that would be painful to debug.
  3. Could have a default implementation of asJava that raises an exception. This way it would fail early if the user tries to pass a Smalltalk object that can't be converted in a Java call (or if they call asJava themselves) and they would know they need to implement an asJava method for their class. I'm currently favouring this approach

The second issue is mapping back from Java objects to Smalltalk. If the Smalltalk->Java is handled neatly by calling asJava the Java->Smalltalk will be kind of horrible as it will need code for all kinds of special cases---initially String, Integer, etc. comes to mind. I'd like to keep the wrappers as clean and lightweight as possible, but I think there needs to be a mapping for the basic types, otherwise it might be a little strange for a Smalltalk String to go in and a Java String to come back out, for example.

Thoughts?

@jamesladd
Copy link
Owner

jamesladd commented Oct 17, 2017 via email

@mattys101
Copy link
Contributor Author

Well, LogFactory itself would not necessarily use an asJava method, but say we had a Smalltalk Logger class and we were interfacing with some Java code and we wanted to make the logging in the Java code as close to that from the Smalltalk Logger as possible. You might have.

Object atSelector: #asJava put: [
    "early warning when we pass a Smalltalk object to a native Java method"
    self notJavaCompatibleSignal raise 
].
String atSelector: #asJava put: [ ^ self ]. "since String is directly compatible with Java"

Logger atSelector: #asJava put: [ 
    | logFactory logger |
    logFactory := (JavaClass for: '...commons.logging.LogFactory') call: 'getFactory'.
    logFactory call: 'setAttribute' with: ... with: ... . "configure the factory"
    logger := logFactory call: 'getInstance' with: (JavaClass for: 'java.lang.Object') .
    ^ logger
].

Then if you were to provide the Logger instance to a Java method asJava would automatically be called and the (wrapped) commons Log instance returned. (This is a bit of a dodgy example, you wouldn't configure the factory each time and you tend to not provide the Log instance, but if you did.)
You could also call asJava yourself to perform the configuration so that any Java code that requests that Log instance would get the correctly configured instance.

The initial definition of asJava on Object I think would make the interface with native Java code robust in that you would get a pretty early warning if you tried to pass a non-compatible Smalltalk object to a Java method call.

@jamesladd
Copy link
Owner

jamesladd commented Nov 2, 2017 via email

@jamesladd
Copy link
Owner

jamesladd commented Nov 2, 2017 via email

@mattys101
Copy link
Contributor Author

The idea is to return an instance of the Java wrapper around an appropriate plain Java object so that you can pass the Logger instance as an argument to a Java method (#call:with:) and it will implicitly retrieve an appropriate (wrapped) Java object. The example is not really the best, unfortunately.

You wouldn't want it to return the plain Java object itself as you wouldn't be able to call any methods on it, it would just error.

You also wouldn't want to return a wrapper around the Logger instance as that would just be a wrapped PrimObject which is most likely not the correct type for the Java api that you want to call.

It's funny, I feel like an #asJava method would be a nice thing to have, but I can't really think of a good example to demonstrate it :-P

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
2 participants
@jamesladd @mattys101