From 79303d36afc8b4b29f1c800e2080ff0d0f1e25ae Mon Sep 17 00:00:00 2001 From: Jiri Date: Thu, 22 Feb 2024 12:05:57 +0100 Subject: [PATCH] added presentation (and the used patch) --- README.md | 374 +++++++++++++++++++++----------------- jrd-WeakPoint/01.markdown | 30 +++ jrd-WeakPoint/02.markdown | 30 +++ jrd-WeakPoint/03.markdown | 30 +++ jrd-WeakPoint/04.markdown | 30 +++ jrd-WeakPoint/05.markdown | 30 +++ jrd-WeakPoint/06.markdown | 30 +++ jrd-WeakPoint/07.markdown | 30 +++ jrd-WeakPoint/08.markdown | 30 +++ jrd-WeakPoint/09.markdown | 30 +++ jrd-WeakPoint/10.markdown | 30 +++ jrd-WeakPoint/11.markdown | 30 +++ jrd-WeakPoint/12.markdown | 30 +++ jrd-WeakPoint/13.markdown | 30 +++ jrd-WeakPoint/14.markdown | 30 +++ jrd-WeakPoint/15.markdown | 30 +++ jrd.markdown | 199 ++++++++++++++++++++ jrd.markdown.patch | 26 +++ 18 files changed, 883 insertions(+), 166 deletions(-) create mode 100644 jrd-WeakPoint/01.markdown create mode 100644 jrd-WeakPoint/02.markdown create mode 100644 jrd-WeakPoint/03.markdown create mode 100644 jrd-WeakPoint/04.markdown create mode 100644 jrd-WeakPoint/05.markdown create mode 100644 jrd-WeakPoint/06.markdown create mode 100644 jrd-WeakPoint/07.markdown create mode 100644 jrd-WeakPoint/08.markdown create mode 100644 jrd-WeakPoint/09.markdown create mode 100644 jrd-WeakPoint/10.markdown create mode 100644 jrd-WeakPoint/11.markdown create mode 100644 jrd-WeakPoint/12.markdown create mode 100644 jrd-WeakPoint/13.markdown create mode 100644 jrd-WeakPoint/14.markdown create mode 100644 jrd-WeakPoint/15.markdown create mode 100644 jrd.markdown create mode 100644 jrd.markdown.patch diff --git a/README.md b/README.md index 2091b2b6..ecd456ee 100644 --- a/README.md +++ b/README.md @@ -1,168 +1,210 @@ -# Java Runtime (De)compiler -*Java Runtime (De)compiler*, or *JRD* for short, allows you to extract bytecode from the running JVM and decompile it with an external decompiler and compile back to running vm. -* talk from devconf 2022: https://www.youtube.com/watch?v=mXcLkaAxNvM&list=PLU1vS0speL2bf9Zfa0pHQLPBhEKbbbsua&index=55 - * quite a good overview -* Main usecases - * inspect and modify incompelte proprietary blobs - * watch how instrumetnation affets your code - * runtime hotpatching - * verifying what is actually running in your vm - * for testig, submit missbehaving or reach unreachable code -## Configuring decompiler agent -In order to start using Java-Runtime-Decompiler, you will need to select the Decompiler Agent's path in *Configure → Agent Path*. -The Decompiler Agent is a built-in project and can usually be found at `./decompiler_agent/target/decompiler-agent-*.jar`. The image should have agent preset. - -## Configuring external decompilers -Internal *javap* and *javap -v* decompiling tools are available by default. In image, we try to keep as many decompilers as possible bundled. - -Additionally, external decompilers are supported and can be configured in *Configure → Plugins*: -* You can download them using the links below and set them up yourself using the *New* button. -* You can use `mvn clean install -PdownloadPlugins` from a terminal in the project's directory and import the necessary files using the *Import* button. - -Currently supported decompilers are: -* [Fernflower](https://github.com/JetBrains/intellij-community/tree/master/plugins/java-decompiler/engine) -* [Procyon](https://bitbucket.org/mstrobel/procyon/downloads/) -* [CFR](https://github.com/leibnitz27/cfr/) -* [JD](https://github.com/java-decompiler/jd-core/) - -Assemblers/Disassemblers -* [jasm](https://github.com/openjdk/asmtools) (jdk 17 and up, with lambdas fixed) -* [jcoder](https://github.com/openjdk/asmtools) (jdk 17 and up) -* [jasm7](https://github.com/openjdk/asmtools) (jdk 11 and down, with known, broken lambdas) -* [jcoder7](https://github.com/openjdk/asmtools) (jdk 11 and down) -* [javap](https://github.com/openjdk/jdk) (disassemble only) -* [Procyon disassembler](https://bitbucket.org/mstrobel/procyon/downloads/)(disassemble only) - -Assemblers/Disassemblers with additional info -* [jasmG](https://github.com/openjdk/asmtools) (jdk 17 and up, with lambdas fixed, with -g switch) -* [jcoderG](https://github.com/openjdk/asmtools) (jdk 17 and up, with -g switch) -* [jasmG7](https://github.com/openjdk/asmtools) (jdk 11 and down, with known, broken lambdas, with -g switch) -* [jcoderG7](https://github.com/openjdk/asmtools) (jdk 11 and down, with -g switch) -* [javap-v](https://github.com/openjdk/jdk) (disassemble only, javap with -v switch) - -### From Fedora repositories -The Java-Runtime-Decompiler is packed together with Fernflower and Procyon decompilers in the main Fedora repositories and in EPEL 7 and EPEL 8 and EPEL 9. -``` -dnf install java-runtime-decompiler -``` -The resulting installation is fully preconfigured. - -## Usage -### Local Processes -The list on the top left shows all currently running processes on the local JVM with the name of their main class and their process ID. -Selecting any item will display the respective Loaded Classes and Decompiled Bytecode screens. -To return to the Welcome screen, simply unselect the item by holding Control while clicking it. -### Remote Processes -The list on the bottom left shows any remote processes. -Connections to remote processes can be established either by using the *+ button* to the right of the heading, or by going to *Connect → New Connection*. -### Local FS (file system) -Here you can turn JRD to casual, user-friendly jar/folder/file decompiler. You create a FS VM by setting up classpath as you are used for JVM. -### Loaded Classes -The middle list contains the classes present in the selected process. -This list can be reloaded with the *Refresh button* at the top and searched through with the *Search field*. -### Decompiled Bytecode -The text area on the right shows decompiled bytecode of the selected class of the selected process. -Different results may be achieved with different decompilers; you can select the decompiler from the dropdown menu at the top right. -### Overwriting classes -Using the *Overwrite button* at the top, you can replace the currently selected class' bytecode with your own compiled .class file via a dialog. - -![](https://user-images.githubusercontent.com/2904395/147475825-973679fe-585b-4afd-bff6-a7b19f02660c.png) - -### CLI -Commandline interface is powerful and allows for bulk processing of VM's, jars and much more...\ - -Enter `./start.sh --help` in a *Linux terminal* or `start.bat --help` in a *Windows CMD* to get started. -Decompile: -``` -$ ./start.sh -decompile classpathless-compiler-api-2.1.jar Cfr '.*' -saveas /tmp/cplc -savelike dir -Processing io.github.mkoncek.classpathless.api.ClassIdentifier -Saved: /tmp/cplc/io/github/mkoncek/classpathless/api/ClassIdentifier.java +# java-runtime-(de)compiler - talk for Brno CZ java user group + +see jrd.markdown for nonformated slides + +presentation runs in https://github.com/tisnik/vim-weakpoint vim plugin + +jrd-WeakPoint are generated slkides by vim-weakpoint: +``` +lua ~/.vim/bundle/vim-weakpoint/WeekPointSplitter.lua ./jrd.markdown -deduct -height 30 -vim +``` + +# java-runtime-(de)compiler + +https://github.com/judovana/java-runtime-decompiler + + * why? Because we can! + * binary blob reproducers + * obfuscated + * debuginfo + * swap lines + * oversee instrumetnations + * what! + * how? This talk! + * standard APIS only + +Jiri Vanek +Red Hat + +--PAGE----PAGE----PAGE----PAGE----PAGE----PAGE----PAGE----PAGE----PAGE----PAGE----PAGE----PAGE--- +# Demo I + + * install + * overview + * list and read +--PAGE----PAGE----PAGE----PAGE----PAGE----PAGE----PAGE----PAGE----PAGE----PAGE----PAGE----PAGE--- +# How - Agent attach I + +https://docs.oracle.com/javase/8/docs/jdk/api/attach/spec/com/sun/tools/attach/VirtualMachine.html + +``` +VirtualMachine vm = VirtualMachine.attach(Integer.toString(pid)); +vm.loadAgent(agentJar,”param1:value1,param:value2,…valueN”); +vm.detach(); +``` + * Agent is just bunch of classes which does nothing. No logic, no communication + * unless you write that +--PAGE----PAGE----PAGE----PAGE----PAGE----PAGE----PAGE----PAGE----PAGE----PAGE----PAGE----PAGE--- +# How - Agent attach II +``` +public static void premain(String agentArgs, Instrumentation inst); +``` + * Is the launched start point after agent is loaded, and instrumentation is what matters + * For example you can open ServerSocket to receive commands + * Agents can not be unloaded. Can be just turned off in best effort + * And you can register transformer! +--PAGE----PAGE----PAGE----PAGE----PAGE----PAGE----PAGE----PAGE----PAGE----PAGE----PAGE----PAGE--- +# Demo II +. + * self modifications + * classes + * classloaders I + * desompilers/disassemblers/hex + * –-patch-module + * (keyword based) code completion + * inner classes + * additional sources/binaries/diffs + * back compile/assemble + * real changes + * bytecode level + * java.lang.Override +--PAGE----PAGE----PAGE----PAGE----PAGE----PAGE----PAGE----PAGE----PAGE----PAGE----PAGE----PAGE--- +# How - Transformers + +https://docs.oracle.com/javase/8/docs/api/java/lang/instrument/Instrumentation.html +https://docs.oracle.com/javase/8/docs/api/java/lang/instrument/ClassFileTransformer.html + +``` +Transformer transformer = new Transformer(); +inst.addTransformer(transformer, true); +//repeat(work-earn-spend)until die +instrumentation.removeTransformer(transformer); +``` + * Classes can NEVER EVER be unloaded + * Depends on JVM? +--PAGE----PAGE----PAGE----PAGE----PAGE----PAGE----PAGE----PAGE----PAGE----PAGE----PAGE----PAGE--- +# Demo III +. + * self modifications + * Class Redefinition - hotswap - limits + * No new methods, no new fields, no renaming, no signatures… + * even non used BYTECODE fieLds limits + * bytecode level! + * Generally you can only change content of methods bodies (including new variables) + * byteman and JRD apis + * not add jar/classes + * dcevm/jrebel + * global counter +--PAGE----PAGE----PAGE----PAGE----PAGE----PAGE----PAGE----PAGE----PAGE----PAGE----PAGE----PAGE--- +# How - Trasnformer api +``` +byte[] transform(ClassLoader loader, String className, Class classBeingRedefined, ProtectionDomain protectionDomain, byte[] classfileBuffer) +Byte[] -> byte[] //… can not be more simple +``` + * asm ow2 + * If the class was never used, it will not be reachable from there + * Calling Class.forName on it will then do the job ... -Saved: /tmp/cplc/io/github/mkoncek/classpathless/api/MessagesListener.java - -``` - -Compile: -``` -$ ./start.sh -compile -cp classpathless-compiler-api-2.1.jar /tmp/cplc/io/github/mkoncek/classpathless/api/ClassIdentifier.java -savelike fqn -saveas . -... -Saved: ./io.github.mkoncek.classpathless.api.ClassIdentifier.class -``` - -Upload: -``` -$ ./start.sh -overwrite classpathless-compiler-api-2.1.jar io.github.mkoncek.classpathless.api.ClassIdentifier io.github.mkoncek.classpathless.api.ClassIdentifier.class -WARNING: Class package do not match directories. -Overwrite of class 'io.github.mkoncek.classpathless.api.ClassIdentifier' successful. -``` - -Disassemble: -``` -$ ./start.sh -decompile classpathless-compiler-api-2.1.jar jasm '.*' -saveas /tmp/cplc -savelike dir -Saved: /tmp/cplc/io/github/mkoncek/classpathless/api/ClassIdentifier.java -... -Saved: /tmp/cplc/io/github/mkoncek/classpathless/api/IdentifiedSource.java -Saved: /tmp/cplc/io/github/mkoncek/classpathless/api/MessagesListener.java -``` - -Assemble: -``` -$ ./start.sh -compile -p jasm /tmp/cplc -r -saveas /tmp/bin -... -jasm compiler caled with input of: 8 -entering into /tmp/jrd-jasm14590159245433727064tmp.dir -writing tmp file into /tmp/jrd-jasm14590159245433727064tmp.dir/src/io.github.mkoncek.classpathless.api.IdentifiedBytecode.java -... -Saved: /tmp/bin/io/github/mkoncek/classpathless/api/ClassIdentifier.class -Saved: /tmp/bin/io/github/mkoncek/classpathless/api/MessagesListener.class -``` - -Don't forget that all operations are same over classpath, remote vm, or process of VM - *runtime* compiler/decompiler! - -## For developers -## Relase -* Should be done in -Plegacy to enable injecting to jdk8 vms. - * note, that newest asmtools, are jdk16 and up -## Installation -* JRD 4 and lower: Note that Git, Maven & [JDK 8](https://adoptopenjdk.net/) with its tools.jar or [JDK11 and higher](https://adoptopenjdk.net/) are required to run or help develop this app.* -* JRD 5 and up: Note that Git, Maven & [JDK11 and higher](https://adoptopenjdk.net/) are required to run or help develop this app.* -### From GIT -#### Initial setup -``` -$ git clone https://github.com/pmikova/java-runtime-decompiler.git -$ cd java-runtime-decompiler -$ mvn clean install # builds the runtime decpompiler and agent, downloads also plugins -$ mvn clean install -PdownloadPlugins # builds the decompiler and downloads the decompiler plugins for future use, if not already downloaded -$ mvn clean install -Pimages # on Linux, bundles the plugins and JRD into a standalone portable image. Make some basic verifications -$ mvn clean install -Plegacy # take care to build agent in oldest resonable way - -# $PLUGINS and $VERIFY_CP variables may help to solve some weird image building issues. -``` -usually the development command is: -``` -$ mvn clean install -Dcheckstyle.skip -Dspotbugs.skip=true -``` -Note, that `-Dcheckstyle.skip -Dspotbugs.skip=true ` is not cowardly avoiding the valid issues those two finds, but to **format sources**. -It alo downloads all the plugins. -usually the release command is: -``` -$ mvn clean install -DskipTests -Pimages -Plegacy -``` -if the development command was never run, then plugins profile must be included -``` -$ mvn clean install -DskipTests -Pimages -Plegacy -PdownloadPlugins -``` -Note, that tests requires valid DISPLAY, to have some top level components (like HexEditor) pass - -Then, in images/target/runtime-decompiler... `./start.sh` in a *Linux terminal* or `start.bat` in a *Windows CMD* to start the application. The usage of top level start.sh/bat is only for development purposes. - -#### Known issues -* `mvn clean install` results in `BUILD FAILURE` with the error -`java.lang.RuntimeException: Unexpected error: java.security.InvalidAlgorithmParameterException: the trustAnchors parameter must be non-empty` -(spotted on Windows) - - **Temporary solution**: Use `mvn clean install -Dmaven.wagon.http.ssl.insecure=true -Dmaven.wagon.http.ssl.allowall=true -Dmaven.wagon.http.ssl.ignore.validity.dates=true` - to disregard SSL certificates when building. -* `mvn clean install` results in `TEST FAILURE` + * So what actually JRD simply does, is **Map** where overrides are stored, ad transform returns hit front hat map if found + * Similarly, if we require class, to send to client, we record it in this method + * You have to return the “modified” bytecode everytime class redefiniton happens (which is pretty often) + * Depends on VM, but on hotspot, one of the few places where it will not be picked up is during loop in method + +--PAGE----PAGE----PAGE----PAGE----PAGE----PAGE----PAGE----PAGE----PAGE----PAGE----PAGE----PAGE--- +# How - chaining of agents/transformers +. + * agents are modyfying and non modyfying + * agents are processed in order of attach + * modyfing (with Transformer) agents processed in order of reistered Transformer + * original class definition (immutable) -> byte[] -> + * Transformer1 -> byte[] -> + * Transformer.. -> byte[] -> + * TransformerN.. -> byte[] -> + * final class definition usage + *the full chain is always called. + * if you unregister, changes are gone + +--PAGE----PAGE----PAGE----PAGE----PAGE----PAGE----PAGE----PAGE----PAGE----PAGE----PAGE----PAGE--- +# Demo IV + + * byteman + * second agent! + * silent non sucess + * chaining of agents + * code coverage instrumentation + * multiple class defintions classloaders II + * no change in gui + * modules + +--PAGE----PAGE----PAGE----PAGE----PAGE----PAGE----PAGE----PAGE----PAGE----PAGE----PAGE----PAGE--- +# Demo V + + * cli + * adding classes + * ServerImplNw + * patch + * not as simple + * split? + * full java files? + * full binary classes? + * Future - debugger attach + * source-codeless IDE + +--PAGE----PAGE----PAGE----PAGE----PAGE----PAGE----PAGE----PAGE----PAGE----PAGE----PAGE----PAGE--- +# More hardcore ways to overcome hotswap limitations +. + * Class Redefinition - hotswap - limits + * No new methods, no new fields, no renaming… + * Generally you can only change content of methods bodies (including new variables) + * DCEVM JDK based on Phd these is removing those limits with (huge) cost of performance + * Take ages after JDK (jdk 17 not yet properly out, last udpate December 2022) + * Jetbrains took it recently over + * Jrebel JVMTI hacks + * Tricks + * In agent predefined map of objects to store fields/methods + * Bytemen do it behind the doors + * JRD offer sclumsy appi for it + * Add a new class with reimplementation and call it where needed + * inst.appendToSystem/BootClassLoader + + +--PAGE----PAGE----PAGE----PAGE----PAGE----PAGE----PAGE----PAGE----PAGE----PAGE----PAGE----PAGE--- +# CPLC I +. + * java compiler api is terrible +``` +public interface ClassesProvider { + Collection getClass(ClassIdentifier... names); + List getClassPathListing(); +} +``` +``` +public interface ClasspathlessCompiler { + Collection compileClass(ClassesProvider classesProvider, + Optional messagesListener, IdentifiedSource... javaSourceFiles); + .... +} +``` +--PAGE----PAGE----PAGE----PAGE----PAGE----PAGE----PAGE----PAGE----PAGE----PAGE----PAGE----PAGE--- +# CPLC II +. + * init of classes + * in case of jrd easy to get dependences from bytecode + * for eg sources based compiler no op + * the default implementations is classical class-path based javac + * sources based plugins and many others + * JEP 458: Launch Multi-File Source-Code Programs + * (class-path based compiling classloader) +--PAGE----PAGE----PAGE----PAGE----PAGE----PAGE----PAGE----PAGE----PAGE----PAGE----PAGE----PAGE--- +# Enjoy! +.... + https://github.com/judovana/java-runtime-decompiler + https://github.com/mkoncek/classpathless-compiler + https://byteman.jboss.org/ + https://dcevm.github.io/ + https://docs.oracle.com/javase/8/docs/jdk/api/attach/spec/com/sun/tools/attach/VirtualMachine.html + https://docs.oracle.com/javase/8/docs/api/java/lang/instrument/Instrumentation.html + https://docs.oracle.com/javase/8/docs/api/java/lang/instrument/ClassFileTransformer.html +.... + https://github.com/tisnik/vim-weakpoint - **Temporary solution**: Use `mvn clean install -DskipTests` to not run test classes when building. diff --git a/jrd-WeakPoint/01.markdown b/jrd-WeakPoint/01.markdown new file mode 100644 index 00000000..e715f875 --- /dev/null +++ b/jrd-WeakPoint/01.markdown @@ -0,0 +1,30 @@ +# java-runtime-(de)compiler + +https://github.com/judovana/java-runtime-decompiler + + * why? Because we can! + * binary blob reproducers + * obfuscated + * debuginfo + * swap lines + * oversee instrumetnations + * what! + * how? This talk! + * standard APIS only + +Jiri Vanek +Red Hat + + + + + + + + + + + + + + diff --git a/jrd-WeakPoint/02.markdown b/jrd-WeakPoint/02.markdown new file mode 100644 index 00000000..8546ecdb --- /dev/null +++ b/jrd-WeakPoint/02.markdown @@ -0,0 +1,30 @@ +# Demo I + + * install + * overview + * list and read + + + + + + + + + + + + + + + + + + + + + + + + + diff --git a/jrd-WeakPoint/03.markdown b/jrd-WeakPoint/03.markdown new file mode 100644 index 00000000..c282eb5c --- /dev/null +++ b/jrd-WeakPoint/03.markdown @@ -0,0 +1,30 @@ +# How - Agent attach I + +https://docs.oracle.com/javase/8/docs/jdk/api/attach/spec/com/sun/tools/attach/VirtualMachine.html + +``` +VirtualMachine vm = VirtualMachine.attach(Integer.toString(pid)); +vm.loadAgent(agentJar,”param1:value1,param:value2,…valueN”); +vm.detach(); +``` + * Agent is just bunch of classes which does nothing. No logic, no communication + * unless you write that + + + + + + + + + + + + + + + + + + + diff --git a/jrd-WeakPoint/04.markdown b/jrd-WeakPoint/04.markdown new file mode 100644 index 00000000..d35721c6 --- /dev/null +++ b/jrd-WeakPoint/04.markdown @@ -0,0 +1,30 @@ +# How - Agent attach II +``` +public static void premain(String agentArgs, Instrumentation inst); +``` + * Is the launched start point after agent is loaded, and instrumentation is what matters + * For example you can open ServerSocket to receive commands + * Agents can not be unloaded. Can be just turned off in best effort + * And you can register transformer! + + + + + + + + + + + + + + + + + + + + + + diff --git a/jrd-WeakPoint/05.markdown b/jrd-WeakPoint/05.markdown new file mode 100644 index 00000000..a02ea141 --- /dev/null +++ b/jrd-WeakPoint/05.markdown @@ -0,0 +1,30 @@ +# Demo II +. + * self modifications + * classes + * classloaders I + * desompilers/disassemblers/hex + * –-patch-module + * (keyword based) code completion + * inner classes + * additional sources/binaries/diffs + * back compile/assemble + * real changes + * bytecode level + * java.lang.Override + + + + + + + + + + + + + + + + diff --git a/jrd-WeakPoint/06.markdown b/jrd-WeakPoint/06.markdown new file mode 100644 index 00000000..6567a030 --- /dev/null +++ b/jrd-WeakPoint/06.markdown @@ -0,0 +1,30 @@ +# How - Transformers + +https://docs.oracle.com/javase/8/docs/api/java/lang/instrument/Instrumentation.html +https://docs.oracle.com/javase/8/docs/api/java/lang/instrument/ClassFileTransformer.html + +``` +Transformer transformer = new Transformer(); +inst.addTransformer(transformer, true); +//repeat(work-earn-spend)until die +instrumentation.removeTransformer(transformer); +``` + * Classes can NEVER EVER be unloaded + * Depends on JVM? + + + + + + + + + + + + + + + + + diff --git a/jrd-WeakPoint/07.markdown b/jrd-WeakPoint/07.markdown new file mode 100644 index 00000000..8678037d --- /dev/null +++ b/jrd-WeakPoint/07.markdown @@ -0,0 +1,30 @@ +# Demo III +. + * self modifications + * Class Redefinition - hotswap - limits + * No new methods, no new fields, no renaming, no signatures… + * even non used BYTECODE fieLds limits + * bytecode level! + * Generally you can only change content of methods bodies (including new variables) + * byteman and JRD apis + * not add jar/classes + * dcevm/jrebel + * global counter + + + + + + + + + + + + + + + + + + diff --git a/jrd-WeakPoint/08.markdown b/jrd-WeakPoint/08.markdown new file mode 100644 index 00000000..2693fe8a --- /dev/null +++ b/jrd-WeakPoint/08.markdown @@ -0,0 +1,30 @@ +# How - Trasnformer api +``` +byte[] transform(ClassLoader loader, String className, Class classBeingRedefined, ProtectionDomain protectionDomain, byte[] classfileBuffer) +Byte[] -> byte[] //… can not be more simple +``` + * asm ow2 + * If the class was never used, it will not be reachable from there + * Calling Class.forName on it will then do the job +... + * So what actually JRD simply does, is **Map** where overrides are stored, ad transform returns hit front hat map if found + * Similarly, if we require class, to send to client, we record it in this method + * You have to return the “modified” bytecode everytime class redefiniton happens (which is pretty often) + * Depends on VM, but on hotspot, one of the few places where it will not be picked up is during loop in method + + + + + + + + + + + + + + + + + diff --git a/jrd-WeakPoint/09.markdown b/jrd-WeakPoint/09.markdown new file mode 100644 index 00000000..4d4a1559 --- /dev/null +++ b/jrd-WeakPoint/09.markdown @@ -0,0 +1,30 @@ +# How - chaining of agents/transformers +. + * agents are modyfying and non modyfying + * agents are processed in order of attach + * modyfing (with Transformer) agents processed in order of reistered Transformer + * original class definition (immutable) -> byte[] -> + * Transformer1 -> byte[] -> + * Transformer.. -> byte[] -> + * TransformerN.. -> byte[] -> + * final class definition usage + *the full chain is always called. + * if you unregister, changes are gone + + + + + + + + + + + + + + + + + + diff --git a/jrd-WeakPoint/10.markdown b/jrd-WeakPoint/10.markdown new file mode 100644 index 00000000..679feb25 --- /dev/null +++ b/jrd-WeakPoint/10.markdown @@ -0,0 +1,30 @@ +# Demo IV + + * byteman + * second agent! + * silent non sucess + * chaining of agents + * code coverage instrumentation + * multiple class defintions classloaders II + * no change in gui + * modules + + + + + + + + + + + + + + + + + + + + diff --git a/jrd-WeakPoint/11.markdown b/jrd-WeakPoint/11.markdown new file mode 100644 index 00000000..db528b8f --- /dev/null +++ b/jrd-WeakPoint/11.markdown @@ -0,0 +1,30 @@ +# Demo V + + * cli + * adding classes + * ServerImplNw + * patch + * not as simple + * split? + * full java files? + * full binary classes? + * Future - debugger attach + * source-codeless IDE + + + + + + + + + + + + + + + + + + diff --git a/jrd-WeakPoint/12.markdown b/jrd-WeakPoint/12.markdown new file mode 100644 index 00000000..9b8e2b35 --- /dev/null +++ b/jrd-WeakPoint/12.markdown @@ -0,0 +1,30 @@ +# More hardcore ways to overcome hotswap limitations +. + * Class Redefinition - hotswap - limits + * No new methods, no new fields, no renaming… + * Generally you can only change content of methods bodies (including new variables) + * DCEVM JDK based on Phd these is removing those limits with (huge) cost of performance + * Take ages after JDK (jdk 17 not yet properly out, last udpate December 2022) + * Jetbrains took it recently over + * Jrebel JVMTI hacks + * Tricks + * In agent predefined map of objects to store fields/methods + * Bytemen do it behind the doors + * JRD offer sclumsy appi for it + * Add a new class with reimplementation and call it where needed + * inst.appendToSystem/BootClassLoader + + + + + + + + + + + + + + + diff --git a/jrd-WeakPoint/13.markdown b/jrd-WeakPoint/13.markdown new file mode 100644 index 00000000..0343d9b2 --- /dev/null +++ b/jrd-WeakPoint/13.markdown @@ -0,0 +1,30 @@ +# CPLC I +. + * java compiler api is terrible +``` +public interface ClassesProvider { + Collection getClass(ClassIdentifier... names); + List getClassPathListing(); +} +``` +``` +public interface ClasspathlessCompiler { + Collection compileClass(ClassesProvider classesProvider, + Optional messagesListener, IdentifiedSource... javaSourceFiles); + .... +} +``` + + + + + + + + + + + + + + diff --git a/jrd-WeakPoint/14.markdown b/jrd-WeakPoint/14.markdown new file mode 100644 index 00000000..668883fe --- /dev/null +++ b/jrd-WeakPoint/14.markdown @@ -0,0 +1,30 @@ +# CPLC II +. + * init of classes + * in case of jrd easy to get dependences from bytecode + * for eg sources based compiler no op + * the default implementations is classical class-path based javac + * sources based plugins and many others + * JEP 458: Launch Multi-File Source-Code Programs + * (class-path based compiling classloader) + + + + + + + + + + + + + + + + + + + + + diff --git a/jrd-WeakPoint/15.markdown b/jrd-WeakPoint/15.markdown new file mode 100644 index 00000000..c1444ce3 --- /dev/null +++ b/jrd-WeakPoint/15.markdown @@ -0,0 +1,30 @@ +# Enjoy! +.... + https://github.com/judovana/java-runtime-decompiler + https://github.com/mkoncek/classpathless-compiler + https://byteman.jboss.org/ + https://dcevm.github.io/ + https://docs.oracle.com/javase/8/docs/jdk/api/attach/spec/com/sun/tools/attach/VirtualMachine.html + https://docs.oracle.com/javase/8/docs/api/java/lang/instrument/Instrumentation.html + https://docs.oracle.com/javase/8/docs/api/java/lang/instrument/ClassFileTransformer.html +.... + https://github.com/tisnik/vim-weakpoint + + + + + + + + + + + + + + + + + + + diff --git a/jrd.markdown b/jrd.markdown new file mode 100644 index 00000000..5b3958a7 --- /dev/null +++ b/jrd.markdown @@ -0,0 +1,199 @@ +# java-runtime-(de)compiler + +https://github.com/judovana/java-runtime-decompiler + + * why? Because we can! + * binary blob reproducers + * obfuscated + * debuginfo + * swap lines + * oversee instrumetnations + * what! + * how? This talk! + * standard APIS only + +Jiri Vanek +Red Hat + +--PAGE----PAGE----PAGE----PAGE----PAGE----PAGE----PAGE----PAGE----PAGE----PAGE----PAGE----PAGE--- +# Demo I + + * install + * overview + * list and read +--PAGE----PAGE----PAGE----PAGE----PAGE----PAGE----PAGE----PAGE----PAGE----PAGE----PAGE----PAGE--- +# How - Agent attach I + +https://docs.oracle.com/javase/8/docs/jdk/api/attach/spec/com/sun/tools/attach/VirtualMachine.html + +``` +VirtualMachine vm = VirtualMachine.attach(Integer.toString(pid)); +vm.loadAgent(agentJar,”param1:value1,param:value2,…valueN”); +vm.detach(); +``` + * Agent is just bunch of classes which does nothing. No logic, no communication + * unless you write that +--PAGE----PAGE----PAGE----PAGE----PAGE----PAGE----PAGE----PAGE----PAGE----PAGE----PAGE----PAGE--- +# How - Agent attach II +``` +public static void premain(String agentArgs, Instrumentation inst); +``` + * Is the launched start point after agent is loaded, and instrumentation is what matters + * For example you can open ServerSocket to receive commands + * Agents can not be unloaded. Can be just turned off in best effort + * And you can register transformer! +--PAGE----PAGE----PAGE----PAGE----PAGE----PAGE----PAGE----PAGE----PAGE----PAGE----PAGE----PAGE--- +# Demo II +. + * self modifications + * classes + * classloaders I + * desompilers/disassemblers/hex + * –-patch-module + * (keyword based) code completion + * inner classes + * additional sources/binaries/diffs + * back compile/assemble + * real changes + * bytecode level + * java.lang.Override +--PAGE----PAGE----PAGE----PAGE----PAGE----PAGE----PAGE----PAGE----PAGE----PAGE----PAGE----PAGE--- +# How - Transformers + +https://docs.oracle.com/javase/8/docs/api/java/lang/instrument/Instrumentation.html +https://docs.oracle.com/javase/8/docs/api/java/lang/instrument/ClassFileTransformer.html + +``` +Transformer transformer = new Transformer(); +inst.addTransformer(transformer, true); +//repeat(work-earn-spend)until die +instrumentation.removeTransformer(transformer); +``` + * Classes can NEVER EVER be unloaded + * Depends on JVM? +--PAGE----PAGE----PAGE----PAGE----PAGE----PAGE----PAGE----PAGE----PAGE----PAGE----PAGE----PAGE--- +# Demo III +. + * self modifications + * Class Redefinition - hotswap - limits + * No new methods, no new fields, no renaming, no signatures… + * even non used BYTECODE fieLds limits + * bytecode level! + * Generally you can only change content of methods bodies (including new variables) + * byteman and JRD apis + * not add jar/classes + * dcevm/jrebel + * global counter +--PAGE----PAGE----PAGE----PAGE----PAGE----PAGE----PAGE----PAGE----PAGE----PAGE----PAGE----PAGE--- +# How - Trasnformer api +``` +byte[] transform(ClassLoader loader, String className, Class classBeingRedefined, ProtectionDomain protectionDomain, byte[] classfileBuffer) +Byte[] -> byte[] //… can not be more simple +``` + * asm ow2 + * If the class was never used, it will not be reachable from there + * Calling Class.forName on it will then do the job +... + * So what actually JRD simply does, is **Map** where overrides are stored, ad transform returns hit front hat map if found + * Similarly, if we require class, to send to client, we record it in this method + * You have to return the “modified” bytecode everytime class redefiniton happens (which is pretty often) + * Depends on VM, but on hotspot, one of the few places where it will not be picked up is during loop in method + +--PAGE----PAGE----PAGE----PAGE----PAGE----PAGE----PAGE----PAGE----PAGE----PAGE----PAGE----PAGE--- +# How - chaining of agents/transformers +. + * agents are modyfying and non modyfying + * agents are processed in order of attach + * modyfing (with Transformer) agents processed in order of reistered Transformer + * original class definition (immutable) -> byte[] -> + * Transformer1 -> byte[] -> + * Transformer.. -> byte[] -> + * TransformerN.. -> byte[] -> + * final class definition usage + *the full chain is always called. + * if you unregister, changes are gone + +--PAGE----PAGE----PAGE----PAGE----PAGE----PAGE----PAGE----PAGE----PAGE----PAGE----PAGE----PAGE--- +# Demo IV + + * byteman + * second agent! + * silent non sucess + * chaining of agents + * code coverage instrumentation + * multiple class defintions classloaders II + * no change in gui + * modules + +--PAGE----PAGE----PAGE----PAGE----PAGE----PAGE----PAGE----PAGE----PAGE----PAGE----PAGE----PAGE--- +# Demo V + + * cli + * adding classes + * ServerImplNw + * patch + * not as simple + * split? + * full java files? + * full binary classes? + * Future - debugger attach + * source-codeless IDE + +--PAGE----PAGE----PAGE----PAGE----PAGE----PAGE----PAGE----PAGE----PAGE----PAGE----PAGE----PAGE--- +# More hardcore ways to overcome hotswap limitations +. + * Class Redefinition - hotswap - limits + * No new methods, no new fields, no renaming… + * Generally you can only change content of methods bodies (including new variables) + * DCEVM JDK based on Phd these is removing those limits with (huge) cost of performance + * Take ages after JDK (jdk 17 not yet properly out, last udpate December 2022) + * Jetbrains took it recently over + * Jrebel JVMTI hacks + * Tricks + * In agent predefined map of objects to store fields/methods + * Bytemen do it behind the doors + * JRD offer sclumsy appi for it + * Add a new class with reimplementation and call it where needed + * inst.appendToSystem/BootClassLoader + + +--PAGE----PAGE----PAGE----PAGE----PAGE----PAGE----PAGE----PAGE----PAGE----PAGE----PAGE----PAGE--- +# CPLC I +. + * java compiler api is terrible +``` +public interface ClassesProvider { + Collection getClass(ClassIdentifier... names); + List getClassPathListing(); +} +``` +``` +public interface ClasspathlessCompiler { + Collection compileClass(ClassesProvider classesProvider, + Optional messagesListener, IdentifiedSource... javaSourceFiles); + .... +} +``` +--PAGE----PAGE----PAGE----PAGE----PAGE----PAGE----PAGE----PAGE----PAGE----PAGE----PAGE----PAGE--- +# CPLC II +. + * init of classes + * in case of jrd easy to get dependences from bytecode + * for eg sources based compiler no op + * the default implementations is classical class-path based javac + * sources based plugins and many others + * JEP 458: Launch Multi-File Source-Code Programs + * (class-path based compiling classloader) +--PAGE----PAGE----PAGE----PAGE----PAGE----PAGE----PAGE----PAGE----PAGE----PAGE----PAGE----PAGE--- +# Enjoy! +.... + https://github.com/judovana/java-runtime-decompiler + https://github.com/mkoncek/classpathless-compiler + https://byteman.jboss.org/ + https://dcevm.github.io/ + https://docs.oracle.com/javase/8/docs/jdk/api/attach/spec/com/sun/tools/attach/VirtualMachine.html + https://docs.oracle.com/javase/8/docs/api/java/lang/instrument/Instrumentation.html + https://docs.oracle.com/javase/8/docs/api/java/lang/instrument/ClassFileTransformer.html +.... + https://github.com/tisnik/vim-weakpoint + diff --git a/jrd.markdown.patch b/jrd.markdown.patch new file mode 100644 index 00000000..a082c93a --- /dev/null +++ b/jrd.markdown.patch @@ -0,0 +1,26 @@ +--- /dev/null ++++ es.ServerImplNw +@@ -0,0 +1,11 @@ ++package es; ++ ++public class ServerImplNw implements IServer { ++ ++ public void send(String s) { ++ System.out.println("Story sending: " + new java.util.Date() + " " + s); ++ } ++ public void recieve(String s) { ++ System.out.println("Story received: " + new java.util.Date() + " " + s); ++ } ++} + +--- es.ServerFactory ++++ es.ServerFactory +@@ -12,6 +12,6 @@ + + public class ServerFactory { + public static IServer createServer() { +- return new ServerImpl(); ++ return new ServerImplNw(); + } + } +