Rewrite original anonymous to allow new ones

[andrewleech]

I've been porting my old (buggy) fully-decompiled / mod / recompiled apk changes to new concise dpx based ones. A lot of my original code makes heavy use of convenient inline / anonymous classes, which are a bit of a pain to have to split out by hand.

One of the tricks I came up with in my previous re-engineering efforts was re-writing the original apk/jar so that the anonymous functions were a little easier to deal with.

Use apktool to extract, then a python script to update all the smali to turn $1 to $Anon1 (and so on) then rebuild. This renames all the anonymous inner classes and the access() functions so can be referenced in a jar and no longer clash with new anonymous functions created by javac

This trick seems to work exceptionally well with Dexpatcher, I've re-written my original APK in this way and now if I set defaultAction to ADD anonymous inner's seem to be working fine.

For reference, this is my basic python script, my target application's code is all in the package com.navdy.* (I didn't want to rewrite classes from other packages)

# java -jar apktool_2.4.0.jar d ../apk/Hud-3080.apk
# python3 ./3080_dupes.py
# java -jar apktool_2.4.0.jar b Hud-3080
# apksigner.bat sign --ks ..\navdy_alelec.jks --out ..\apk\Hud-3080-Anon.apk Hud-3080\dist\Hud-3080.apk

import sys
import os, re
import shutil
import fileinput
from pathlib import Path

for root, dirs, files in os.walk("./Hud-3080"):
   for name in files:

        fpath = os.path.join(root, name)
        fname = origname = Path(fpath)

        if 'com/navdy/' not in fpath:
            continue

        if not fpath.endswith('.smali'):
            continue
       
        print(fname.name)

        if re.search(r'\$\d', fname.name):
            target = fname.with_name(fname.name.replace('$', '$Anon'))            
            print(fname, "->", target)
            fname.rename(target)
            
        else:
            target = fname

        text = target.read_text()
        def add_anon(match):
            return "$Anon" + match.group(1)
        text = re.sub(r'\$(\d+)', add_anon, text)
        target.write_text(text)

I haven't looked into the internals of dexpatcher yet but surely this same trick would be fairly easy to achieve in the dexpatcher tool or the gradle plugin? Is this likely to break something else (other than the general issues to look out for when defaultAction = ADD) ?

[Lanchon]

defaultAction is really for extreme cases. it WILL bite you back if you use it, sooner or later. why do you need it? where are you using it? classes by default are add, and you can't change that. this includes nested/inner/anon classes.

renaming source classes is a big no for me. it cant be done in a way thats guaranteed to work. you could rename patch classes, but i dont see the point. anon classes dont add much, you can use named inner classes instead.

if you have to write too many anon classes in one particular case, you can use a hooking class (DexEdit) that calls into a worker class (DexAdd). the worker class can use anons.

accessors are not needed: just make things package private instead of private.

if you have to patch an anon class, then yes, you are in trouble. in many cases, anon classes are trivial, so you could just patch the outer class and make it create a named class instance instead, which you completely redefine.

otherwise, if you are willing to patch the anon class by its $N number (ugly), you can do a cross-class edit.

dxp would need a query language for this: eg: target the anon class(es) of MyClass that implements Runnable and has method xxx().

[andrewleech]

I'm beginning to understand the pain of defaultAction already, I think it's complicating the class/patch I'm working on at the moment. I think it was only needed when access functions are involved, I didn't realise it was possible to have inner classes that didn't result in them being generated (the original apk has a lot of them). I'll try more @DexEdit access modifiers on fields to change their permissions to see if that works in a cleaner fashion.

The original project I'm working with uses a lot of large inner anonymous classes, up to 4 deep at times.
There was a lot of chained deferring of functions via handler/Runnable as well as a lot of listener interfaces at various levels. Many of these pass a lot of variables through to the inner anonymous which is much quicker/easier in an inline class or lambda function.

This project isn't a typical app, it's a heads up display for a car running android behind the scenes where the one large apk is basically the entire single purpose of the device.

The global once-off rewrite of all anon seems to be reliable for me, I'll probably keep it in place as it avoids any name clashes.

[Lanchon]

the rename might be ok for you, but it can break stuff in the general case. so keep it.

dont use defaultAction. define your members more visible instead.

if you want to access an existing private field from a new inner class, edit the field to increase its visibility. you can also define your explicit access methods in the outer class.

if you want to access an existing private INSTANCE METHOD from a new inner class, you CANT edit the method to increase its visibility: you would be changing a direct method into a virtual method. these method types need different dex code to be invoked, so you would be braking existing code. you'll get a warning or error from dexpatcher if you try. (static methods are all direct, so no issue there.) so you have to define your explicit access methods in the outer class to invoke the private method instead.

[andrewleech]

Thanks, I spent a few good hours yesterday writing patches and I think I'm starting to get the hang of it. I do need to pay closer attention to the warnings, ran into a lot of those virtual/direct runtime errors before I figured out it was a private in original code vs package in patch and that I needed my own access type fn to bridge the difference.
That might be a good one for your mini-faq just what that error means... or maybe just tell people to look closer at warnings ;-)

Thanks again!

[Lanchon]

hey,

i put out the beta, this release is taking shape now, i hope u subscribed to releases and saw it.

i have a detailed grand magnum plan for all these things that will never see the light of that because i simply don't have the time to code all the stuff, but it would beautifully handle obfuscated code and the kind of issues you are up against.

but you got me thinking again on a half-baked idea i had before the grand scheme and i sort of got it all sorted out now. it isn't beautiful, it doesn't "help" in patching obfuscated code but it does enable it, and overall it broadens the options and makes a lot of use cases more comfortable. like yours. the good news is that, although it is not beautiful, this can be implemented in a reasonable time frame.

so although ill be travelling the following months while i wait for summer to come back to the southern hemisphere, now at least i have a down to earth plan that will very much solve issues like patching anon classes and functions... if not beautifully. just letting you know.

how is progress with you HUD? btw, what car is it for? or is it aftermarket?

[andrewleech]

That sounds neat, I'll keep a watch for it. Enjoy your travels!
I'll just hide out in the cold down here furiously coding :-(

The rebuilding of my features in dexpatcher is going great! It's for navdy, which is an aftermarket hud from a now-closed crowd funded company. Thankfully they didn't use any obfuscater on the code so it decompiles relatively ok... though they clearly did use a bit of kotlin which confuses things a bit. I'm basically using just a reference decompilation from jadx to inform what to patch as I add new features into the original apk's (a couple on the hud iteslf, and a separate companion phone app)

If you ever want any not-best-practice examples of dexpatcher in action it's all happening at https://gitlab.com/alelec/navdy/alelec_navdy_client/commits/develop_dxp and https://gitlab.com/alelec/navdy/navdy-display-Hud-java/commits/3080_develop

fwiw I am still using my renamed anon's and a fair bit of defaultAction=ADD which isn't failing much yet... I look forward to your not-as-nasty method in future!

[Lanchon]

you know you can access decompiled code directly in idea right? idea code navigation features work there.

[andrewleech]

Yeah but it's built in decompiler (fernflower) struggles a lot more on my apk's than jadx, there's plenty it can't decompile at all.
Having the reference decompilation on disk makes it easier to track in git exactly what changes I'm making too.

[Lanchon]

is it fern? my idea uses CFR (although it passes to little info to it), i must be using a plugin

[andrewleech]

Sorry yeah I think I is CFR, either way I've got far more accurate, configurable and comprehensive results from jadx (and I've tried them all many times, mostly through the tools konloch/bytecode-viewer, luyten and jadx-gui).

I've been meaning to figure out how to package up the jadx output into a source-jar and pointing the project to it properly, bit haven't got around to that.

I have made a quick script though to regex add most of the needed @DexIgnore (and comment out static assignments) to a jadx file and copy or into my Java tree, which makes it really quick to pull in a file to start editing.

[Lanchon]

i definitely have to try jadx then. i always got better results from CFR and i stopped trying others, but its been a long while. i'll look into it at some point.

i don't like importSymbols that much and "batch" importing of full classes. and i definitely don't like DexIgnore. read bullet points 3 to 4 here: #14

[Lanchon]

hey @andrewleech, how is your project going?

i'm finally having some time to add dex transforms to dexpatcher. for now only transforms for basic but full handling of obfuscated code are in place.

but...

take a good look at what's coming here... c565348

i made those transforms for you :)

they are a bit different from what you are using, because a 2-level nested anonymous class Foo$3$1 would be translated to something like Foo$Anon3$AnonAnon1, because i found out that Java cannot define an inner class matching the name of one of its outer classes (so Foo$Anon3$Anon1 is not a helpful mapping).

all transforms are done on-the-fly during dex process and output write phases whenever any transformed items are visited. dxp doesn't build or keep transformed models of dex files in memory, so memory usage is generally not impacted by the use of chained transforms.

the flipside to this is that diagnostics of transforms come at unexpected times during processing instead of lumped in one place. and --dry-run doesn't even transform the whole way, maybe.

because of this, transforms do use memory when logging, to cache the log items as events happen and ensure duplicate events are never output. so extra memory is used in the end, but only if errors occur (or debug logging is enabled).

i may add optional thorough-visiting stages to force lumping of transform log items, but these stages will slow down processing for the luxury of a more ordered log.

so in the linked commit you'll find source and patch samples and also the output of processing. since you are sort of already using this feature, do you have any comments to this implementation before it gets written in stone?

would you like me to prepare an alpha release for you to test?

thanks!

[andrewleech]

Hey, cheers, yeah I saw some changes come through on main dexpatcher project, thought they looked releated!

Yep that inner not allowed the same as outer is quite annoying, it does complicate working with decompiled code.

What you've got there does look good to me, what I can follow easily at least. The AnonAnon1 doesn't particularly worry me, it's a valid workaround to keep the compiler happy!

I'd be very happy to test an alpha release, I've actually been using DexPatcher a heap lately on a new, second project: https://gitlab.com/alelec/fossil_smartwatches_alelec_android/
I've recently got a fossil hybrid smart watch, the android app for which is pretty limited - so I'm improving it!
DexPatcher's been working great, though I did use my "re-write the apk with Anon's renames in the smali" trick to get it going. It'd be a great test case to try without needing that pre-processing as I'll likely want to keep updating my project with new official apk's that're officially released.

One little unrelated thing I've noticed with the android studio gradle plugin, often every second build fails due to the compiler not finding a bunch of things I've changed with DexReplace. I hit build again and it always works though. Have you seen this? Thinking about it now, it might be from cases

[Lanchon]

One little unrelated thing I've noticed with the android studio gradle plugin, often every second build fails due to the compiler not finding a bunch of things I've changed with DexReplace. I hit build again and it always works though. Have you seen this? Thinking about it now, it might be from cases

one problem with the project is the small user base. i'm not really dogfooding it because i no longer have time to hack apps myself: either i work on dxp or hack apps, but can't do both. so no, i haven't seen this behavior so far. if you have a project state that reproduces this consistently, post the full project if you can (maybe just tag a commit in your project?) and i'd gladly take a look at it (but possibly not immediately).

regarding the small user base: i am now 100% technically sure that i can implement DexPatcher-live for apps!!!... in android 9 :(

live is an android system mod to apply dxp patches on the fly without patching the apk files themselves and without invalidating any signatures. it might be possible also for framework, but not in the first release. it is comparable to xposed but safe: patches don't have any rights except over the app they apply on, while xposed modules are always root and could all be trojans. they'd be easy to write instead of the nightmare that is writing xposed modules. would not require reboots to update patches, just an app restart. on the other hand, the first release would patch only code: not resources and not the manifest, which is a drag. now that xposed is kind of dying, this could be an opportunity for the project... if i only had more time.

android 10 has changed things drastically with the new APEX packages, so it would be very hard to patch the framework: inside layers of overlapping APEX packages, which contain ext4 partitions, which contain... odexed files! which need to be deodexed, then live-patched. it's hell. but apps on android 10 might still be possible. and if you are willing to deodex your rom (if that can even be done on android 10), then maybe the framework too.

now back to the subject...

i'll try to get an alpha release ready tonight. i can surely use your testing. thanks!!

[andrewleech]

Geez, I hadn't heard of APEX before, yeah that's some more layers of lockdown there.

Live patching of apps would be incredible! It'd be great for the fossil app I'm working with at the moment as I'm about to do a first announcement / release.... and the instructions are going to be something like: uninstall the existing app, yes you will lose your settings. If you have Titanium, backup the app first. Then install mine from apk, you'll probably need to agree to security/premissions popup. Then restore data from titanium, be sure to only select data, not apk. profit.
A live patch system would be amazing, and yes I agree it could really entice more people to get involved.

On the build issue I get, it's reliable for me with this project, this commit: https://gitlab.com/alelec/fossil_smartwatches_alelec_android/tree/9c8fecfea815dbf33c68e13b5c78498bce3773b8
It should just checkout and build, I'm currently on Android Studio 3.5.1 (though I've had this issue for a long time now)
Basically build, it works. build again, it's still fine (it doesn't re-compile anything). If I make one change (even just change formatting on a file, no functional change) the build fails, with the first failure being

> Task :compileDebugJavaWithJavac
C:\Users\corona\src\fossil_app\src\main\java\com\portfolio\platform\MigrationManager.java:761: error: constructor Device in class Device cannot be applied to given types;
                Device device = new Device(
                                ^
  required: String,String,String,String,int,Integer,boolean
  found: String,String,String,String,int,int,boolean,int,<null>

In this case, jadx decompiles a number of functions like this with a couple more args than the builtin decompilers comes up with. So I've got the jadx copy sitting there completely DexIgnored and that satisfies the compiler (every second time) and function calls I make to the long version seem to work fine in practice. But every other time, the compiler only sees the jar version and complains the signature's wrong. Not the end of the world, I just compile again, but a bit odd nonetheless.

Don't rush for a release, chances are I'm going to continue working on this on-and-off for a few weeks at least :-P Cheers!

[Lanchon]

thank you!!! your bug is filed here DexPatcher/dexpatcher-gradle#28 for when time permits :)

[Lanchon]

your alpha release is ready :)
https://github.com/DexPatcher/dexpatcher-tool/releases/tag/v1.8.0-alpha3

besides the obvious, i'd appreciate comments on usability if you have any:

• how is the perceived performance hit of transforms in your workflow?
• do you experience issues related to greater memory usage during patching?

thanks!!

[andrewleech]

Sorry for my ignorance, but is there a way to set the --deanon-source and --reanon-source args in dexpatcher-gradle settings in build.gradle to use this automatically? I haven't used dexpatcher outside of Android Studio so not familiar with the workflow.

[Lanchon]

yes! extraArgs on the task for now. please read alpha2 release notes for details.

[Lanchon]

were you able to test?

[andrewleech]

Oh yes, I did try to use it but didn't have much luck, meant to follow up! I think I've got the config wrong...
I tried switching to the original apk with the following settings

dexpatcherConfig {
    dexpatcher {
        extraArgs = ["--verbose", "--deanon-source", "Anon[]", "--reanon-source", "Anon[]"]
    }
}

        case 'dexpatcher':
            add name, 'dexpatcher-repo.dexpatcher.dexpatcher-tool:dexpatcher:1.8.0-alpha3'

I've got a decompiled class I'm trying to target called

@DexEdit(contentOnly = true)
@sc4(c = "com.portfolio.platform.data.source.WatchFaceRepository", f = "WatchFaceRepository.kt", l = {28}, m = "getWatchFacesFromServer")
public final class WatchFaceRepository$getWatchFacesFromServer$Anon1 extends ContinuationImpl {
    ...
}

but I get the errror:

error: type 'com.portfolio.platform.data.source.WatchFaceRepository$getWatchFacesFromServer$Anon1': (WatchFaceRepository$getWatchFacesFromServer$Anon1.java:0): target not found

It's currently decompiled into a file of its own, I guess in this format it's less important to have the auto renaming in place as the name's already been bulked out.

I might try re-decompiling the top level class with anon's inlined and try it then.

Is there a way to get a copy of the source dex/jar after deanon-source has been run to decompile straight into the form that dexpatcher has modified?
I guess in this case it's

[Lanchon]

hey,

I tried switching to the original apk with the following settings

extraArgs = ["--verbose", "--deanon-source", "Anon[]", "--reanon-source", "Anon[]"]

better if you use extraArgs.addAll [...] instread of setting the collection.

• "--verbose": don't. the task already controls verbosity. use the provided controls or you'll get undefined behavior. use extraArgs only for options not explicitly supported by the task (in this case, transforms). btw, use DEBUG at the beginning to see all transformed items.

• "--deanon-source", "Anon[]": ok.

• "--reanon-source", "Anon[]": don't. --reanon-source is used, in general, to pre-process stuff (like patches) in a non-patching invocation of dexpatcher. and when you use it, you use it with a different plan than used for --deanon-source.

for your case the options should be:

• '--deanon-source', 'Anon[]', : this is what you need to feed the non pre-processed app.
• '--deanon-patch', '[]_patch', : this is in general a good thing to have, to safely use anon classes in the patch and have them not interfere with the source in any way.
• '--reanon-output', 'Anon[]', : if using BOTH of the above, you can reanonymize the source classes (and corresponding patch classes) so that you don't change their names in the final output. advantages include supporting reflection that might be present in the original app, keeping the stack traces compatible with those of the original app, and keeping the output patch-compatible (if somebody else writes another patch that you want to apply on top of yours).

Is there a way to get a copy of the source dex/jar after deanon-source has been run to decompile straight into the form that dexpatcher has modified?

yes. when the the dxp-tool 1.8 is finished, dxp-gradle will be updated with the possibility of 1) importing transformed symbols to the compiler, 2) decompile on-demand the transformed classes, and maybe 3) pre-transform the source to speed up the code-build-test cycle.

for now you have to do it manually:

dexpatcher --deanon-source Anon[] --output processed-source.dex source.dex

additionally you can define a Gradle task with type DexpatcherTask to do this kind of thing for you.

[andrewleech]

Thanks, that's really helped. I think I'm up and running well now on the original unmodified apk.

        extraArgs.addAll([
                "--deanon-source", "Anon[]",
                "--deanon-patches", "[]_patch",
                "--reanon-output", "Anon[]"
        ])

Thanks for the manual command advice, that has helped.
I grabbed the original apk's dex files, ran them through dexpatcher-1.8.0-alpha3.jar with the suggested command, then looked at them with the decompiler to inspect exactly how it was handling the long anon classes.

Some of the renaming gets pretty verbose with the repeating Anon's, Eg.
ActivitiesRepository$getActivityList$1$1$loadFromDb$1$1
becomes
ActivitiesRepository$getActivityList$Anon1$AnonAnon1$loadFromDb$AnonAnonAnon1$AnonAnonAnonAnon1
After looking a couple of examples in the decompiled code though it's straightforward to replicate, and as mentioned it's at least safe from name clashes!

When I first enabled --deanon-patches it gave me some errors like

error: reanonymize output: type 'com.fossil.blesdk.obfuscated.us2$a$1': class 'us2$a$1' cannot be 1-level reanonymized

But after a couple of cleans / rebuilds it seemed to come good, looks like it's working well.

FWIW when the gradle integration is done it would help if the error messages could have a same sort of "jump to file" links in the android studio build log that the existing warning messages get, eg.

warning: type 'com.misfit.frameworks.buttonservice.model.notification.DianaNotificationObj$AApplicationName': (DianaNotificationObj.java:0): 'enum' modifier removed from replaced type
warning: type 'com.portfolio.platform.data.AppType': (AppType.java:0): 'enum' modifier removed from replaced type
stats: patch process: 19732 types, 303 ms, 15 us/type
error: reanonymize output: type 'com.portfolio.platform.data.source.WatchFaceRepository$getWatchFacesFromServer$1': class 'WatchFaceRepository$getWatchFacesFromServer$1' cannot be 1-level reanonymized
error: reanonymize output: type 'com.fossil.blesdk.obfuscated.us2$a$1': class 'us2$a$1' cannot be 1-level reanonymized

In the (existing) warning messages (DianaNotificationObj.java:0) is a blue clickable link, but in the new error messages (all in red) they don't have this.

[Lanchon]

glad it worked!

renaming gets pretty verbose

i want to be able to:

• avoid clashes in Java after renaming
• avoid clashes with existing AnonNN classes in the source
• restore the same names with reanon under all possible conditions, including preexisting AnonNN source classes

it can get a little weird under some conditions, and so the chosen scheme of repeating anons. i thought of Anon_ (say Anon25_3 instead of AnonAnonAnon25) however, thought experiments seem to break that down under some conditions. it might be fixable, but with so many things of more importance pending on dxp, i think the current solution is enough.

In the (existing) warning messages (DianaNotificationObj.java:0) is a blue clickable link

yeah. unfortunately transforms work just in time when the code is processed to save memory and because of other considerations. so context information is not easily obtainable. line numbers are out of the question (too much work for low return) but at least including files might be possible under some circumstances. consider that simply producing diagnostics is complicated. the diagnostics messages happen at any time at all, and the transforms have diagnostics caches that make sure that messages are not repeated.

anyway. the most important thing: you notice a performance hit? thanks!

[andrewleech]

I haven't noticed a performance issue, if there is one it's not enough to cause me any pain. Certainly the gradle caching deals with most of it anyway.

That being said, if you were worried about it the re-write could be done once-off on an apklib similar to what I've previously done to the smali with regex's.
Unless the app's using reflection against one of the anon classes (surely unlikely) this shouldn't cause issues?

[Lanchon]

pre-transforming would only help one-way. reanon and other restoring transforms have to run after the patch anyway, so can't be cached.

i want to introduce transforms that allow arbitrary mappings to work with obfuscated code. mappings would be configurable via a file, and analysis workflow would be iterative: decompile (on the fly with IDEA), decide on a good name for an item, add it to mappings, repeat. so map updates would be frequent. writing dex is 10 times slower than online transform, so pre-transforming is only worth it if you'll apply patches 10 times using the same mappings. each time to update maps, you'd be generating new dex files which is a huge performance hit.

so it makes sense to do it if you are importing symbols or decompiling based on transformed stuff (which is extra slow anyway because of the dex2jar step), but not because of performance, i think.

in dxp-gradle, the visibility of transforms (ie in symbols and decompilation) -and thus pre-transforming- is going to be configurable, but in general you would enable it for reasons other than performance (eg: feeding dex2jar with transformed stuff).

i will try to avoid feature creep around pre-transforming support for performance only, especially because performance is very dependent on frequency of mapping changes, but also because you'll probably mostly want to always have visibility of transforms (which require pre-transforming).

[Lanchon]

FYI,

you now have this build:
https://github.com/DexPatcher/dex2jar/releases/tag/v2.1-20190905-lanchon

available in the repo:
https://dexpatcher.github.io/dexpatcher-repo/m2/dexpatcher-repo/dexpatcher/dex2jar/dex-tools/maven-metadata.xml

so you can use that and retire your local copy of dex2jar

[Lanchon]

a heads up for you, some changes are coming up and impact you:

• the next release changes anon class handling a bit:

  dexpatcher-tool/test/patch/src/main/java/test/Main.java

  Lines 743 to 820 in f6ad920

  	// DexPatcher tool v1.8.0 adds support for processing anonymous classes via
  	// optional code transform passes that deanonymize and reanonymize them.
  	// They allow easier patching of existing anonymous classes in the source.
  	// And they also allow defining new anonymous classes in the patch without
  	// risk of them name-clashing with existing ones in the source.
  	// The deanonymize and reanonymize transforms apply a configurable plan,
  	// which is a string that represents an anonymous class renaming template
  	// and has the form '<prefix>[<infix>]<suffix>'. The infix cannot be empty,
  	// and the prefix and suffix cannot be both empty. The brackets are literal
  	// characters in the string, and the characters immediately adjacent to the
  	// brackets cannot be digits.
  	// During deanonymization, an anonymous class named number 'N' in bytecode
  	// and nested 'L' levels deep will be renamed to '<prefix>N<suffix>' if L
  	// is 1 or to '<prefix>N<infix>L<suffix>' if L is larger than 1. The
  	// nesting level is included in the new name to support patching of nested
  	// anonymous classes and to avoid clashes with existing class names that
  	// happen to match the plan, should they exist.
  	// For example, to allow defining new anonymous classes in a patch, the
  	// patch could be deanonymized with plan '[_]_patch' to rename its
  	// anonymous classes and make sure they do not clash with the names of
  	// anonymous classes already existing in the source. In this case, a patch
  	// anonymous class named 'Main$1' would end up being called 'Main$1_patch'.
  	// Also, to make it easier to patch existing anonymous classes, the source
  	// could be deanonymized with plan 'Anon[_Level]'. This would cause a
  	// source anonymous class named 'Main$3' to be renamed to 'Main$Anon3' for
  	// easy targeting with DexPatcher tags. Additionally, the original
  	// anonymous class names can be restored after patching by reanonymizing
  	// the output using the same plan used to deanonymize the source.
  	// Finally, when it is time to publish the patch, the patch itself could be
  	// pre-transformed (so that it can be applied to the source by users
  	// without needing any further transforms) using this command:
  	// dexpatcher patch.dex --output release-patch.dex
  	// --deanon-source [_]_patch --reanon-source Anon[_Level]
  	// Modify members of an anonymous class:
  	// Note: For this to work, the source dex file must be deanonymized using
  	// '--deanon-source Anon[_Level]'. It is recommended to reanonymize the
  	// output dex using '--reanon-output Anon[_Level]'. This restores the
  	// original names of the anonymous classes right after the patching step.
  	@DexEdit
  	public static class AnonymousClasses {
  	@DexEdit(contentOnly = true)
  	public static class Anon1 {
  	@DexEdit(contentOnly = true)
  	public static class Anon1_Level2 implements Runnable {
  	@DexReplace
  	@Override public void run() {
  	p("replaced AnonymousClasses::<anon>::<anon>::run");
  	}
  	}
  	}
  	// Use anonymous classes in the patch:
  	// Note: Anonymous classes in the patch can name-clash with anonymous
  	// classes in the source. To avoid clashes, the patch dex file can be
  	// deanonymized using something like '--deanon-patch [_]_patch'.
  	@DexAppend
  	public static void print() {
  	new Runnable() {
  	@Override
  	public void run() {
  	p("added AnonymousClasses::<anon> (" + this.getClass() + ")");
  	}
  	}.run();
  	}
  	// Test unexpected anonymous classes during reanonymization:
  	// Note: The output must be reanonymized using '--no-anon-errors'.
  	// Note: When reanonymizing the output, all classes are expected to be
  	// deanonymized. This inserts an non-deanonymized anonymous class named
  	// '42' in the output to test the handling of this error condition.
  	static class __UnexpectedAnonymousClass_$$_42__ {}
  	}

• also has small changes in the command line that won't affect you.

• unfortunately alpha 2 and 3 builds are affected by a race condition. this would never affect the resulting dex files, but it can affect the output log. (logging the transforms is more complex than one would expect, even without source file and line numbers.) because of this, these builds will be retired.

really sorry for the changes. being alphas, i don't really struggle for backwards compatibility.

sorry and thanks!

[Lanchon]

here is a backup of alpha3 for you to use locally, because it will be retired. sorry!

https://github.com/DexPatcher/dexpatcher-repo/raw/79412c7e94635ead0ad84038644bd81e2aed4fc2/m2/dexpatcher-repo/dexpatcher/dexpatcher-tool/dexpatcher/1.8.0-alpha3/dexpatcher-1.8.0-alpha3.jar

[andrewleech]

Thanks but I'm more than happy to figure out what's needed to move the the next beta, I expect alpha level stuff to change and break!

Thanks for the heads up!

And in future don't event spend a second trying to maintain alpha compatibility, build whatever makes the most sense to you.

[Lanchon]

thank you! :)

[Lanchon]

there will be a default <plan> for source and output.

since you use this, what should be the default plan?

1. Anon[_Level] eg: Anon1_Level2
2. Anon[_] eg: Anon1_2
3. something else?

isn't option 1 too verbose?

[andrewleech]

Yeah 1 seems unnecessarily long, 2 makes sense to me.
I'm currently using

                "--deanon-source", "Anon[_]",
                "--deanon-patches", "[_]_patch",
                "--reanon-output", "Anon[_]"

[Lanchon]

i totally agree. thanks!

[Lanchon]

it's done. for next release...

[Lanchon]

well i implemented it but i don't like it. because i can't do this in one pass:

	// Finally, when it is time to publish the patch, the patch itself could be
	// pre-transformed (so that it can be applied to the source by users
	// without needing any further transforms) using this command:
	// dexpatcher patch.dex --output release-patch.dex
	//     --deanon-source [_]_patch --reanon-source Anon[_Level]

so i don't know what i'm gonna do...

[Lanchon]

so in the end i implemented this... thanks!!

[andrewleech]

Nice, yeah I doubt there's many times the full flexibility of setting a custom plan at each step is really needed, I'm personally happy to take a working suggestion/example and go with it!

PS. On an unrelated note I've recently had to rebuild my patches on a new official release of an apk, where I rewrote them using more of dexwrap to reduce the amount of proguard obfuscated code I needed to touch. Geez it works well, so quick to use and lower risk of new decompilation bugs! Thanks again!

[Lanchon]

plans have to be configurable for the same reason dxp annotation package and code markers are. these options might never ever be used, but they still serve their purpose: they are attack deterrents. app authors (or obfuscation tool authors) could easily attack dexpatcher if they weren't. but knowing that these options can be reconfigured and thus attacks can be easily averted, no attacks will probably exist.

a similar thing: --no-reanon-errors was changed to --no-anon-errors when i changed the anonymizer to numeric levels (ie: Anon[] to Anon[_]). this was because a specially crafted dex can be made to make the anonymization level increase whatever type of variable was used to express it be it int or long. so i detected the case and switched the option to --no-anon-errors to account for the fact that deanonymizations now could fail (but only when attacked). i accepted this because the countermeasure to an attack was simply to change the plan.

but... thinking of it later: if a patch is published, then the app author could attack the anonymizer in the next app version, forcing the patch author to change the plan. but that requires refactoring the patch codebase to adjust it to the new plan, which is non-trivial work. so yesterday i implemented infinite precision math for the anon level numbers, so no overflow can ever happen. deanonymizations are back to their original 'never failing' status, and the option is back to --no-reanon-errors (and reanons only fail due to patch author errors, never because of attacks).

PS. On an unrelated note

i never knew your source app was obfuscated! i already implemented another method to deal with obfuscation that is complementary to identifier codes but i haven't published it. i'll push it now and be back...

[Lanchon]

that was an easy rewind to a pushable state. i pushed the mapping support and the new (and hopefully final) implementation of the anonymizer (which should have no impact on your end at all).

this way of dealing with obfuscation is based on a mapping file (similar to proguard's) such as:
https://github.com/DexPatcher/dexpatcher-tool/blob/master/test/mapping.txt

where you give meaningful names to all stuff that you need to refer to in your patch. when patching, you --map-source and --unmap-output, so patching is done in your deobfuscated namespace (including all diagnostic messages) but the output is returned to its original obfuscation (to avoid issues with reflection).

when a new app version is published, you only need to update the mapping file and not your code.

also the map will serve to rev engineer the apk, as IDEA's JIT decompilation will be updated with the meaningful names as you add mappings to it. but this requires a new dxp-gradle, so a preview of the current code would not do that (i only have dxp-tool functionality for now).

i'm working now on a 3rd synergistic method for handling obfuscation and when finished. tool's 1.8.0 feature set should be about frozen. then i can publish a beta and switch to dxp-gradle.

[Lanchon]

1.8.0 feature set should be around complete. it's pushed but nothing is tested. i suppose its time to close this. thanks!