watchOS support

[Dids]

Could this potentially support watchOS out of the box?

The steps for symbolicating watchOS crash logs are here. Haven't found any other articles or documentation on how to do it, except for that single post (from 2015).

Having to symbolicate one line at a time, as well as manually fetching UUIDs, checking architectures and combing through log files is a pain, so I thought about creating a small console app that does this for me and happened to stumble upon your symbolicator. :)

I'm happy to help test/develop this if you're interested.

[tomaz]

Hmm, don't know, never really needed watchOS symbolication, but it seems to use the same underlying formats and tools. So it should be supported as long as symbol files are available for watch app - though perhaps they are burried inside iPhone symbols and you need to use cmd line argument to point symbolicator to...

[Dids]

I can confirm that the steps in the link in my original post work for watchOS.

Just tried symbolicator by just specifying a path to the crash log (it's sitting next to the watchOS extension app/bundle), but unfortunately it doesn't work out of the box:

symbolicator watchOS.Extension/bin/iPhone/AdHoc/MyApp.watchOS.Extension.crash 
symbolicator: version 1.0 (build 99)

Symbolizing 1 crash logs...

Symbolizing watchOS.Extension/bin/iPhone/AdHoc/MyApp.watchOS.Extension.crash...
Detected com.company.myapp.watchkitapp.watchkitextension ARM [MyApp.watchOS.Extension 3830 (1.0)]
Found 53 lines that need symbolication
WARNING: Didn't find starting address for MyApp.watchOS.Extension
...

[Dids]

Here's also an line from the log:

3   MyApp.watchOS.Extension  	0x010c5f6e 0x24000 + 17440622

[tomaz]

I think it's just not finding symbol files - are they present in Xcode Archives folder? Otherwise you can try with --archives option

[Dids]

Ah, that might be it as this is a Xamarin project.

What does the symbolicator expect to find in the archives folder?
If it's just a specific directory structure etc. I could try to emulate it.

[tomaz]

Xcode uses ~/Library/Developer/Xcode/Archives, the contents are folders with date time-stamps as their names, then inside those there's binary + dSYMs subfolder and in it you find dSYM bundles inside which DWARF files are located (they have the same name as the bundle and are found under Content/Resources/DWARF subpath). So all together:

~/Library/Developer/Xcode/Archives/<DATE>/<BINARY>/dSYMs/<BINARY>.app.dSYM/Contents/Resources/DWARF/<BINARY>

(this is from memory so there might be some small difference).

Symbolicator expects --archives path to be pointing to the base archives path (so ~/Library/Developer/Xcode/Archives by default). If Xamarin places DWARF files to some other location but retains the same subfolder structure then just point symbolicator to it, but if it uses completely different folder structure, then dwarfPathWithIdentifier(_:version:build:) function from ArchiveHandler class needs adjusting so it can work with arbitrary file.

[Dids]

Thanks for the clarification!

In Xamarin's case, an example folder structure would be MyApp/iOS/bin/iPhone/Release/MyApp.app.dSYM, so I guess adding arguments for specifying the relative or absolute paths for both the binary (.app) and dSYM (.app.dSYM) would work in Xamarin's case.

[tomaz]

In this case try --archives ~/.../MyApp/iOS/bin - symbolicator blindly jumps over 2 folders (at least it wouldn't hurt :)

[Dids]

No dice yet, unfortunately. Even if the path to the archives is non-existent, it doesn't actually complain about not finding the dwarf files (contrary to what I see in the code), so I'm not sure what's going on. :/

[Dids]

Tried printing inside dwarfPathWithIdentifier, but nothing is printed, so I'm guessing the function doesn't even get called.

[Dids]

Okay, narrowed it down to this:

FileSymbolicator.swift (symbolicateString method)

let binary = (match.groups[1] as! RxMatchGroup).value!.trimmingCharacters(in: whitespace)
guard let baseAddress = baseAddressForSymbolication(contents, identifier: binary) else {
	// This is being hit for every line/match
	continue
}

[Dids]

If it helps, here's an actual crash report.
It does find the lines, as it reports that it's symbolicating 53 lines, but it can't seem to parse the addresses, not sure why.

[tomaz]

Crash report seems normal. Seems like regexes in baseAddressForSymbolication function don't resolve the base address for the symbol (hex value in third column) for some reason... But can't determine more without debugging in Xcode

[tomaz]

To be more precise: "base address" should return 0x24000 from this line in crashlog:

0x24000 - 0x1183fff Tyomaarain.watchOS.Extension armv7k  <d5c85926c8ff3f698540e5dd14fb69e3> /var/containers/Bundle/Application/

[tomaz]

baseAddressForSymbolication(_:identifier:) tries to search first for:

0xADDR1 - 0xADDR2 [IDENTIFIER]...

(where IDENTIFIER is Tyomaarain.watchOS.Extension in your case) and if this doesn't produce result, it falls down to find all lines looking like:

0xADD1 - 0xADDR2 *

And then iterates over all and uses 0xADDR1 value from the first one where the string contains IDENTIFIER

[Dids]

Hmm, I tried printing the match.value in symbolicateString and it might be a tab issue with regex?

"2   Tyomaarain.watchOS.Extension  \t0x01065aaa 0x24000 + 17046186"

[Dids]

Never mind, that's not it either. Tried replacing tabs with spaces for the entire log contents, but same result.

[Dids]

Oh, I guess it can't find the identifier because the format is Tyomaarain.watchOS.Extension 0x01065aaa 0x24000 + 17046186, and not 0x01065aaa 0x24000 Tyomaarain.watchOS.Extension, right?

[Dids]

Never mind, got confused for a bit there and didn't remember that it's looking for the identifier/UUID under binary images, where the addresses don't really matter.

[Dids]

Got it. I marked the whitespace checks in the regex optional (see here) and it's now properly detecting the identifier. :)

[tomaz]

Oh, cool! Thought it was something with regex/file format, am not behind computer till today evening or tomorrow so couldn't test. Would you mind creating pull request with fix? I want to test with existing crashes to make sure it works everywhere and merge so others can benefit from the solution...

[Dids]

To be honest I'd rather wait until I have it working with these crash logs, then push a PR, if that's alright?

Trying to get it to find the dwarf files now, since it's not finding them.

EDIT: It's not actually enumerating any of the subdirectories under the base path, not sure why.

[tomaz]

Ok, that's perfect! It's first time Xamarin issue, so adding support for it out of the box would be real nice:

1. If possible automatically enumerate Xamarin "archives" path (that's if Xamarin saves them at single path similar to Xcode). Though judging from path above it's saved on per-project basis. In this case specifying project root and then automatically search for dwarf in there would also work (this would require extending ArchiveHandler so it checks paths for both tools).

2. Changd regex pattern

I can help along the way, though don't have Xamarin so can't test. But I can prepare the basics then you can tackle the details from there... Anyway, ping me when/if you decide to proceed - no pressure :)