internal / underlying blake2 symbols aren't mangled or private #109

cartazio · 2016-11-03T14:17:38Z

I'm helping some folks with some crypto heavy code and this change would make our lives much easier. If I helped get this sorted, would it be possible to turn around a release with that hange pretty fast ?

cartazio · 2016-11-03T17:22:36Z

as a bit of context haskell-hvr/argon2#10 and


duplicate symbol _blake2b_init_key in:
    /Users/carter/.cabal/store/ghc-8.0.1/cryptonite-0.20-0af5802ff1a1deb4cb611283f80d9b336faa42a9e6e78f7dbab10d01ae0d2eeb/lib/libHScryptonite-0.20-0af5802ff1a1deb4cb611283f80d9b336faa42a9e6e78f7dbab10d01ae0d2eeb.a(blake2b.o)
    /Users/carter/.cabal/store/ghc-8.0.1/argon2-1.2.0-b9f6a4cdf7fc9c099739f9e37ad85d14069af01351740db465356c55afa097c1/lib/libHSargon2-1.2.0-b9f6a4cdf7fc9c099739f9e37ad85d14069af01351740db465356c55afa097c1.a(blake2b.o)
duplicate symbol _blake2b_init in:
    /Users/carter/.cabal/store/ghc-8.0.1/cryptonite-0.20-0af5802ff1a1deb4cb611283f80d9b336faa42a9e6e78f7dbab10d01ae0d2eeb/lib/libHScryptonite-0.20-0af5802ff1a1deb4cb611283f80d9b336faa42a9e6e78f7dbab10d01ae0d2eeb.a(blake2b.o)
    /Users/carter/.cabal/store/ghc-8.0.1/argon2-1.2.0-b9f6a4cdf7fc9c099739f9e37ad85d14069af01351740db465356c55afa097c1/lib/libHSargon2-1.2.0-b9f6a4cdf7fc9c099739f9e37ad85d14069af01351740db465356c55afa097c1.a(blake2b.o)
duplicate symbol _blake2b_init_param in:
    /Users/carter/.cabal/store/ghc-8.0.1/cryptonite-0.20-0af5802ff1a1deb4cb611283f80d9b336faa42a9e6e78f7dbab10d01ae0d2eeb/lib/libHScryptonite-0.20-0af5802ff1a1deb4cb611283f80d9b336faa42a9e6e78f7dbab10d01ae0d2eeb.a(blake2b.o)
    /Users/carter/.cabal/store/ghc-8.0.1/argon2-1.2.0-b9f6a4cdf7fc9c099739f9e37ad85d14069af01351740db465356c55afa097c1/lib/libHSargon2-1.2.0-b9f6a4cdf7fc9c099739f9e37ad85d14069af01351740db465356c55afa097c1.a(blake2b.o)
duplicate symbol _blake2b_final in:
    /Users/carter/.cabal/store/ghc-8.0.1/cryptonite-0.20-0af5802ff1a1deb4cb611283f80d9b336faa42a9e6e78f7dbab10d01ae0d2eeb/lib/libHScryptonite-0.20-0af5802ff1a1deb4cb611283f80d9b336faa42a9e6e78f7dbab10d01ae0d2eeb.a(blake2b.o)
    /Users/carter/.cabal/store/ghc-8.0.1/argon2-1.2.0-b9f6a4cdf7fc9c099739f9e37ad85d14069af01351740db465356c55afa097c1/lib/libHSargon2-1.2.0-b9f6a4cdf7fc9c099739f9e37ad85d14069af01351740db465356c55afa097c1.a(blake2b.o)
duplicate symbol _blake2b_update in:
    /Users/carter/.cabal/store/ghc-8.0.1/cryptonite-0.20-0af5802ff1a1deb4cb611283f80d9b336faa42a9e6e78f7dbab10d01ae0d2eeb/lib/libHScryptonite-0.20-0af5802ff1a1deb4cb611283f80d9b336faa42a9e6e78f7dbab10d01ae0d2eeb.a(blake2b.o)
    /Users/carter/.cabal/store/ghc-8.0.1/argon2-1.2.0-b9f6a4cdf7fc9c099739f9e37ad85d14069af01351740db465356c55afa097c1/lib/libHSargon2-1.2.0-b9f6a4cdf7fc9c099739f9e37ad85d14069af01351740db465356c55afa097c1.a(blake2b.o)
duplicate symbol _blake2b in:
    /Users/carter/.cabal/store/ghc-8.0.1/cryptonite-0.20-0af5802ff1a1deb4cb611283f80d9b336faa42a9e6e78f7dbab10d01ae0d2eeb/lib/libHScryptonite-0.20-0af5802ff1a1deb4cb611283f80d9b336faa42a9e6e78f7dbab10d01ae0d2eeb.a(blake2b.o)
    /Users/carter/.cabal/store/ghc-8.0.1/argon2-1.2.0-b9f6a4cdf7fc9c099739f9e37ad85d14069af01351740db465356c55afa097c1/lib/libHSargon2-1.2.0-b9f6a4cdf7fc9c099739f9e37ad85d14069af01351740db465356c55afa097c1.a(blake2b.o)
ld: 6 duplicate symbols for architecture x86_64
clang-3.8: error: linker command failed with exit code 1 (use -v to see invocation)
`clang' failed in phase `Linker'. (Exit code: 1)

cartazio · 2016-11-03T17:23:02Z

cc @vincenthz @glguy @yogsototh

cartazio · 2016-11-03T17:33:48Z

cryptonite/cryptonite.cabal

Lines 266 to 277 in 3c087f0

    
           if arch(x86_64) || flag(support_blake2_sse) 
        
             C-sources:      cbits/blake2/sse/blake2s.c 
        
                           , cbits/blake2/sse/blake2sp.c 
        
                           , cbits/blake2/sse/blake2b.c 
        
                           , cbits/blake2/sse/blake2bp.c 
        
             include-dirs: cbits/blake2/sse 
        
           else 
        
             C-sources:      cbits/blake2/ref/blake2s-ref.c 
        
                           , cbits/blake2/ref/blake2sp-ref.c 
        
                           , cbits/blake2/ref/blake2b-ref.c 
        
                           , cbits/blake2/ref/blake2bp-ref.c 
        
             include-dirs: cbits/blake2/ref

seems to be the offending bit,

and those implementations dont have the name mangling and they're not static function impls

centromere · 2016-11-03T18:06:26Z

@cartazio What is the output of:

readelf -s dist/build/libHScryptonite-0.20-....so | grep blake2b_init_key

cartazio · 2016-11-03T18:35:25Z

i'm on mac osx....

On Thu, Nov 3, 2016 at 2:06 PM, John Galt [email protected] wrote:

cartazio · 2016-11-03T18:35:38Z

what would the appropriate command analogue there?

On Thu, Nov 3, 2016 at 2:35 PM, Carter Schonwald <[email protected]

wrote:

i'm on mac osx....

On Thu, Nov 3, 2016 at 2:06 PM, John Galt [email protected]
wrote:

centromere · 2016-11-03T18:46:11Z

I don't use OSX so I don't know. This could be helpful though: http://stackoverflow.com/questions/3286675/readelf-like-tool-for-mac-os-x

vincenthz · 2016-11-03T21:57:48Z

might be much faster to port the tiny argon library (specially considering that blake is already in) to cryptonite than muck around the blake C files; Obviously would be nice to solve the exports list problem too, but need much more careful thoughts.

cartazio · 2016-11-03T22:16:16Z

Port?

I do know that if Argon2 is set to dyload the libargon that the collision
doesn't happen.

But yeah, I'll get yah yer info

On Thursday, November 3, 2016, Vincent Hanquez [email protected]
wrote:

might be much faster to port the tiny argon library (specially considering
that blake is already in) to cryptonite than muck around the blake C files;
Obviously would be nice to solve the exports list problem too, but need
much more careful thoughts.

—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
#109 (comment),
or mute the thread
https://github.com/notifications/unsubscribe-auth/AAAQwmETMaiLsAMGLN5Z4CA-VolOvbfPks5q6ljdgaJpZM4KoaO-
.

cartazio · 2016-11-03T22:38:12Z

That said, I'm not sure if OS X supports the hidden symbol attribute on
static libs, so it might be worth adding name mangling to hidden but not
static symbols ...

On Thursday, November 3, 2016, Carter Schonwald [email protected]
wrote:

Port?

I do know that if Argon2 is set to dyload the libargon that the collision
doesn't happen.

But yeah, I'll get yah yer info

On Thursday, November 3, 2016, Vincent Hanquez <[email protected]
javascript:_e(%7B%7D,'cvml','[email protected]');> wrote:

might be much faster to port the tiny argon library (specially
considering that blake is already in) to cryptonite than muck around the
blake C files; Obviously would be nice to solve the exports list problem
too, but need much more careful thoughts.

—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
#109 (comment),
or mute the thread
https://github.com/notifications/unsubscribe-auth/AAAQwmETMaiLsAMGLN5Z4CA-VolOvbfPks5q6ljdgaJpZM4KoaO-
.

vincenthz · 2016-11-04T07:12:48Z

port = adding argon2 in cryptonite.

centromere · 2016-12-20T19:03:56Z

@cartazio Is this still an issue for you?

cartazio · 2016-12-20T19:27:36Z

it is an issue, but it looks like cryptonite isn't willing to consider mangling private symbols to make sure they're not conflicting on OSX, so it sounds like any projects that have this collision just have to maintain a vendored version of one lib or the other :(

…

On Tue, Dec 20, 2016 at 2:03 PM, John Galt ***@***.***> wrote: @cartazio <https://github.com/cartazio> Is this still an issue for you? — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#109 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AAAQwqSQ4Anj2iEYNO0gMV-l2EaCLJydks5rKCadgaJpZM4KoaO-> .

centromere · 2016-12-20T19:54:09Z

I think the proper solution is to ensure that the symbols are hidden properly. Is there really no way to do this with static libraries on OSX?

cartazio · 2016-12-20T22:53:08Z

yes, prefix them with `cryptonite_` or something similar like `cryptoniteHS_ `is probably the only approach that will be properly robust, and is generally good practice :) as best as I understand it, the core mach / mach-o stuff on macOS is really dynamic linker first. if you have specific mac code experiments you'd like me to do, i'm happy to copy and paste them, but based on my own investigations, name mangling is the ONLY robust solution in the presence of static linking. everything else has sharp bits and gotchas. (and thats also ignoring the fact that you cant quite do relocatable dylib lookups sortah on macOS, so static linking is favored for simple binaries that aren't in a .app or the like)

…

On Tue, Dec 20, 2016 at 2:54 PM, John Galt ***@***.***> wrote: I think the proper solution is to ensure that the symbols are hidden properly. Is there really no way to do this with static libraries on OSX? — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#109 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AAAQwqIu9biu8W7YV29K2ejTYSH8bZleks5rKDJigaJpZM4KoaO-> .

centromere · 2016-12-21T04:40:48Z

For the blake2 source files in cryptonite, everywhere you see __attribute__ ((visibility ("hidden"))), could you replace it with __attribute__((weak)) and attempt to recompile?

cartazio · 2016-12-21T04:58:38Z

apple docs indicate that wont work https://developer.apple.com/library/content/documentation/DeveloperTools/Conceptual/MachOTopics/1-Articles/executing_files.html#//apple_ref/doc/uid/TP40001829-98432-TPXREF120

vincenthz · 2017-02-24T13:58:47Z

argon2 is now in cryptonite

cartazio · 2017-02-24T18:31:56Z

Sweet. Thanks.

…

On Fri, Feb 24, 2017 at 8:58 AM Vincent Hanquez ***@***.***> wrote: argon2 is now in cryptonite — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#109 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AAAQwiVZ9JenPxwYOhwwDZpOYOx2qMopks5rfuIXgaJpZM4KoaO-> .

cartazio changed the title ~~Add cryptonite_ prefix to all nonstatic / exported c symbols~~ internal / underlying blake2 symbols aren't mangled or private Nov 3, 2016

centromere self-assigned this Nov 3, 2016

conor10 mentioned this issue Dec 21, 2016

Duplicate symbol _blake2b_init_key causes build failure on OS X Sierra Consensys/constellation#7

Closed

QuLogic mentioned this issue Apr 5, 2018

Support building against system argon2. #227

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

internal / underlying blake2 symbols aren't mangled or private #109

internal / underlying blake2 symbols aren't mangled or private #109

cartazio commented Nov 3, 2016

cartazio commented Nov 3, 2016

cartazio commented Nov 3, 2016

cartazio commented Nov 3, 2016

centromere commented Nov 3, 2016

cartazio commented Nov 3, 2016

cartazio commented Nov 3, 2016

centromere commented Nov 3, 2016

vincenthz commented Nov 3, 2016

cartazio commented Nov 3, 2016

cartazio commented Nov 3, 2016

vincenthz commented Nov 4, 2016

centromere commented Dec 20, 2016

cartazio commented Dec 20, 2016 via email

centromere commented Dec 20, 2016

cartazio commented Dec 20, 2016 via email

centromere commented Dec 21, 2016

cartazio commented Dec 21, 2016 via email •

edited by vincenthz

Loading

vincenthz commented Feb 24, 2017

cartazio commented Feb 24, 2017 via email

internal / underlying blake2 symbols aren't mangled or private #109

internal / underlying blake2 symbols aren't mangled or private #109

Comments

cartazio commented Nov 3, 2016

cartazio commented Nov 3, 2016

cartazio commented Nov 3, 2016

cartazio commented Nov 3, 2016

centromere commented Nov 3, 2016

cartazio commented Nov 3, 2016

cartazio commented Nov 3, 2016

centromere commented Nov 3, 2016

vincenthz commented Nov 3, 2016

cartazio commented Nov 3, 2016

cartazio commented Nov 3, 2016

vincenthz commented Nov 4, 2016

centromere commented Dec 20, 2016

cartazio commented Dec 20, 2016 via email

centromere commented Dec 20, 2016

cartazio commented Dec 20, 2016 via email

centromere commented Dec 21, 2016

cartazio commented Dec 21, 2016 via email • edited by vincenthz Loading

vincenthz commented Feb 24, 2017

cartazio commented Feb 24, 2017 via email

cartazio commented Dec 21, 2016 via email •

edited by vincenthz

Loading