[RFC] gcc/g++ default flags for Habitat builds

[ghost]

I am wondering if any thought has been given to maybe setting default compile flags for gcc/g++. I know one of the use cases for Habitat is increased security and it appears that other package maintainers have these as part of their standard like https://fedoraproject.org/wiki/Changes/Harden_All_Packages.

Just food for thought!

[ghost]

Thanks @reset. I would have brought this up in the Slack channel but felt it may be worth preserving the conversation in GitHub as it may wind up in broad sweeping change if implemented ...

[stevendanna]

I'm +1 on improving the default flags used throughout core-plans. core-plans should provide artifacts that balance speed, security, and debug-ability.

Some things we may want to consider:

• gcc -fstack-protector-strong
• gcc -O2 (or some other default optimization level)
• gcc -g (although we strip symbols later anyway, so maybe not)
• gcc -­fPIE
• gcc ­D_FORTIFY_SOURCE=2
• gcc ­-Wformat -­Wformat­security
• ld -­z relro
• ld -z now
• ld ­-pic

[eeyun]

I'm definitely +1 on this in concept. But I have some feedback and a request. I think a change like this that (if implemented) touches a significant portion of the packages in core needs to undergo the newly defined RFC process. That process right now is outlined in an actual RFC that's undergoing it's own RFC process (meta, right?). That being said I'd like to get some core plans maintainership feedback on this but I would prefer that this get closed and resubmitted following that process.

I know this has been open for quite some time and I'm sorry it's taken so long for us to get any kind of reasonable feedback on it. @pantocrator27 or @stevendanna if you still feel passionately about this change would either of you be willing to spearhead the RFC effort around this?

[stevendanna]

@eeyun I submitted a PR here to add some tooling into core-plans to make it possible to incrementally add these flags to builds: #881

I'm happy to do an RFC; however, there is an extent to which I think an incremental approach and gathering information might be preferable to setting policy since we have a lot to learn still.

[eeyun]

@stevendanna I could totally see that. I'm all for an incremental approach the only challenge is any changes against the default flags with either of these packages are going to touch like 90% of the world. We can adopt incrementally but we might need to talk about what that looks like in implementation/test? All of this to say, I trust you and if you've got ideas about how to do it you've 100% got my support.

[bdangit]

👍 for security. Would like to see how many of our plans would be affected by these extra flags and this could be accomplished if we ran your security script against any plans that require gcc or maybe all of the binaries.

[lilianmoraru]

I have been doing this for almost a year on my habitat origins(I really didn't like that all the packages are compiled without any optimizations).
While I customize per package, here are some general flags that will work for every package:

$ export RUSTFLAGS="-C target-cpu=x86-64"

# "-Wno-error -Wno-error=implicit-fallthrough" - newer GCCs throw more warnings and some packages treat warnings as errors
$ export CFLAGS="-O3 -DNDEBUG -fomit-frame-pointer -ftree-vectorize -march=x86-64 -pipe -Wno-error -Wno-error=implicit-fallthrough"

# lzop is the only package limited to gnu++11, without it "-std=gnu++14" will also work
$ export CXXFLAGS="-std=gnu++11 -O3 -DNDEBUG -fomit-frame-pointer -ftree-vectorize -march=x86-64 -pipe -Wno-error -Wno-error=implicit-fallthrough"
$ export CPPFLAGS="-pipe -Wno-error -Wno-error=implicit-fallthrough"

A few more points(these could probably easily be used and then in do_prepare to remove the flags from the environment variables, per package):

• export LDFLAGS="-Wl,-Bsymbolic-functions -Wl,-z,relro" works on everything except busybox
• export CXXFLAGS="-std=gnu++14" works on everything except lzop
• export CFLAGS/CXXFLAGS="-fstack-protector-strong" works on everything except: gcc, glibc, grub, valgrind

Addressing some of the flags @stevendanna mentioned:

• D_FORTIFY_SOURCE=2 will fail to compile on a lot of packages
• ­-Wformat -­Wformat­security because a good amount of packages treat warnings as errors, this will probably fail a lot of them, also, throwing a warning does not improve the final binary.
• -z now full relro is hard to introduce, most packages fail with it.
• -­fPIE/-pic is hard to use correctly "globally" because these need to apply only to executables(PIE) or only to libraries(PIC) - a project can compile multiple libraries and then link them into an executable(you cannot decide whether to use PIE or PIC).

[stevendanna]

@lilianmoraru Thanks for all of the awesome info. I made a quick stab at adding some of these flags in #881 but didn't apply it nearly as generally as have.

-z now full relro is hard to introduce, most packages fail with it.

What kind of failures have you seen here? I haven't hit any but I've only rebuild a handful of simple package.

-­fPIE/-pic is hard to use correctly "globally" because these need to apply only to executables(PIE) or only to libraries(PIC) - a project can compile multiple libraries and then link them into an executable(you cannot decide whether to use PIE or PIC).

Yup, I ended up punting on this as well. I think to apply this on a large scale we would have to go the route of some of the distributions and have pretty complicated wrappers to decide on the right gcc flags on a per-invocation basis.

[lilianmoraru]

It seems I got some of the flags wrong there(I've revisited them and some work with relro for example).
You can see what I use here: https://github.com/be-plans/be
The flags are in defaults.sh and to search for disabled flags you need to search for be_remove_compiler_flag and be_remove_linker_flag.

For Habitat core plans I would recommend to avoid -O3 and -D_GNU_SOURCE(although ninja really loves that flag).
Projects that do not handle undefined behavior well in C/C++ projects, can hit bugs with -O3(I've not hit any issues with it yet).

I will actually apply later the flags cleaner, without an "out of source" script, so defaults.sh will disappear.