Appimage support #579

Murmele · 2023-06-13T08:31:33Z

Currently I am trying to create an appimage for Gittyup and integrating it into the CI. Currently we have only a Flatpak, there is an unofficial version for arch in aur. Do you recommend using appimage-builder or using linuxdeploy?

probonopd · 2023-06-14T18:59:42Z

Thanks @Murmele for adding an AppImage. As for which tools to use, I'd probably use https://github.com/probonopd/go-appimage/tree/master/src/appimagetool myself, but mainly because I wrote this one and am not as familiar with all the other options.

There are many great options out there:
https://github.com/AppImageCommunity/awesome-appimage#deployment-tools-for-compiled-applications

Which one works best for your application and workflow I don't know, maybe you'd like to try out a few and then decide?

Murmele · 2023-06-19T14:37:13Z

@probonopd thank you for the proposal. I tried out appimage tool, but the application panicked:

2023/06/19 14:34:58 Got qt_prfxpath but it does not contain 'plugins'
panic: runtime error: invalid memory address or nil pointer dereference
[signal SIGSEGV: segmentation violation code=0x1 addr=0x30 pc=0xb29cea]

goroutine 1 [running]:
github.com/probonopd/go-appimage/internal/helpers.FilesWithSuffixInDirectoryRecursive.func1({0xc000409068, 0x15}, {0x0?, 0x0?}, {0x14?, 0xc000491310?})
	github.com/probonopd/go-appimage/internal/helpers/helpers.go:105 +0x4a
path/filepath.Walk({0xc000409068, 0x15}, 0xc000491320)
	path/filepath/path.go:515 +0x50
github.com/probonopd/go-appimage/internal/helpers.FilesWithSuffixInDirectoryRecursive({0xc000409068?, 0xc000409068?}, {0x30f434?, 0x30c481?})
	github.com/probonopd/go-appimage/internal/helpers/helpers.go:104 +0x73
main.getQtPrfxpath(0xc000479380?, {0x2f?, 0x0?}, 0xc0004914a8?)
	github.com/probonopd/go-appimage/src/appimagetool/appdirtool.go:1424 +0x309
main.handleQt({{0xc0000fad20, 0x6}, {0xc0000aeb70, 0x16}, {0xc0000aec78, 0x16}}, 0x5)
	github.com/probonopd/go-appimage/src/appimagetool/appdirtool.go:1181 +0x16c
main.AppDirDeploy({0x7fffd05e8b01?, 0x313ddd?})
	github.com/probonopd/go-appimage/src/appimagetool/appdirtool.go:270 +0x685
main.bootstrapAppImageDeploy(0xc0001da2c0)
	github.com/probonopd/go-appimage/src/appimagetool/cli.go:36 +0x1fd
github.com/urfave/cli/v2.(*Command).Run(0xc0000fd7a0, 0xc0001da1c0)
	github.com/urfave/cli/[email protected]/command.go:177 +0x719
github.com/urfave/cli/v2.(*App).RunContext(0xc000114380, {0x3f4c78?, 0xc0000b6020}, {0xc0000b4000, 0x4, 0x4})
	github.com/urfave/cli/[email protected]/app.go:387 +0x1035
github.com/urfave/cli/v2.(*App).Run(...)
	github.com/urfave/cli/[email protected]/app.go:252
main.main()
	github.com/probonopd/go-appimage/src/appimagetool/cli.go:288 +0x6df

If you need more information, see latest ci build

probonopd · 2023-07-05T17:46:52Z

@Murmele you do not need linuxdeployqt, AppImageKit's appimagetool nor patchelf (the latter somes with go-appimage's appimagetool).

Actually this is all it takes; the following works for me on Ubuntu 20.04.6 LTS:

git submodule init
git submodule update

# Build OpenSSL; why does it need a private (self-built) version?
cd dep/openssl/openssl
./config -fPIC
make -j $(npoc)
cd ../../../

# Install Qt
sudo add-apt-repository ppa:beineri/opt-qt-5.15.2-focal -y
sudo apt-get update -qq
sudo apt-get -y install qt515base qt515tools libgl1-mesa-dev
. /opt/qt*/bin/qt*-env.sh

# Install ninja
sudo pip install ninja

# Build
mkdir -p build/release
cd build/release
cmake -G Ninja -DCMAKE_BUILD_TYPE=Release -DCMAKE_INSTALL_PREFIX=/usr/ ../..
ninja
DESTDIR=./appdir ninja install ; find ./appdir
rm -rf ./appdir/usr/include/

# Deploy dependencies into AppDir
wget -c https://github.com/$(wget -q https://github.com/probonopd/go-appimage/releases/expanded_assets/continuous -O - | grep "appimagetool-.*-x86_64.AppImage" | head -n 1 | cut -d '"' -f 2)
chmod +x appimagetool-*.AppImage
LD_LIBRARY_PATH=/opt/qt515/lib/ ./appimagetool-*.AppImage -s deploy ./appdir/usr/share/applications/*.desktop --appimage-extract-and-run # Bundle EVERYTHING

# Turn AppDir into AppImage
./appimagetool-*.AppImage ./appdir --appimage-extract-and-run # turn AppDir into AppImage

# Two files have been generated, both must be uploaded to GitHub Releases for AppImageUpdate to work
ls -lh Gittyup-*
-rwxr-xr-x 1 codespace codespace  43M Jul  5 23:44 Gittyup-b0d1c70-x86_64.AppImage
-rw-rw-rw- 1 codespace codespace 150K Jul  5 23:44 Gittyup-b0d1c70-x86_64.AppImage.zsync

For AppImageUpdate to work, don't forget to upload both generated files to GitHub Releases.

Here you see the AppImage running:

Murmele · 2023-07-06T07:57:30Z

Hi @probonopd . thank you I will try it out!

probonopd · 2023-07-08T12:32:29Z

On https://github.com/probonopd/Gittyup/releases there is an AppImage built on GitHub Releases like this:
https://github.com/probonopd/Gittyup/blob/master/.github/workflows/build.yml

There is one remaining issue that can probably only be fixed by editing the source code:
Somehow, local paths from the build system get compiled into the binary:

~/Downloads/Gittyup-14-x86_64.AppImage --appimage-extract # Extract the AppImage
./squashfs-root/AppRun # Run the application
PANIC: unprotected error in call to Lua API (cannot open /home/runner/work/Gittyup/Gittyup/conf/System.lua: No such file or directory)

/home/runner/work/Gittyup/Gittyup/conf/System.lua possibly existed on the build machine, but for sure it doesn't exist on the user's machine.

Where is it compiled into?

grep -r /home/runner/work/Gittyup/Gittyup squashfs-root
Binary file squashfs-root/usr/bin/gittyup matches
Binary file squashfs-root/usr/bin/indexer matches
(...)
strings squashfs-root/usr/bin/gittyup | grep "home/runner"
/home/runner/work/Gittyup/Gittyup/conf
/home/runner/work/Gittyup/Gittyup/l10n

strings squashfs-root/usr/bin/indexer | grep "home/runner"
/home/runner/work/Gittyup/Gittyup/conf
/home/runner/work/Gittyup/Gittyup/l10n
(...)

That needs to be changed.

Murmele · 2023-07-10T11:12:24Z

/Gittyup/conf/System.lua

Hi @probonopd,

thank you for the CI setup!

I installed your appimage:
On a RHEL 7.9 I am getting the following error message:
/tmp/.mount_GittyuhibPFE/usr/bin/gittyup: error while loading shared libraries: libQt5Core.so.5: cannot open shared object file: No such file or directory
I extracted the appimage and indeed, no qt dependencies are in there

I found the problem in my build. I set QT_DIR but your tool is searching for QTDIR.
I published GittyupAppdir (Appdir directory with all dependencies) as artifact and executed it. I got the same panic as you (I will have a look into it), but creating the appimage file does not work with the following error:

LD_LIBRARY_PATH='' find AppDir -type f -exec ldd {} 2>&1 \; | grep '=>' | grep -v AppDir
2023/07/10 12:45:51 Architecture of libQt5Concurrent.so.5: x86_64
Gittyup
ERROR Determine architecture: Could not detect a valid architecture
2023/07/10 12:45:51 git root: /home/runner/work/Gittyup/Gittyup
2023/07/10 12:45:51 Target AppImage filename: Gittyup-1.0-x86_64.AppImage
2023/07/10 12:45:51 Icon file: AppDir/usr/share/icons/hicolor/256x256/apps/gittyup.png
Trying to validate AppStream information with the appstreamcli tool
ERROR appstreamcli: signal: segmentation fault (core dumped)

probonopd · 2023-07-10T17:24:31Z

I installed your appimage:
On a RHEL 7.9 I am getting the following error message:
/tmp/.mount_GittyuhibPFE/usr/bin/gittyup: error while loading shared libraries: libQt5Core.so.5: cannot open shared object file: No such file or directory
I extracted the appimage and indeed, no qt dependencies are in there

Interesting! Are you sure you used the AppImage from https://github.com/probonopd/Gittyup/releases?
It definitely contains libQt5Core.so.5, I just double checked.
And it starts on openSUSE, but then immediately crashes with a Lua API panic due to a missing file in /home.

This is the strace output:

I will retry it on a CentOS 7.8 Live ISO.
Edit: I can reproduce it there, libQt5Core.so.5 does not get loaded. I am looking into it.

probonopd · 2023-07-10T17:25:48Z

Trying to validate AppStream information with the appstreamcli tool
ERROR appstreamcli: signal: segmentation fault (core dumped)

For now, remove the AppStream metainfo file and see if it builds then. We can sort out the AppStream file later.

Murmele · 2023-07-10T17:37:27Z

Sorry. I tested again and now on Arch it works (without considering the LUA problem).
For some reason I don't get any debug() output. I tested locally and I get it. Tried also to log to a file but without success. I have to check

probonopd · 2023-07-10T17:44:00Z

This is an interesting problem. On a CentOS 7.8 Live ISO, when I extract the AppImage and then run strace ./squashfs-root/AppRun, it appears like it finds and successfully open (no ENOENT) libQt5Core.so.5, does a short pread on it, and then closes it again with close(3), then continues to search for it in other locations. It is almost like it decides it doesn't like that library.

The kernel on CentOS 7.8 is 3.10.0, this is a reeeeally old kernel. Inspecting libQt5Core.so.5 with the file command shows that it claims to need at least kernel 3.17.0 (which is from October 2013; almost 10 years ago). It seems like we would need to find another build of Qt that supports older kernels.

What kernel version are you using?

Murmele · 2023-07-10T18:11:57Z

RHEL7.9 has 3.10.0 as kernel version

I tried no my version of the appimage and I get (On an up to date Arch Linux)

Something went wrong trying to read the squashfs image.

Cannot mount AppImage, please check your FUSE setup.
You might still be able to extract the contents of this AppImage 
if you run it with the --appimage-extract option. 
See https://github.com/AppImage/AppImageKit/wiki/FUSE 
for more information
open dir error: No such file or directory

probonopd · 2023-07-10T18:33:59Z

RHEL7.9 has 3.10.0 as kernel version

It seems that the kernel is too old to run that build of Qt.

I tried no my version of the appimage and I get (On an up to date Arch Linux)

What does which fusermount ; which fusermount3 say on your system?
Does it work with --appimage-extract-and-run?
Does my AppImage run on that same system?

Murmele · 2023-07-10T18:41:24Z

which fusermount ; which fusermount3

/usr/bin/fusermount
/usr/bin/fusermount3

Yes your build is working fine.

The problem with the PANIC seems to be that the qt applicationDirPath() does not direct to the gittyup executable, but to the linux-ld lib
https://github.com/Murmele/Gittyup/blob/master/src/conf/Settings.cpp#L198-L203

Because of this all other paths (themes, config, plugins, ...) are wrong and therefore Lua panics, because it cannot find the required files

probonopd · 2023-07-10T18:47:59Z

That is an unintended side effect of using -s deploy. You can remove the -s and see what happens. Downside is that it will run only on systems no older than the build system.

And I have one more idea, but I need to test that one first...

probonopd · 2023-07-10T19:01:41Z

Tested this locally, seems to be working:

https://github.com/probonopd/Gittyup/blob/e9ce88ab710a347817d4980ce101db75bf759af6/appimage-builder/build.sh#L37-L40

This way you can use -s deploy and it will still work.

The next build of my AppImage should contain the fix.

Maybe I will change go-appimage appimagetool to do something like this automatically in the future (possibly by using symlinks).

probonopd · 2023-07-10T19:12:34Z

My AppImage now works properly (on systems with Linux kernel 3.17.0 and newer). If you need this to work on older kernels, we need to find/build another Qt.

Murmele · 2023-07-10T19:26:00Z

Great :D I am waiting until the ci is finished and then I hope I can say the same thing :)

Which qt version is available on CentOS? I will try it on RHEL7.9

probonopd · 2023-07-10T20:15:50Z

Which qt version is available on CentOS? I will try it on RHEL7.9
https://centos.pkgs.org/7/centos-x86_64/qt5-qtbase-5.9.7-4.el7.x86_64.rpm.html

Apparently Qt 5.9.7. But I have another idea, let me try.

Murmele · 2023-07-10T20:27:12Z

Which qt version is available on CentOS? I will try it on RHEL7.9
https://centos.pkgs.org/7/centos-x86_64/qt5-qtbase-5.9.7-4.el7.x86_64.rpm.html

Apparently Qt 5.9.7. But I have another idea, let me try.

Oh thats low. On Arch my build works now as well. Thank you very much. Now the easy part of cleaning up is missing ^^
On RHEL7.9 it works also pretty nice. The indexer crashes during startup, but after it restarts it seems to work fine

probonopd · 2023-07-10T23:47:37Z

Tried to build on Ubuntu xenial (that Qt would likely run on RHEL 7), but it doesn't build there.

Murmele · 2023-07-11T05:31:00Z

Tried to build on Ubuntu xenial (that Qt would likely run on RHEL 7), but it doesn't build there.

Whats the error message you are getting?

probonopd · 2023-07-12T17:31:06Z

https://github.com/probonopd/Gittyup/actions/runs/5513710024/jobs/10052156512#step:4:10994

Murmele · 2023-07-13T08:34:55Z

On RHEL7.9, remote workstation

qt.glx: qglx_findConfig: Failed to finding matching FBConfig for QSurfaceFormat(version 2.0, options QFlags<QSurfaceFormat::FormatOption>(), depthBufferSize -1, redBufferSize 1, greenBufferSize 1, blueBufferSinterval 1, colorSpace QSurfaceFormat::DefaultColorSpace, profile  QSurfaceFormat::NoProfile)
No XVisualInfo for format QSurfaceFormat(version 2.0, options QFlags<QSurfaceFormat::FormatOption>(), depthBufferSize -1, redBufferSize 1, greenBufferSize 1, blueBufferSize 1, alphaBufferSize -1, stencilBufferefaultColorSpace, profile  QSurfaceFormat::NoProfile)
Falling back to using screens root_visual.

https://github.com/flameshot-org/flameshot/pull/1034/files

RPCS3/rpcs3#4509

Murmele · 2023-07-13T09:46:26Z

On RHEL7.9, remote workstation

qt.glx: qglx_findConfig: Failed to finding matching FBConfig for QSurfaceFormat(version 2.0, options QFlags<QSurfaceFormat::FormatOption>(), depthBufferSize -1, redBufferSize 1, greenBufferSize 1, blueBufferSinterval 1, colorSpace QSurfaceFormat::DefaultColorSpace, profile  QSurfaceFormat::NoProfile)
No XVisualInfo for format QSurfaceFormat(version 2.0, options QFlags<QSurfaceFormat::FormatOption>(), depthBufferSize -1, redBufferSize 1, greenBufferSize 1, blueBufferSize 1, alphaBufferSize -1, stencilBufferefaultColorSpace, profile  QSurfaceFormat::NoProfile)
Falling back to using screens root_visual.

https://github.com/flameshot-org/flameshot/pull/1034/files

RPCS3/rpcs3#4509

Fixed with rm ./AppDir/lib/x86_64-linux-gnu/libxcb-glx.so.0
removing all xcb leads to a SegFault rm ./AppDir/lib/x86_64-linux-gnu/libxcb*

Murmele · 2023-07-13T10:01:19Z

https://github.com/probonopd/Gittyup/actions/runs/5513710024/jobs/10052156512#step:4:10994

Does the ci build do not work on RHEL7?

Murmele · 2023-07-14T11:23:06Z

@probonopd on RHEL7.9 the indexer application crashes with a segmentation fault. Tried to removing everything except the main(), but I had no success with it

#595

probonopd · 2023-07-15T09:54:44Z

Fixed with rm ./AppDir/lib/x86_64-linux-gnu/libxcb-glx.so.0

That's not a good idea for applications deployed with -s deploy because then the application starts pulling in system libraries, which can lead to truly unpredictable results.

probonopd · 2023-07-15T09:55:46Z

Does the ci build do not work on RHEL7?

You mean, building on RHEL? I have never tried that.

Why is RHEL 7 so important for your use case? Even RH will maintain it only for another year:

https://www.redhat.com/en/blog/end-maintenance-red-hat-enterprise-linux-7-almost-here

Murmele · 2023-07-17T09:21:12Z

You mean, building on RHEL? I have never tried that.

Mine is almost working fine on RHEL, just a segmentation fault with the indexer I have to find out. At work we are using RHEL7.9

RHEL 7.x is still supported: https://en.wikipedia.org/wiki/Red_Hat_Enterprise_Linux

probonopd · 2023-07-18T20:31:51Z

Right, still supported by RH for a year.

Murmele · 2023-07-24T07:13:05Z

At work we are getting RHEL9 by the end of the year, so I will test it there again, maybe we don't need this hack with deleting the library

…onfig for QSurfaceFormat

This reverts commit 9bcfefe.

…hing FBConfig for QSurfaceFormat" This reverts commit 0c5035f.

Murmele · 2023-11-12T10:58:08Z

Fixed with rm ./AppDir/lib/x86_64-linux-gnu/libxcb-glx.so.0

That's not a good idea for applications deployed with -s deploy because then the application starts pulling in system libraries, which can lead to truly unpredictable results.

I removed this again

Murmele marked this pull request as draft June 13, 2023 09:09

Murmele changed the title ~~initial commit~~ Appimage support Jun 14, 2023

Murmele force-pushed the appimage branch from bbdab04 to d59296d Compare June 14, 2023 16:05

Murmele mentioned this pull request Jun 16, 2023

feat: add deb and rpm package #198

Open

jmoraleda mentioned this pull request Jun 16, 2023

Build system should not assume everything is vendored and bundled #581

Open

Murmele force-pushed the appimage branch from f67cfba to eacb8db Compare June 19, 2023 14:38

Murmele mentioned this pull request Jun 28, 2023

qt_prfxpath is wrong if not system/locally built qt is used probonopd/go-appimage#216

Open

probonopd mentioned this pull request Jul 10, 2023

appimagetool -s deploy breaks things like qApp->applicationDirPath() probonopd/go-appimage#260

Open

Murmele force-pushed the appimage branch 2 times, most recently from c261bdc to 776649c Compare July 10, 2023 20:46

Murmele force-pushed the appimage branch from e346bca to b27f4c9 Compare July 11, 2023 08:54

Murmele force-pushed the appimage branch from e7d3451 to 7ccbf23 Compare July 13, 2023 09:45

Murmele force-pushed the appimage branch from 7ccbf23 to 0c5035f Compare July 22, 2023 18:49

Murmele mentioned this pull request Jul 22, 2023

Offline install flatpak #478

Open

Martin Marmsoler and others added 11 commits November 12, 2023 11:57

Reason:

897f658

disable updating over gui

7b82477

do not publish linux artifacts

a21cf7e

publish also appimagetool

ceb597b

cleanup

52db196

trying to fix qt.glx: qglx_findConfig: Failed to finding matching FBC…

34aef5b

…onfig for QSurfaceFormat

add Debugging

0e6e1e4

Debug build

2326e01

use release build

f81c01f

Revert "add Debugging"

3f2d236

This reverts commit 9bcfefe.

Revert "trying to fix qt.glx: qglx_findConfig: Failed to finding matc…

1b74bf7

…hing FBConfig for QSurfaceFormat" This reverts commit 0c5035f.

Murmele force-pushed the appimage branch from f1b641d to 1b74bf7 Compare November 12, 2023 10:57

Murmele marked this pull request as ready for review November 12, 2023 10:58

Murmele merged commit 2394355 into master Nov 12, 2023
11 checks passed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Appimage support #579

Appimage support #579

Murmele commented Jun 13, 2023 •

edited

Loading

probonopd commented Jun 14, 2023

Murmele commented Jun 19, 2023 •

edited

Loading

probonopd commented Jul 5, 2023 •

edited

Loading

Murmele commented Jul 6, 2023

probonopd commented Jul 8, 2023

Murmele commented Jul 10, 2023 •

edited

Loading

probonopd commented Jul 10, 2023 •

edited

Loading

probonopd commented Jul 10, 2023

Murmele commented Jul 10, 2023

probonopd commented Jul 10, 2023 •

edited

Loading

Murmele commented Jul 10, 2023 •

edited

Loading

probonopd commented Jul 10, 2023 •

edited

Loading

Murmele commented Jul 10, 2023

probonopd commented Jul 10, 2023 •

edited

Loading

probonopd commented Jul 10, 2023 •

edited

Loading

probonopd commented Jul 10, 2023

Murmele commented Jul 10, 2023

probonopd commented Jul 10, 2023

Murmele commented Jul 10, 2023

probonopd commented Jul 10, 2023

Murmele commented Jul 11, 2023

probonopd commented Jul 12, 2023

Murmele commented Jul 13, 2023

Murmele commented Jul 13, 2023

Murmele commented Jul 13, 2023

Murmele commented Jul 14, 2023

probonopd commented Jul 15, 2023

probonopd commented Jul 15, 2023 •

edited

Loading

Murmele commented Jul 17, 2023

probonopd commented Jul 18, 2023

Murmele commented Jul 24, 2023

Murmele commented Nov 12, 2023

Appimage support #579

Appimage support #579

Conversation

Murmele commented Jun 13, 2023 • edited Loading

probonopd commented Jun 14, 2023

Murmele commented Jun 19, 2023 • edited Loading

probonopd commented Jul 5, 2023 • edited Loading

Murmele commented Jul 6, 2023

probonopd commented Jul 8, 2023

Murmele commented Jul 10, 2023 • edited Loading

probonopd commented Jul 10, 2023 • edited Loading

probonopd commented Jul 10, 2023

Murmele commented Jul 10, 2023

probonopd commented Jul 10, 2023 • edited Loading

Murmele commented Jul 10, 2023 • edited Loading

probonopd commented Jul 10, 2023 • edited Loading

Murmele commented Jul 10, 2023

probonopd commented Jul 10, 2023 • edited Loading

probonopd commented Jul 10, 2023 • edited Loading

probonopd commented Jul 10, 2023

Murmele commented Jul 10, 2023

probonopd commented Jul 10, 2023

Murmele commented Jul 10, 2023

probonopd commented Jul 10, 2023

Murmele commented Jul 11, 2023

probonopd commented Jul 12, 2023

Murmele commented Jul 13, 2023

Murmele commented Jul 13, 2023

Murmele commented Jul 13, 2023

Murmele commented Jul 14, 2023

probonopd commented Jul 15, 2023

probonopd commented Jul 15, 2023 • edited Loading

Murmele commented Jul 17, 2023

probonopd commented Jul 18, 2023

Murmele commented Jul 24, 2023

Murmele commented Nov 12, 2023

Murmele commented Jun 13, 2023 •

edited

Loading

Murmele commented Jun 19, 2023 •

edited

Loading

probonopd commented Jul 5, 2023 •

edited

Loading

Murmele commented Jul 10, 2023 •

edited

Loading

probonopd commented Jul 10, 2023 •

edited

Loading

probonopd commented Jul 10, 2023 •

edited

Loading

Murmele commented Jul 10, 2023 •

edited

Loading

probonopd commented Jul 10, 2023 •

edited

Loading

probonopd commented Jul 10, 2023 •

edited

Loading

probonopd commented Jul 10, 2023 •

edited

Loading

probonopd commented Jul 15, 2023 •

edited

Loading