Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

SABnzbd: use cross/7zz and cross/unrar7 #6366

Merged
merged 16 commits into from
Jan 25, 2025
Merged

SABnzbd: use cross/7zz and cross/unrar7 #6366

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
16 commits
Select commit Hold shift + click to select a range
1058a4a
add cross/7zip
hgy59 Dec 25, 2024
1323aea
add cross/unrar7
hgy59 Dec 25, 2024
6969245
cross/7zip: use xz archive for source
Jan 2, 2025
4f78b4e
Merge branch 'master' of https://github.com/hgy59/spksrc into add_7zip
Jan 2, 2025
afa3d4a
rename cross/7zip to cross/7zz
Jan 3, 2025
8b5f912
add cross/7za
Jan 3, 2025
ed99274
sabnzbd: use cross/7za and cross/unrar7 for DSM 7
Jan 3, 2025
c3042d5
sabnzbd: fix comcerto2k
Jan 3, 2025
fb5f963
fix 7zip for armv7 and older compilers (DSM 6)
hgy59 Jan 3, 2025
678dfd1
Merge branch 'master' of https://github.com/hgy59/spksrc into add_7zip
hgy59 Jan 11, 2025
673f08a
sabnzbd: update changelog and package revision
hgy59 Jan 11, 2025
91f147a
sabnzbdd: revert unrar in SPK_COMMANDS
hgy59 Jan 11, 2025
9108893
sabnzb: remove cross/busybox dependency
hgy59 Jan 13, 2025
f29708e
sabnzbd: update cross/par2cmdline-turbo
hgy59 Jan 13, 2025
fbbc933
fix 7zip
hgy59 Jan 13, 2025
e642275
sabnzbd: replace 7za by 7zz
hgy59 Jan 14, 2025
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions Dockerfile
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -59,6 +59,7 @@ RUN apt update && apt install --no-install-recommends -y \
mlocate \
moreutils \
nasm \
p7zip \
patchelf \
php \
pkg-config \
Expand Down
60 changes: 60 additions & 0 deletions cross/7za/Makefile
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,60 @@
PKG_NAME = 7za
PKG_VERS = 24.09
PKG_EXT = tar.xz
PKG_DIST_NAME = 7z$(subst .,,$(PKG_VERS))-src.$(PKG_EXT)
PKG_DIST_SITE = https://github.com/ip7z/7zip/releases/download/$(PKG_VERS)
PKG_DIR = $(PKG_NAME)-$(PKG_VERS)/CPP/7zip/Bundles/Alone

# source archive does not have a package named top folder
# https://github.com/ip7z/7zip/issues/45
EXTRACT_PATH = $(WORK_DIR)/$(PKG_NAME)-$(PKG_VERS)

DEPENDS =

# compiler too old
UNSUPPORTED_ARCHS = $(OLD_PPC_ARCHS) $(ARMv5_ARCHS)

HOMEPAGE = https://www.7-zip.org/
COMMENT = 7-Zip is a file archiver with a high compression ratio.
LICENSE = https://github.com/ip7z/7zip/blob/main/DOC/License.txt

CONFIGURE_TARGET = nop
COMPILE_MAKE_OPTIONS = -f makefile.gcc

PATCH_TARGET = 7z_patch
INSTALL_TARGET = 7z_install

include ../../mk/spksrc.cross-cc.mk

# disable unsupported HW capabilities:
ifeq ($(findstring $(ARCH),$(ARMv7_ARCHS) $(ARMv7L_ARCHS)),$(ARCH))
# ARMv7 fails with some undefined symbols
ADDITIONAL_CFLAGS += -DHWCAP2_CRC32=0 -DHWCAP2_SHA1=0 -DHWCAP2_SHA2=0 -DHWCAP2_AES=0
endif
ifeq ($(findstring $(ARCH),$(ARMv7L_ARCHS)),$(ARCH))
# ARMv7L is missing one more symbol:
ADDITIONAL_CFLAGS += -DHWCAP_NEON=0
endif

ENV += ADDITIONAL_CFLAGS="$(ADDITIONAL_CFLAGS)"

.PHONY: 7z_patch
7z_patch:
ifneq ($(strip $(PATCHES)),)
@$(MSG) Make files to patch writable
chmod +w $(EXTRACT_PATH)/CPP/7zip/7zip_gcc.mak
chmod +w $(EXTRACT_PATH)/CPP/7zip/Crypto/MyAes.cpp
chmod +w $(EXTRACT_PATH)/C/XzCrc64Opt.c
@$(MSG) Run patch in $(EXTRACT_PATH) instead of $(WORK_DIR)/$(PKG_DIR)
@for patchfile in $(PATCHES) ; \
do \
echo "patch -p$(PATCHES_LEVEL) < $${patchfile}" ; \
cat $${patchfile} | (cd $(EXTRACT_PATH) && patch -p$(PATCHES_LEVEL)) ; \
done
endif

.PHONY: 7z_install
7z_install:
@$(MSG) Install target file
@install -d -m 755 $(STAGING_INSTALL_PREFIX)/bin
@install -m 755 $(WORK_DIR)/$(PKG_DIR)/_o/7za $(STAGING_INSTALL_PREFIX)/bin
1 change: 1 addition & 0 deletions cross/7za/PLIST
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
bin:bin/7za
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

For some reason SABnzbd isn't seeing it and jumping to the old system 7zip.

2025-01-13 22:37:33,269::INFO::[SABnzbd:460] UNRAR binary... found (/volume1/@appstore/sabnzbd/bin/unrar)
2025-01-13 22:37:33,269::INFO::[SABnzbd:468] UNRAR binary version 7.10
2025-01-13 22:37:33,270::INFO::[SABnzbd:475] 7za binary... found (/bin/7z)
2025-01-13 22:37:33,270::INFO::[SABnzbd:477] 7za binary version 16.02

We just look at the PATH and then check for any of the listed executable names in each part of the PATH.
https://github.com/sabnzbd/sabnzbd/blob/7de08693765fac39192e4cbcffdcc725244a943e/sabnzbd/newsunpack.py#L148-L154

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@Safihre thanks for the feedback, guess I fixed it.

3 changes: 3 additions & 0 deletions cross/7za/digests
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,3 @@
7z2409-src.tar.xz SHA1 fb0ef0558895ad5f33f6ba7e096c90db3f60f146
7z2409-src.tar.xz SHA256 49c05169f49572c1128453579af1632a952409ced028259381dac30726b6133a
7z2409-src.tar.xz MD5 310660dea1e78cfd676cf142f8dc247a
15 changes: 15 additions & 0 deletions cross/7za/patches/001-allow-additional-cflags-for-gcc.patch
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,15 @@
# add ADDITIONAL_CFLAGS to makefile for gcc
#
# allow to define additional cflags (required to fix build for 32-bit arm archs)
#
--- CPP/7zip/7zip_gcc.mak.orig 2024-11-25 17:00:00.000000000 +0000
+++ CPP/7zip/7zip_gcc.mak 2025-01-03 17:13:03.955679931 +0000
@@ -50,7 +50,7 @@
endif

# CFLAGS_BASE_LIST = -S
-CFLAGS_BASE = -O2 $(CFLAGS_BASE_LIST) $(CFLAGS_WARN_WALL) $(CFLAGS_WARN) \
+CFLAGS_BASE = -O2 $(CFLAGS_BASE_LIST) $(CFLAGS_WARN_WALL) $(CFLAGS_WARN) $(ADDITIONAL_CFLAGS) \
$(CFLAGS_DEBUG) -D_REENTRANT -D_FILE_OFFSET_BITS=64 -D_LARGEFILE_SOURCE \
-fPIC

53 changes: 53 additions & 0 deletions cross/7za/patches/002-fix-hw_aes_256-for-older-gcc.patch
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,53 @@
# Fix provided by Igor Pavlov for:
# Inquiry Regarding Build Error Encountered While Compiling 7-Zip:undefined reference to `AesCbc_Decode_HW_256'
# in https://sourceforge.net/p/sevenzip/discussion/45797/thread/2a49e0adde/
#
--- CPP/7zip/Crypto/MyAes.cpp.orig 2024-03-01 09:00:00.000000000 +0000
+++ CPP/7zip/Crypto/MyAes.cpp 2025-01-03 18:38:35.330153025 +0000
@@ -153,7 +153,26 @@
#ifndef Z7_EXTRACT_ONLY

#ifdef MY_CPU_X86_OR_AMD64
- #define USE_HW_AES
+
+ #if defined(__INTEL_COMPILER)
+ #if (__INTEL_COMPILER >= 1110)
+ #define USE_HW_AES
+ #if (__INTEL_COMPILER >= 1900)
+ #define USE_HW_VAES
+ #endif
+ #endif
+ #elif defined(Z7_CLANG_VERSION) && (Z7_CLANG_VERSION >= 30800) \
+ || defined(Z7_GCC_VERSION) && (Z7_GCC_VERSION >= 40400)
+ #define USE_HW_AES
+ #if defined(__clang__) && (__clang_major__ >= 8) \
+ || defined(__GNUC__) && (__GNUC__ >= 8)
+ #define USE_HW_VAES
+ #endif
+ #elif defined(_MSC_VER)
+ #define USE_HW_AES
+ #define USE_HW_VAES
+ #endif
+
#elif defined(MY_CPU_ARM_OR_ARM64) && defined(MY_CPU_LE)

#if defined(__ARM_FEATURE_AES) \
@@ -186,15 +205,15 @@
#define SET_AES_FUNC_2(f2) \
if (algo == 2) if (g_Aes_SupportedFunctions_Flags & k_Aes_SupportedFunctions_HW) \
{ f = f2; }
- #ifdef MY_CPU_X86_OR_AMD64
+ #ifdef USE_HW_VAES
#define SET_AES_FUNC_23(f2, f3) \
SET_AES_FUNC_2(f2) \
if (algo == 3) if (g_Aes_SupportedFunctions_Flags & k_Aes_SupportedFunctions_HW_256) \
{ f = f3; }
- #else // MY_CPU_X86_OR_AMD64
+ #else // USE_HW_VAES
#define SET_AES_FUNC_23(f2, f3) \
SET_AES_FUNC_2(f2)
- #endif // MY_CPU_X86_OR_AMD64
+ #endif // USE_HW_VAES
#else // USE_HW_AES
#define SET_AES_FUNC_23(f2, f3)
#endif // USE_HW_AES
14 changes: 14 additions & 0 deletions cross/7za/patches/003-fix-XzCrc64Opt.patch
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,14 @@
# fix preprocessor commands
# https://sourceforge.net/p/sevenzip/discussion/45797/thread/d9908ffd01/
#
--- C/XzCrc64Opt.c.orig 2023-12-08 08:00:00.000000000 +0000
+++ C/XzCrc64Opt.c 2025-01-03 19:08:34.320848333 +0000
@@ -235,7 +235,7 @@
v = Q32BE(1, w1) ^ Q32BE(0, w0);
v ^= Q32BE(3, d1) ^ Q32BE(2, d0);
#endif
-#elif
+#else
#error Stop_Compiling_Bad_CRC64_NUM_TABLES
#endif
p += Z7_CRC64_NUM_TABLES_USE;
60 changes: 60 additions & 0 deletions cross/7zz/Makefile
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,60 @@
PKG_NAME = 7zz
PKG_VERS = 24.09
PKG_EXT = tar.xz
PKG_DIST_NAME = 7z$(subst .,,$(PKG_VERS))-src.$(PKG_EXT)
PKG_DIST_SITE = https://github.com/ip7z/7zip/releases/download/$(PKG_VERS)
PKG_DIR = $(PKG_NAME)-$(PKG_VERS)/CPP/7zip/Bundles/Alone2

# source archive does not have a package named top folder
# https://github.com/ip7z/7zip/issues/45
EXTRACT_PATH = $(WORK_DIR)/$(PKG_NAME)-$(PKG_VERS)

DEPENDS =

# compiler too old
UNSUPPORTED_ARCHS = $(OLD_PPC_ARCHS) $(ARMv5_ARCHS)

HOMEPAGE = https://www.7-zip.org/
COMMENT = 7-Zip is a file archiver with a high compression ratio.
LICENSE = https://github.com/ip7z/7zip/blob/main/DOC/License.txt

CONFIGURE_TARGET = nop
COMPILE_MAKE_OPTIONS = -f makefile.gcc

PATCH_TARGET = 7z_patch
INSTALL_TARGET = 7z_install

include ../../mk/spksrc.cross-cc.mk

# disable unsupported HW capabilities:
ifeq ($(findstring $(ARCH),$(ARMv7_ARCHS) $(ARMv7L_ARCHS)),$(ARCH))
# ARMv7 fails with some undefined symbols
ADDITIONAL_CFLAGS += -DHWCAP2_CRC32=0 -DHWCAP2_SHA1=0 -DHWCAP2_SHA2=0 -DHWCAP2_AES=0
endif
ifeq ($(findstring $(ARCH),$(ARMv7L_ARCHS)),$(ARCH))
# ARMv7L is missing one more symbol:
ADDITIONAL_CFLAGS += -DHWCAP_NEON=0
endif

ENV += ADDITIONAL_CFLAGS="$(ADDITIONAL_CFLAGS)"

.PHONY: 7z_patch
7z_patch:
ifneq ($(strip $(PATCHES)),)
@$(MSG) Make files to patch writable
chmod +w $(EXTRACT_PATH)/CPP/7zip/7zip_gcc.mak
chmod +w $(EXTRACT_PATH)/CPP/7zip/Crypto/MyAes.cpp
chmod +w $(EXTRACT_PATH)/C/XzCrc64Opt.c
@$(MSG) Run patch in $(EXTRACT_PATH) instead of $(WORK_DIR)/$(PKG_DIR)
@for patchfile in $(PATCHES) ; \
do \
echo "patch -p$(PATCHES_LEVEL) < $${patchfile}" ; \
cat $${patchfile} | (cd $(EXTRACT_PATH) && patch -p$(PATCHES_LEVEL)) ; \
done
endif

.PHONY: 7z_install
7z_install:
@$(MSG) Install target file
@install -d -m 755 $(STAGING_INSTALL_PREFIX)/bin
@install -m 755 $(WORK_DIR)/$(PKG_DIR)/_o/7zz $(STAGING_INSTALL_PREFIX)/bin
1 change: 1 addition & 0 deletions cross/7zz/PLIST
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
bin:bin/7zz
3 changes: 3 additions & 0 deletions cross/7zz/digests
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,3 @@
7z2409-src.tar.xz SHA1 fb0ef0558895ad5f33f6ba7e096c90db3f60f146
7z2409-src.tar.xz SHA256 49c05169f49572c1128453579af1632a952409ced028259381dac30726b6133a
7z2409-src.tar.xz MD5 310660dea1e78cfd676cf142f8dc247a
15 changes: 15 additions & 0 deletions cross/7zz/patches/001-allow-additional-cflags-for-gcc.patch
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,15 @@
# add ADDITIONAL_CFLAGS to makefile for gcc
#
# allow to define additional cflags (required to fix build for 32-bit arm archs)
#
--- CPP/7zip/7zip_gcc.mak.orig 2024-11-25 17:00:00.000000000 +0000
+++ CPP/7zip/7zip_gcc.mak 2025-01-03 17:13:03.955679931 +0000
@@ -50,7 +50,7 @@
endif

# CFLAGS_BASE_LIST = -S
-CFLAGS_BASE = -O2 $(CFLAGS_BASE_LIST) $(CFLAGS_WARN_WALL) $(CFLAGS_WARN) \
+CFLAGS_BASE = -O2 $(CFLAGS_BASE_LIST) $(CFLAGS_WARN_WALL) $(CFLAGS_WARN) $(ADDITIONAL_CFLAGS) \
$(CFLAGS_DEBUG) -D_REENTRANT -D_FILE_OFFSET_BITS=64 -D_LARGEFILE_SOURCE \
-fPIC

53 changes: 53 additions & 0 deletions cross/7zz/patches/002-fix-hw_aes_256-for-older-gcc.patch
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,53 @@
# Fix provided by Igor Pavlov for:
# Inquiry Regarding Build Error Encountered While Compiling 7-Zip:undefined reference to `AesCbc_Decode_HW_256'
# in https://sourceforge.net/p/sevenzip/discussion/45797/thread/2a49e0adde/
#
--- CPP/7zip/Crypto/MyAes.cpp.orig 2024-03-01 09:00:00.000000000 +0000
+++ CPP/7zip/Crypto/MyAes.cpp 2025-01-03 18:38:35.330153025 +0000
@@ -153,7 +153,26 @@
#ifndef Z7_EXTRACT_ONLY

#ifdef MY_CPU_X86_OR_AMD64
- #define USE_HW_AES
+
+ #if defined(__INTEL_COMPILER)
+ #if (__INTEL_COMPILER >= 1110)
+ #define USE_HW_AES
+ #if (__INTEL_COMPILER >= 1900)
+ #define USE_HW_VAES
+ #endif
+ #endif
+ #elif defined(Z7_CLANG_VERSION) && (Z7_CLANG_VERSION >= 30800) \
+ || defined(Z7_GCC_VERSION) && (Z7_GCC_VERSION >= 40400)
+ #define USE_HW_AES
+ #if defined(__clang__) && (__clang_major__ >= 8) \
+ || defined(__GNUC__) && (__GNUC__ >= 8)
+ #define USE_HW_VAES
+ #endif
+ #elif defined(_MSC_VER)
+ #define USE_HW_AES
+ #define USE_HW_VAES
+ #endif
+
#elif defined(MY_CPU_ARM_OR_ARM64) && defined(MY_CPU_LE)

#if defined(__ARM_FEATURE_AES) \
@@ -186,15 +205,15 @@
#define SET_AES_FUNC_2(f2) \
if (algo == 2) if (g_Aes_SupportedFunctions_Flags & k_Aes_SupportedFunctions_HW) \
{ f = f2; }
- #ifdef MY_CPU_X86_OR_AMD64
+ #ifdef USE_HW_VAES
#define SET_AES_FUNC_23(f2, f3) \
SET_AES_FUNC_2(f2) \
if (algo == 3) if (g_Aes_SupportedFunctions_Flags & k_Aes_SupportedFunctions_HW_256) \
{ f = f3; }
- #else // MY_CPU_X86_OR_AMD64
+ #else // USE_HW_VAES
#define SET_AES_FUNC_23(f2, f3) \
SET_AES_FUNC_2(f2)
- #endif // MY_CPU_X86_OR_AMD64
+ #endif // USE_HW_VAES
#else // USE_HW_AES
#define SET_AES_FUNC_23(f2, f3)
#endif // USE_HW_AES
14 changes: 14 additions & 0 deletions cross/7zz/patches/003-fix-XzCrc64Opt.patch
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,14 @@
# fix preprocessor commands
# https://sourceforge.net/p/sevenzip/discussion/45797/thread/d9908ffd01/
#
--- C/XzCrc64Opt.c.orig 2023-12-08 08:00:00.000000000 +0000
+++ C/XzCrc64Opt.c 2025-01-03 19:08:34.320848333 +0000
@@ -235,7 +235,7 @@
v = Q32BE(1, w1) ^ Q32BE(0, w0);
v ^= Q32BE(3, d1) ^ Q32BE(2, d0);
#endif
-#elif
+#else
#error Stop_Compiling_Bad_CRC64_NUM_TABLES
#endif
p += Z7_CRC64_NUM_TABLES_USE;
16 changes: 11 additions & 5 deletions cross/coreutils/Makefile
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,17 +1,23 @@
PKG_NAME = coreutils
PKG_VERS = 8.21
PKG_VERS = 9.5
PKG_EXT = tar.xz
PKG_DIST_NAME = $(PKG_NAME)-$(PKG_VERS).$(PKG_EXT)
PKG_DIST_SITE = https://ftp.gnu.org/gnu/$(PKG_NAME)
PKG_DIST_SITE = https://ftp.gnu.org/gnu/coreutils
PKG_DIR = $(PKG_NAME)-$(PKG_VERS)

DEPENDS =

HOMEPAGE = https://www.gnu.org/s/coreutils/
COMMENT = Basic file, shell and text manipulation utilities of the GNU operating system
LICENSE = GPL
COMMENT = Basic file, shell and text manipulation utilities of the GNU operating system. We use 'nice' only.
LICENSE = GPLv3

GNU_CONFIGURE = 1
CONFIGURE_ARGS = --disable-nls

include ../../mk/spksrc.cross-cc.mk

# 32-bit archs require glibc >= 2.34 (i.e. DSM 7.2) to support 64 bit timestamps
ifneq ($(findstring $(ARCH),$(x64_ARCHS)),$(ARCH))
ifeq ($(call version_lt, $(TC_GLIBC), 2.34),1)
CONFIGURE_ARGS += --disable-year2038
endif
endif
Loading
Loading