Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

DS418 - DSM 7.2 last version - don´t work with CONTAINERMANAGER_FOR_ALL_ARMV8 #1

Closed
buenpico opened this issue Feb 22, 2024 · 47 comments

Comments

@buenpico
Copy link

Hello, I have followed the steps and once you send it to install an error appears.

Invalid file format. Contact the package developer.

@kaspar76
Copy link

I have the same problem

@Totix92
Copy link

Totix92 commented Feb 25, 2024

Same problem here

@007revad
Copy link
Owner

I had a quick look at it the other day and could not see any reason for this error. But I can reproduce the error on an x86 NAS so at least I'll know when I've fixed it.

@santaklon
Copy link

Same problem here, hoping you'll be able to fix it!
Thank you so much for creating this in the first place!

@thorbeenwiedemann
Copy link

thorbeenwiedemann commented Mar 10, 2024

Same problem here - any solution so far?

RS819
DSM 7.2.1-69057 Update 3

Invalid file format.

Thanks

@AdrianCiobanu
Copy link

Same problem here:
DS418
DSM 7.2.1-69057 Update 3
"Invalid file format"

@marmurek
Copy link

marmurek commented Mar 29, 2024

@007revad idk if this helps but after install attempt synoscgi.log and synopkgmgr.log says this:

2024-03-29T18:03:36+01:00 synology synoscgi_SYNO.Core.Package.Installation_1_upload[5588]: SYNOPKGMGR: Failed to extract package INFO, from=[/volume1/@tmp/upload_tmp.55880] to=[/volume1/@tmp/synopkg/info.epp9QZ]
2024-03-29T18:03:36+01:00 synology synoscgi_SYNO.Core.Package.Installation_1_upload[5588]: SYNOPKGMGR: Failed to query info
2024/03/29 18:03:36 Failed to extract package INFO, from=[/volume1/@tmp/upload_tmp.55880] to=[/volume1/@tmp/synopkg/info.epp9QZ]
2024/03/29 18:03:36 Failed to query info

EDIT: some people say that many DSM7+ packages may have such error when they need to run as root. I cant find any information inside your spk file if this is the case. I've tried to play with INFO file, even totally replacing it with one that i gather from working spk package and still - file format error occurs. This must be something else.

EDIT2: This repo: https://github.com/hbalint2100/Docker-ARM64-Synology seems to confirm that root things may cause problem, as this starts instalation properly.

Same here:
SynoCommunity/spksrc#5950 (comment)

EDIT3: How did u package this spk file? Did u just use tar+gz or something dedicated?
NVM, simle tar -cvf package.spk * does the trick. Previuosly i was just "taring" using 7zip and file wasnt even extracted to /volume1/@tmp/synopkg

EDIT4: Ok, removing log_collector parameter from INFO file gives different error (no longer invalid format) but this:
obraz

EDIT5: i've added conf/privilege file :

{
	"defaults":{
		"run-as": "package"
	},
	"username": "ContainerManager",
	"join-groupname": "system"
}

And error changed to this:
obraz

I think we should find workaround for "use resource exclusively for ShitSynology"... I'll try going futher later...

@007revad
Copy link
Owner

007revad commented Mar 29, 2024

@marmurek You are awesome!

I'm getting the same log entries. I also came to the conclusion that it's not the INFO file itself but something else.

Docker needing to have root privileges makes sense. DSM 7 makes it hard to install packages that need root privileges and most 3rd party packages that need root privileges instruct the user how to allow root privileges for their package after the package is installed. I believe installing a package with root privileges may be reserved for Synology's own, signed, packages.

Nice find on that other Synology docker repo. I hadn't seen that one.

To create the spk I'm using basically the same as your tar -cvf package.spk *.

In the screenshot of your last error message I see that resource is a link. What does it show when you click on that resource link?

EDIT1 I tried to see that resource error message myself, but because I don't have an armv8 Synology I get "This package is not supported on the platform of Synology NAS or is incompatible with the current DSM version." instead.

EDIT3 I created an x86_64 package, got the same error, did the same steps as you, and got as far as the resource link... which I can now see takes you to https://help.synology.com/developer-guide/resource_acquisition/resources.html

EDIT2 I assume the resource error relates to one of these resources in /config/resource

"webapi-desc"
"web-config"
"sysnotify"
"syslog-config"
"feasibility-config"
"userdata-collector"
"worker-plugin"
"sdk-plugin"
"usr-local-linker"
"synolog"
"synocrond"

@007revad
Copy link
Owner

I just tried installing the package (without the resource file) via SSH as root but got an "invalid package privilege content" error 319

# synopkg install /volume1/test/ContainerManager-x86_64-20.10.23-1437.spk
{
    "error": {
        "code": 0
    },
    "results": [{
            "action": "install",
            "beta": false,
            "betaIncoming": false,
            "error": {
                "code": 319,
                "description": "invalid package privilege content"
            },
            "installReboot": false,
            "installing": true,
            "language": "enu",
            "last_stage": "prepare_install",
            "package": "ContainerManager",
            "packageName": "Container Manager",
            "spk": "/volume1/test/ContainerManager-x86_64-20.10.23-1437.spk",
            "stage": "install_failed",
            "success": false,
            "username": "",
            "version": "20.10.23-1437"
        }
    ],
    "success": false
}

@marmurek
Copy link

marmurek commented Mar 30, 2024

I just woke up and im happy to see you :)

Im not sure this privilege file is valid. I copied it from another package and changed username to match ContainerManager.

I was messing around with that resources file yesterday and seems that almost all of parameters need root.
Leaving only usr-local-linker inside cause error to gone, but I didnt try to install such package as i dont know what can go wrong.
Same as above, with full content resources file, but broken (i left trailling comma) allowed to start instalation. This time, brave me, clicked install but it failed later on "no resources file" :D
Propably parsing json went wrong and synology handles this as "proceed and fail later"...

Edit: I have no idea what we can lose if we remove everything except usr-local-linker, but since it says webapi i assume that configuration via web admin panel will not work.

@007revad
Copy link
Owner

I'll do a hyper backup full system backup of my DS720+ then try installing container manager with just usr-local-linker in the resources file and see what happens.

@marmurek
Copy link

Great! Looking forward to hear from you :)

@007revad
Copy link
Owner

After stripping down resource to just usr-local-linker I now get the "The package is provided by third party developers" warning. It then starts the installation and after a while it gives a Failed to install "Container Manager" error.

The logs show:
Failed to verify package, spk "spk is not from synology"
Failed to run preinst script for ContainerManager
Failed to install package, spk "failed to run preinst script" "failed to locate given package

Full log:
2024/03/31 03:53:20 Failed to verify package, spk=[/volume1/@tmp/upload_tmp.156720] result=[{"action":"prepare","beta":false,"betaIncoming":false,"error":{"code":289,"description":"spk is not from synology"},"installReboot":false,"package":"ContainerManager","packageName":"Container Manager","stage":"prepare","success":false,"version":"20.10.23-1437"}]

2024/03/31 03:53:30 Failed to verify package, spk=[/volume1/@tmp/upload_tmp.156720] result=[{"action":"prepare","beta":false,"betaIncoming":false,"error":{"code":289,"description":"spk is not from synology"},"installReboot":false,"package":"ContainerManager","packageName":"Container Manager","stage":"prepare","success":false,"version":"20.10.23-1437"}]

2024/03/31 03:53:40 Failed to run preinst script for ContainerManager, 1

2024/03/31 03:53:41 Failed to install package, spk=[/volume1/@tmp/upload_tmp.156720] result=[{"action":"install","beta":false,"betaIncoming":false,"error":{"code":276,"description":"failed to run preinst script"},"finished":true,"installReboot":false,"installing":true,"language":"enu","last_stage":"preinstall","package":"ContainerManager","packageName":"Container Manager","pid":17166,"scripts":[{"code":1,"message":"","type":"preinst"}],"spk":"/volume1/@tmp/upload_tmp.156720","stage":"install_failed","status":"non_installed","status_code":255,"status_description":"failed to locate given package","success":false,"username":"Dave","version":"20.10.23-1437"}]

So it looks like once it failed to verify package, spk was from synology it deleted the tmp files but continued trying to install the package.

If I could have got to it to install the next step would have been trying to work out how to use SSH to add the necessary resources after the package was installed.

@marmurek
Copy link

Did u try to install via ssh as root as previously? Since it failed on preinst we should Look what it does - i saw it inside scripts folder

@007revad
Copy link
Owner

007revad commented Mar 30, 2024

Trying to install via SSH gives the errors as the log.

The preinst script checks if /volume#/docker exists, and gets it ACLs

The preinst script creates the /volume#/docker share if it doesn't exist, and creates a log entry if creating the docker share fails. It runs these commands:

  • synoacltool -get <-- this runs without sudo or root
  • synoacltool -add <-- did not check this one
  • synoshare -add <-- this needs sudo or root
  • logger <-- did not check this one

I edited preinst so it only contained:

#!/bin/sh
# Copyright (c) 2000-2015 Synology Inc. All rights reserved.

exit 0

And I was able to install container manager, yay!

Except it shows as:
docker_broken

So I deleted /var/packages/ContainerManager/startFailed and added the following line back to INFO

log_collector="/var/packages/ContainerManager/scripts/synodockerlogcollector.sh"

And Container Manager in Package Center changed from "Repair" to Stopped"... but clicking on Run resulted in same failed to start error with the repair button.

Now I just have to figure out how to use SSH as sudo or root to add the resources I removed from the resource file.

I should also run /var/packages/ContainerManager/scripts/start-stop-status start via SSH to see where it's failing.

@007revad
Copy link
Owner

I should also run /var/packages/ContainerManager/scripts/start-stop-status start via SSH to see where it's failing.

start-stop-status start does output this:

find: `/var/packages/ContainerManager/etc/projects': No such file or directory

But it continues after that. And my DS1821+ does not have /var/packages/ContainerManager/etc/projects either and docker runs just fine on it.

Apart from that it appears as if docker has started.

# /var/packages/ContainerManager/scripts/start-stop-status start
[pkg-ContainerManager-event-watcherd] started.
Sun Mar 31 06:30:20 AM AEDT 2024: start_docker_daemon: try start docker daemon
Sun Mar 31 06:30:20 AM AEDT 2024: start_docker_daemon: start daemon..
[pkg-ContainerManager-dockerd] started.
Sun Mar 31 06:30:28 AM AEDT 2024: start_docker_daemon: daemon started. start to wait for daemon ready
Sun Mar 31 06:30:28 AM AEDT 2024: start_docker_daemon: daemon is ready
find: `/var/packages/ContainerManager/etc/projects': No such file or directory
[pkg-ContainerManager-termd] started.

@marmurek
Copy link

Wow, awesome findings! Grats!

Propably of the script create missing directories, maybe we can add them later, after instalation manually using ssh?

Does container manager expose configuration app into dsm website? Does it work?

@007revad
Copy link
Owner

The Container Manager package is not running, so there's nothing in the DSM UI, apart from the damaged package that package center wants to repair.

Via SSH, docker itself appears to be running.

root@SENNA:~# docker ps
CONTAINER ID   IMAGE     COMMAND   CREATED   STATUS    PORTS     NAMES

But my 3 containers that were leftover from when I uninstalled the real DS720+ Container Manager version are not running.

root@SENNA:~# docker info
Client:
 Context:    default
 Debug Mode: false

Server:
 Containers: 3
  Running: 0
  Paused: 0
  Stopped: 3
 Images: 2
 Server Version: 20.10.23
 Storage Driver: btrfs
  Build Version: Btrfs v4.0
  Library Version: 101
 Logging Driver: db
 Cgroup Driver: cgroupfs
 Cgroup Version: 1
 Plugins:
  Volume: local
  Network: bridge host ipvlan macvlan null overlay
  Log: awslogs db fluentd gcplogs gelf journald json-file local logentries splunk syslog
 Swarm: inactive
 Runtimes: io.containerd.runc.v2 io.containerd.runtime.v1.linux runc
 Default Runtime: runc
 Init Binary: docker-init
 containerd version: b23a389d8c181697302d163356e97dec04eb8d88
 runc version: 5af893d
 init version: ed96d00
 Security Options:
  apparmor
 Kernel Version: 4.4.302+
 Operating System: Synology NAS

 OSType: linux
 Architecture: x86_64
 CPUs: 4
 Total Memory: 5.641GiB
 Name: SENNA
 ID: SCHB:FSVY:WIHU:RYUZ:PRYS:7NM2:EGA7:5SBH:3YVP:M7RT:TAHN:T22S
 Docker Root Dir: /volume1/@docker
 Debug Mode: false
 Registry: https://index.docker.io/v1/
 Labels:
 Experimental: false
 Insecure Registries:
  127.0.0.0/8
 Live Restore Enabled: false

WARNING: No kernel memory TCP limit support
WARNING: No cpu cfs quota support
WARNING: No cpu cfs period support
WARNING: No blkio throttle.read_bps_device support
WARNING: No blkio throttle.write_bps_device support
WARNING: No blkio throttle.read_iops_device support
WARNING: No blkio throttle.write_iops_device support

NOTE Those 7 warnings are normal on a Synology NAS.

But systemctl commands say it's not running:

root@SENNA:~# systemctl status docker
● docker.service
   Loaded: not-found (Reason: No such file or directory)
   Active: inactive (dead)

root@SENNA:~# synosystemctl stop docker
Fail to stop [docker].

root@SENNA:~# systemctl status docker
● docker.service
   Loaded: not-found (Reason: No such file or directory)
   Active: inactive (dead)

root@SENNA:~# systemctl start docker
Failed to start docker.service: Unit docker.service failed to load: No such file or directory.

@007revad
Copy link
Owner

I just remembered that a few months ago I managed to install the DVA1622's Surveillance Station with AI Deep Learning on my DS720+. Maybe what I did then would work for Container Manager.

Can you try this:

  1. Note the output of synogetkeyvalue /etc.defaults/synoinfo.conf unique because you'll need it in step 8.
  2. sudo synosetkeyvalue /etc/synoinfo.conf unique synology_rtd1619b_ds423
  3. sudo synosetkeyvalue /etc.defaults/synoinfo.conf unique synology_rtd1619b_ds423
  4. Download https://global.synologydownload.com/download/Package/spk/ContainerManager/20.10.23-1437/ContainerManager-armv8-20.10.23-1437.spk
  5. Do a Manual Install in Package Center of the .spk file you downloaded.
  6. Stop Container Manager.
  7. Edit /var/packages/ContainerManager/INFO to delete the line that starts with exclude_model=
  8. Repeat steps 2 and 3 but replace synology_rtd1619b_ds423 with the output from step 1.
  9. Start Container Manager and check if it's working.
  10. If it is working prevent Container Manager from auto updating: "Package Center > Settings > Auto-update > select "Only packages below" > Untick Container Manager"

If that works I'll write a script that does steps 1 to 9 to make it easy for other people.

@marmurek
Copy link

Ok, i will try after i return to home from easter vacation :) very interesting method!

@007revad
Copy link
Owner

007revad commented Mar 31, 2024

@baoshenti @buenpico @kaspar76 @Totix92 @santaklon @thorbeenwiedemann @AdrianCiobanu @marmurek

I've created a pre-release script that downloads and installs Container Manager and instructs you how to prevent Container Manager from updating. https://github.com/007revad/ContainerManager_for_all_armv8/releases/tag/v1.0.2

https://github.com/007revad/ContainerManager_for_all_armv8/releases/tag/v1.2.4

I've tested as much as I can on my DS720+. It should work on a RS819, DS418, DS418j, DS218, DS218play, DS118 (and maybe DS119j).

@AdrianCiobanu
Copy link

@007revad : Thank you 👍 . I think I get an error. Can you guide me through this?

image

@007revad
Copy link
Owner

007revad commented Mar 31, 2024

@AdrianCiobanu
Copy link

image

@007revad
Copy link
Owner

Sadly it looks like something in the container manager package is still stopping it from installing on a DS418.

I just realized that because the script failed, it never restored your model number. Can you run the following 2 commands to fix it.

sudo synosetkeyvalue /etc/synoinfo.conf unique synology_rtd16296_ds418
sudo synosetkeyvalue /etc.defaults/synoinfo.conf unique synology_rtd16296_ds418

Can you run the following command and report back what it returns:

cat /var/log/synopkgmgr.log | grep ContainerManager

@marmurek
Copy link

marmurek commented Apr 2, 2024

@007revad Big thanks for help! I can confirm it worked for my DS218 @ DSM 7.2.1-69057 Update 3 ! :)

@007revad
Copy link
Owner

007revad commented Apr 2, 2024

@AdrianCiobanu
Copy link

Sadly it looks like something in the container manager package is still stopping it from installing on a DS418.

I just realized that because the script failed, it never restored your model number. Can you run the following 2 commands to fix it.

sudo synosetkeyvalue /etc/synoinfo.conf unique synology_rtd16296_ds418
sudo synosetkeyvalue /etc.defaults/synoinfo.conf unique synology_rtd16296_ds418

Can you run the following command and report back what it returns:

cat /var/log/synopkgmgr.log | grep ContainerManager

On sunday, after running the codes from above all my apps were unavailable. I have restarted the system and it required a DSM reinstall. After 10 min I was back again.
I have tried now again v1.2.4 and here is the result:

image

@marmurek
Copy link

marmurek commented Apr 2, 2024

@007revad i used the script

@007revad
Copy link
Owner

007revad commented Apr 4, 2024

@AdrianCiobanu Is your Synology actually a DS418?

Or did it only say it was a DS418 after running the older script version that failed with an error?

And what does the following command return?

uname -a

@AdrianCiobanu
Copy link

@AdrianCiobanu Is your Synology actually a DS418?

Or did it only say it was a DS418 after running the older script version that failed with an error?

And what does the following command return?


uname -a

I'm 100% sure that it is DS418
image

image

@pericary
Copy link

pericary commented Apr 6, 2024

Hey 007revad,

I tried to run your newest version of the script today on my rs819, but I got this error message:

image

I followed all the steps from your readme, but now I am stuck. Any clue what could be the problem?

Thanks!

@007revad
Copy link
Owner

007revad commented Apr 6, 2024

@pericary

What does the following command return?

sudo grep Failed /var/log/synopkgmgr.log

@kongwoojin
Copy link

Hello @007revad
I tried your script on my DS118 running DSM 7.2.1-69057, and I faced same error with this.

But the 10 steps that you posted here are work.

Thanks!

@007revad
Copy link
Owner

007revad commented Apr 7, 2024

@kongwoojin

The only difference between the script and those 10 steps are steps 4 and 5,

Are you willing to uninstall Container Manager then repeat the 10 steps except replace steps 4 and 5 with:

sudo synopkg install_from_server ContainerManager /volume1

And let me know if works.

@kongwoojin
Copy link

@kongwoojin

The only difference between the script and those 10 steps are steps 4 and 5,

Are you willing to uninstall Container Manager then repeat the 10 steps except replace steps 4 and 5 with:

sudo synopkg install_from_server ContainerManager /volume1

And let me know if works.

It not works and error message is

Failed to query package list from server

@007revad
Copy link
Owner

007revad commented Apr 7, 2024

Failed to query package list from server

That would explain why the script is not working for many people.

Maybe the synopkg install_from_server command uses the Synology CPU arch from uname -m.

I'll change the script to download the armv8 .spk then use synopkg install ContainerManager-armv8-20.10.23-1437.spk. The only downside is we can't select which volume it installs on.

@marmurek
Copy link

marmurek commented Apr 7, 2024

Perhaps u can catch that error and fallback to instalation using file if user wanted

@007revad
Copy link
Owner

007revad commented Apr 7, 2024

Perhaps u can catch that error and fallback to instalation using file if user wanted

Good idea.

@pericary
Copy link

pericary commented Apr 7, 2024

@pericary

What does the following command return?

sudo grep Failed /var/log/synopkgmgr.log

It's interesting, the failed installation is not listed in the log file.

@pericary
Copy link

pericary commented Apr 7, 2024

I now installed the Container Manager acc. to the 10 step procedure and it works!

Unfortunately now some other packages (e.g. synology photos and storage manager) now need a repair but won't update when I click "repair". The description of the failure is "package does not support the current arch/model". But I changed the unique name back to my rs819.

@007revad
Copy link
Owner

007revad commented Apr 7, 2024

It's interesting, the failed installation is not listed in the log file.

That could be because synopkg failed to query package list from server.

@007revad
Copy link
Owner

007revad commented Apr 7, 2024

Unfortunately now some other packages (e.g. synology photos and storage manager) now need a repair but won't update when I click "repair". The description of the failure is "package does not support the current arch/model". But I changed the unique name back to my rs819.

Did you change it back in both synoinfo.conf files?

sudo synosetkeyvalue /etc/synoinfo.conf unique synology_rtd196_rs819
sudo synosetkeyvalue /etc.defaults/synoinfo.conf unique synology_rtd196_rs819

Then check with:

synogetkeyvalue /etc/synoinfo.conf unique
synogetkeyvalue /etc.defaults/synoinfo.conf unique

Also try closing and re-opening Package Center.

@pericary
Copy link

pericary commented Apr 7, 2024

Well, I guess the closing and re-opening did the job :-D

Now everything is working just fine! Thank you very much for your support!!

@007revad
Copy link
Owner

007revad commented Apr 8, 2024

New version of the script: https://github.com/007revad/ContainerManager_for_all_armv8/releases

Changes:

  • If Container Manager failed to install due to "Failed to query package list from server" error:
    • Script now pauses so you can do a manual package install.
    • Explains how to do a manual package install.
  • Added message to close and re-open Package Center if you have it open.
  • Backs up synoinfo.conf in case the original unique string is needed.
  • Save the output from synopkg to /tmp/installcm.txt for debugging if needed.
  • Minor improvements.

@AdrianCiobanu
Copy link

I can now confirm that it worked for me also! @007revad: Thank you very much for your patience and support!
https://github.com/007revad/ContainerManager_for_all_armv8/releases/tag/v1.3.8
DS418 - DSM 7.2.1-69057 Update 4 - [ContainerManager-armv8-20.10.23-1437.spk]

image

@CorentinWicht
Copy link

Hi everyone, I also confirm that the script is working smoothly:

XXX@MediaServer:~/ContainerManager_for_all_armv8-1.3.8$ sudo -s sh install_container_manager.sh
ContainerManager_for_all_armv8 v1.3.8
DS118 DSM 7.2.1-69057-1
CPU is aarch64
synology_rtd1296_ds118

ContainerManager is not installed

Backed up synoinfo.conf

Editing synoinfo.conf

Installing ContainerManager on /volume1
Stopping Container Manager
Editing ContainerManager INFO

Restoring synoinfo.conf

Starting Container Manager

Finished

If Package Center is already open close it and re-open it.

You need to prevent Container Manager from auto updating:
  1. Go to 'Package Center > Settings > Auto-update'
  2. Select 'Only packages below'
  3. Untick Container Manager
  4. Click OK

image

image

Thanks for your work!

Best,

C.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests