502 bad gateway #1430
Comments
|
Hey @anuj-kumar1SF! 👋 Thanks for flagging this bug! 🐛🔍 You're our superhero bug hunter! 🦸♂️🦸♀️ Before we suit up to squash this bug, could you please: 📚 Double-check our documentation: https://rengine.wiki Once again - thanks for your vigilance! 🛠️🚀 |
|
During setup either the rengine user is not being created in postgresql or it's password is not set properly Here's the error output: Traceback (most recent call last):
File "/usr/src/app/manage.py", line 26, in <module>
main()
File "/usr/src/app/manage.py", line 22, in main
execute_from_command_line(sys.argv)
File "/usr/local/lib/python3.10/dist-packages/django/core/management/__init__.py", line 419, in execute_from_command_line
utility.execute()
File "/usr/local/lib/python3.10/dist-packages/django/core/management/__init__.py", line 413, in execute
self.fetch_command(subcommand).run_from_argv(self.argv)
File "/usr/local/lib/python3.10/dist-packages/django/core/management/base.py", line 354, in run_from_argv
self.execute(*args, **cmd_options)
File "/usr/local/lib/python3.10/dist-packages/django/core/management/base.py", line 398, in execute
output = self.handle(*args, **options)
File "/usr/local/lib/python3.10/dist-packages/django/core/management/base.py", line 89, in wrapped
res = handle_func(*args, **kwargs)
File "/usr/local/lib/python3.10/dist-packages/django/core/management/commands/migrate.py", line 75, in handle
self.check(databases=[database])
File "/usr/local/lib/python3.10/dist-packages/django/core/management/base.py", line 419, in check
all_issues = checks.run_checks(
File "/usr/local/lib/python3.10/dist-packages/django/core/checks/registry.py", line 76, in run_checks
new_errors = check(app_configs=app_configs, databases=databases)
File "/usr/local/lib/python3.10/dist-packages/django/core/checks/model_checks.py", line 34, in check_all_models
errors.extend(model.check(**kwargs))
File "/usr/local/lib/python3.10/dist-packages/django/db/models/base.py", line 1303, in check
*cls._check_indexes(databases),
File "/usr/local/lib/python3.10/dist-packages/django/db/models/base.py", line 1695, in _check_indexes
connection.features.supports_covering_indexes or
File "/usr/local/lib/python3.10/dist-packages/django/utils/functional.py", line 48, in __get__
res = instance.__dict__[self.name] = self.func(instance)
File "/usr/local/lib/python3.10/dist-packages/django/db/backends/postgresql/features.py", line 92, in is_postgresql_11
return self.connection.pg_version >= 110000
File "/usr/local/lib/python3.10/dist-packages/django/utils/functional.py", line 48, in __get__
res = instance.__dict__[self.name] = self.func(instance)
File "/usr/local/lib/python3.10/dist-packages/django/db/backends/postgresql/base.py", line 329, in pg_version
with self.temporary_connection():
File "/usr/lib/python3.10/contextlib.py", line 135, in __enter__
return next(self.gen)
File "/usr/local/lib/python3.10/dist-packages/django/db/backends/base/base.py", line 603, in temporary_connection
with self.cursor() as cursor:
File "/usr/local/lib/python3.10/dist-packages/django/utils/asyncio.py", line 33, in inner
return func(*args, **kwargs)
File "/usr/local/lib/python3.10/dist-packages/django/db/backends/base/base.py", line 259, in cursor
return self._cursor()
File "/usr/local/lib/python3.10/dist-packages/django/db/backends/base/base.py", line 235, in _cursor
self.ensure_connection()
File "/usr/local/lib/python3.10/dist-packages/django/utils/asyncio.py", line 33, in inner
return func(*args, **kwargs)
File "/usr/local/lib/python3.10/dist-packages/django/db/backends/base/base.py", line 218, in ensure_connection
with self.wrap_database_errors:
File "/usr/local/lib/python3.10/dist-packages/django/db/utils.py", line 90, in __exit__
raise dj_exc_value.with_traceback(traceback) from exc_value
File "/usr/local/lib/python3.10/dist-packages/django/db/backends/base/base.py", line 219, in ensure_connection
self.connect()
File "/usr/local/lib/python3.10/dist-packages/django/utils/asyncio.py", line 33, in inner
return func(*args, **kwargs)
File "/usr/local/lib/python3.10/dist-packages/django/db/backends/base/base.py", line 200, in connect
self.connection = self.get_new_connection(conn_params)
File "/usr/local/lib/python3.10/dist-packages/django/utils/asyncio.py", line 33, in inner
return func(*args, **kwargs)
File "/usr/local/lib/python3.10/dist-packages/django/db/backends/postgresql/base.py", line 187, in get_new_connection
connection = Database.connect(**conn_params)
File "/usr/local/lib/python3.10/dist-packages/psycopg2/__init__.py", line 122, in connect
conn = _connect(dsn, connection_factory=connection_factory, **kwasync)
django.db.utils.OperationalError: connection to server at "db" (172.19.0.2), port 5432 failed: FATAL: password authentication failed for user "rengine" |
|
Hi @anuj-kumar1SF @its-mr-monday If both of your problems are related, I can see password auth failed. Do you mind checking your .env file? the postgres password should be there. I believe you would have changed it earlier during setup, check it and let me know if that's changed now. We dont have any change on the release. Also can you tell me how did you update reNgine? |
|
Running into the same issue updated from 2.1.3 to 2.2.0 using the update.sh, make build make up process. |
|
Hi @justin-davis-uplight sorry for the inconvenience. I looked at the code and see that somebody had sent this PR for update.sh earlier and has stashed the .env changes that's why we arent able to connect to postgres with latest changes. I will fix that in next release by creating a backup of .env file. For now lets focus on fixing this temporarily. Can you do |
|
My .env looks correct, getting http 500 now. I did a make down and make up |
|
Getting same issue , Server Error (500), getting this for sudo make logs |
|
I was able to manually set the psql password but now I am getting the same Server Error (500) page along with the same logs as above |
|
It's because the migrations are not being made properly for existing installs. @yogeshojha every time you make a major upgrade (same thing happened from 2.0.7 -->2.10) and now again to 2.2.0, you merge all your migrations into a file called 0001_initial.py. This breaks any existing install. That means we need to do a clean install anytime you do that. Unless we do the changes manually to the db. If you do not want to break existing installs, you need to leave the migrations intact or else any change to the db you make will break existing installs. In this particular scenario, it has to do with the user preferences in the dashboard. And I am sure once this is fixed, others will pop up. |
|
@ncharron I wasnt aware of this. Yes I did have to clean up the 0001_initial.py And maybe that is why I didn't notice this breaking changes on my local instances. For future releases, I will take that in mind. can you help with intermediate solution? I am trying to write script that does create the missing table, other than that if you can offer some help would appreciate. |
|
Yeah absolutely! Well at the moment I would have to find all the differences between the last db and this one. It will be on a case by case basis to be honest depending from which point a person is upgrading. Let me see what I can do from my end, it would have to be, revert back to where I was for me, import the new models, let django figure out the new migrations, take those migration files and import them into 2.2.0. |
|
Thanks @ncharron! I will do the fresh 2.1.3 install myself and try to do the update let me see if I can find the differences in migrations. For this specific case, do you think if I create a script to populate the db manually will it work? |
|
Yes, you could create a script that would do the changes manually and then it should be fine. However depending on the relationships, you might run into issues with that. But it is worth a shot for now! |
|
Thank you for the heads up, I will try to fix this temporarily now, and in the future, as you said will not touch the initial migrations. |
|
So quick way to to do this: Take the 0001_initial.py files from the 2.1.0 repo and put it in a 2.2.0 repo. Then make up on the 2.2.0, django should automatically create all the migrations you need. Currently trying to test it but have data so it's getting tricky :) |
|
@ncharron I have been doing the same thing, I am manually putting back 0001_initial from 2.1.3 and also creating 0002_initial based on the new model |
|
I will be pushing the changes to this branch: https://github.com/yogeshojha/rengine/tree/hotfix/2.2.0_release and will let you know if/when I make enough changes so that some one could try it out. |
|
So we did manage to fix it using that method. We just had to make a few tweaks on our end because we have a custom model. But in essence, replacing the 0001_initial.py with the previous version, then running makemigrations and the migrate, fixed the issue for the db. |
|
@ncharron Did you test the branch I pushed above? or are you sending PR? |
|
We haven't been able to test that fix exactly because ours is slightly different because I was still at 2.1.0. But that is the same steps I took to make it work. If someone is coming from 2.1.3, That should resolve the issue AS LONG as the db is still in the 2.1.3 format. |
|
Hi @ncharron @its-mr-monday @vipul-soman @justin-davis-uplight @anuj-kumar1SF I have fixed this issue in this branch https://github.com/yogeshojha/rengine/tree/hotfix/2.2.0_release I have tested it out, upgrading from 2.1.3 to 2.2.0, and it works fine, with no migration issues. I'm sorry for this silly mistake. @ncharron Thank you for letting me know about the 0001_initial migration. There's always something new to learn, and I hope to rectify this in future releases. Steps to test this hotfix and do as usual Please somebody confirm if it is working and I will merge it Thanks |
|
@yogeshojha I can confirm it is working. |
|
@sanyam2602 Thank you for testing it out. I have merged the changes. please Thanks @ncharron again for the help. Much appreciated I will keep this issue open for some time to continue the conversation |
|
No problem! To clarify for anyone out there if it wasn't clear what was happening: After every release, the database changes were all merged into 1 file called 0001_initial.py for the different parts of the app. These are created automatically by django based on your model files. These files (0001_initial.py) are found in the migrations folder under different parts of the app. And this was easily done by simply deleting all the files in the migrations folder and letting the app make a clean version of the migrations. These files are automatically created to note the changes between your Models files and the database structure every time you run the command manage.py makemigrations from the web docker container. This is useful when developping because if ever you create a new Model for the db or makes changes to a model and therefore to the db, they are automatically picked up and then you can apply them by doing manage.py migrate from the web docker container. These commands are mostly run automatically based on the entrypoint.sh files for the docker containers every time you start them. So every time you restart your containers it automatically picks up changes to the files, therefore the db and applies them. And every time you make changes to the models, a new SEQUENTIAL file is created based on those changes to tell the app in what order to apply the migrations. Also all db's managed by django using this method keeps track of which migration you are landed at so it doesn't try to apply all the changes all the time. You typically end up with different files called 0001_initial.py, 0002_comment_about_changes.py, 0003_something.py, etc.... Now this gets messy after a while but if ever you get rid of all of those and make a 0001_initial.py again, unless that database structure is already where it is supposed to be at, your db will think you are at migration 0003 when all there is in the new release is 0001_initial.py. And so it will not pick all the most recent changes made to the db. Now the app will work as intended until you run into one of the new models or differences in the structure and it tries to call those columns/tables that do not exist. That's how we get the 500 error. In this case it was the user_preferences in the dashboard section of the app. TL:DR; If you remove all the migrations from the app to make it cleaner, your old db will not know about the changes being made to the code and will crash when trying to fetch something that doesn't exist. This is ONLY a problem for existing installations and not for new installations. Hope this helps! |
|
It's working out of the box now with all previous data intact |
|
I think I had my directory content messed up. I ended up backing up my rengine dir, did a fresh clone, copied my .env file and the secrets dir and ran the update.sh script again |
|
@ncharron Once again thank you for the detailed explanation and a lesson for me to never touch 0001_initial and further migration files ;p |
|
I was also getting bad gateway. Now i wouldnt apply the hotfix. Below is the output git fetch && git checkout hotfix/2.2.0_release error: pathspec 'hotfix/2.2.0_release' did not match any file(s) known to git |
|
❯ git pull
❯ git pull but i am still getting 502 bad gateway. |
|
Okay i had the problem after automatically running in installation process. I manually sudo make down && sudo make up and it works now |
|
Yeah so to be clear your issue was this line: That happens when there is a docker network problem and you have to do sudo make down (to delete the network) and then sudo make up (or just sudo docker compose up -d since sudo make up rebuilds the entire containers) to bring back up the networks and the containers will be able to speak to each other again. |
|
Same error on a fresh install (tested on: ubuntu 20.04, 22.04, debian 11 and debian 12). Funfact. This morning it was working fine.. now it fails. COMPOSE_DOCKER_CLI_BUILD=1 docker compose -f docker-compose.yml exec web python3 manage.py createsuperuser pikepdf._core | pikepdf C++ to Python logger bridge initialized pikepdf._core | pikepdf C++ to Python logger bridge initialized Thank you for installing reNgine, happy recon!! |
and others
Is there an existing issue for this?
Current Behavior
Rengine was working fine 3 days ago from raising this bug. 2 days ago I updated the tool and since then I'm facing 502 bad gateway. My kali linux is already up to date and have all the dependencies required to run this tool. I've look for any solution but nothing is working. I've tried reinstalling it but still no luck. Please somebody help.
Expected Behavior
It should have worked correctly.
Steps To Reproduce
Install the tool
open browser
localhost or 127.0.0.1
Environment
Anything else?
The text was updated successfully, but these errors were encountered: