Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

feat: two apache for off #399

Open
wants to merge 1 commit into
base: develop
Choose a base branch
from
Open

feat: two apache for off #399

Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
65 changes: 65 additions & 0 deletions docs/reports/2024-09-02-off-server-double-apache.md
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,65 @@
# 2024-09-02 OFF server double apache

## Goal

We experience latency problems on Open Food Facts because the Apache instance is busy processing long requests (facets / searches / etc.).

We can't prioritize requests based on URL in Apache2 (or dedicate workers).

We will setup a second Apache instance which will only serve certain requests:

* Product read (api or not)
* root page for every country
* product writes (api or not)

The rest of the requests will be handled by the other apache2 server.

## Reflexion on how to setup the new apache2 instance

On debian, apache2 is managed by systemd. There is:
* a default `apache2.service` service definition, using /etc/apache2/ configuration directory
* and an `apache2@<instance>.service` definition which use /etc/apache2.%i/ configuration directory (where %i is the instance name)

Both use the apache2ctl script to start apache2.
So we can use APACHE_ARGUMENTS to add arguments to httpd daemon program,
and this can be used to add -D arguments to add variables.

Here we want to create a second apache2 instance where the only differences are:
* the port apache2 is listening on
* the log file names

For the log file names, we will modify startup_apache2.pl to use environment variable to get the log configuration file.

For ports, we need to modify ports.conf file to use a variable that we will give thanks to a -D option to apache2 with APACHE_ARGUMENTS variable.

To be more consistent, we will drop the `apache2.service` instance and use two new instances:
* [email protected] - for product read, root pages and product writes
* [email protected] - for the rest

## Doing it in Product-Opener

See https://github.com/openfoodfacts/openfoodfacts-server/pull/10766

## Installation / Migration

1. checkout the new release / code
2. symlink /srv/$SERVER_NAME/conf/systemd/[email protected] to /etc/systemd/system/[email protected]
2. enable the [email protected] [email protected]
2. start [email protected]
2. and test it's working using `curl http://127.0.0.1:8005/ -H "Host: world.openfoodfacts.org"`
2. check nginx configuration is ok (`nginx -t`) and restart the service
3. check both apache2 are working:
* `curl http://127.0.0.1/ -H "Host: world.openfoodfacts.org"`
* `curl http://127.0.0.1/discover -H "Host: world.openfoodfacts.org"`
and using your browser
2. stop apache2.service
2. start [email protected]
3. test it's working using curl commands above and using your browser
1. deactivate apache2.service
1. unlink the /etc/systemd/system/apache2.service
1. unlink /srv/$SERVER_NAME/log.conf

Celebrate !
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm a bit worried about memory usage if we get a lot of traffic and we go to the limits set in mods-enabled/mpm_prefork.conf:

<IfModule mpm_prefork_module>
        StartServers                     5
        MinSpareServers           5
        MaxSpareServers          10
        MaxRequestWorkers         50
        ServerLimit             55
        MaxConnectionsPerChild   100
</IfModule>

If we have 2 Apache servers, then maybe we should halve the max processes they can have.


**FIXME:** modify doc explaining off installation
**FIXME:** add the deployment documentation