VM System Upgrade
Steps & Timeline
-
Get swap-over VM from Eckard -
Install latest Ubuntu server LTS (https://ubuntu.com/download/server) -
Install and configure Postgres 15 -
Import nacsos2 database
-
-
Install and configure Postgres 12 -
Import nacsos (OG) database
-
-
Make nacsos (OG) run as systemd service (digital-ocean guide) -
Transfer climate-health-app and make it run as systemd service on new vm (+ update nginx) -
Transfer GitLab runners and nacsos2 services -
Install and configure nginx -
set up proxies for nacsos (OG), climate-map -
set up proxies for nacsos-core, nacsos-pipes, -
set up static file serving (landing page, nacsos-docs, nacsos-web) -
set up apsis group page with quicklinks at root
-
-
Transfer solr directory and start service on new VM (moving to new machine for this deferred for now) -
Transfer cdr-landscape service -
Install and set up jupyter-hub service -
Verify all services are running properly -
Transfer home directories data -
Set up weekly cron for database dump backups -
Shut down old VM
We will inform users shortly ahead of time and make the server inaccessible for a week (or so). In this way, we don't need to worry about uptime and hot-swapping live backups.
Side projects
- Update nacsos (OG) dependencies, especially celery and database adapters
- Reduce nacsos (OG) dependencies (there are some problematic postgres GIS plugins...)
- ...
Background
Our server doesn't work very well, we get strange errors about moving to a bigger server, but we don't know where they come from.
@mhansen @timrepke , we discussed today that upgrading our OS may help. In fact, starting from a fresh install and installing things in a sensible way could be a good opportunity. We need to upgrade this year in any case.
Here is a list of the services we run on srv-mcc-apsis, with some steps for how to transfer these to an upgraded server. Please expand
Postgres12 (725 GB)
This is the database behind Nacsos1. It is supposed to be backed up in /usr/local/apsis/db_backup/, although these folders are empty. Perhaps the backup script failed because of lack of available space? We should probably check this
Nacsos Django application
This runs on python3.6. We will need to do some careful upgrading to get it to work with newer versions of python. This includes web serving via apache server, celery for distributed tasks, scrapers, etc.
Jupyterhub server
We have a jupyterhub server open to internal use
Rstudio server
We have the same for Rstudio, but I don't think anyone uses it.... Maybe Will does?
Other
We have an instance of monit running to check resource use and warn about server overload. We also run some recurring tasks like database backup dumps etc. through cron.
...