QFQ

{{child_pages}}

{{toc}}

h1. lean: OS Update U18>U22, T3 V9>V11, PHP 7.2>8.1, MariaDB 10.1>10.6

| Backup / Datenaustausch | @[tl] /scratch/tmp/30/lean/@ OS:lean.system.tgz, DB:*.sql, html, dev |

| Lokaler Dump | @[lean] /var/tmp/dev@ |

| Installtool | https://lean.geo.uzh.ch/typo3/install.php |

| MariaDB | Update 10.1 > 10.6 - all columns need a default value if INSERT does not specify the column |

* The use of dump files doesn't include sql functions. Exporting them with phpMyAdmin from source to target is currently the best solution. In most cases not necessary because upgrade happens on an existing instance with given databases.

* Sometimes T3 Cache could make problems while upgrading. Deleting the typo3temp folder helps.

* Using relative paths in typo3 template editor could make problems in v11. It helps to change them to absolute paths. Good example for these situations are the use of 'cd.stylesheet =' options.

h2. Konsole

* Via GEO IT

h2. Preparation on production

<pre>

# Check das der neueste Kernel aktiv ist, falls nicht reboot

$ uname -a

$ dpkg --list | grep linux

#

# Alle alten Kernel entfernen! Das ist wichtig damit der do-release-upgrade Process nicht unnoetig alte Kernel Module aktualisiert.

#

$ CURRENT=`uname -a | cut -f 3 -d ' '`

$ ALL=`dpkg --list | grep -e linux-modules -e linux-image | grep -v $CURRENT | awk '{ print $2 }'`

$ for II in $ALL; do apt purge -y $II; done

# Backup Dir anlegen:

[user@tlX] mkdir /scratch/tmp/30/lean

# Latest T3 nach Production kopieren

[root@dev] cd /var/www; tar zcf /var/tmp/lean.typo3_src-11.5.25.tgz  typo3_src-11.5.25; scp /var/tmp/lean.typo3_src-11.5.25.tgz crose@ssh:/scratch/tmp/30/lean/

[user@tlX] scp /scratch/tmp/30/lean/lean.typo3_src-11.5.25.tgz lean.geo.uzh.ch:/var/tmp/

[user@tlX] scp /scratch/tmp/30/lean/lean.typo3_src-11.5.25.tgz lean-gate.geo.uzh.ch:/var/tmp/

# T3 schonmal auspacken

[root@lean] cd /var/html; tar zxf typo3_src-11.5.25.tgz

[root@lean-gate] cd /var/html; tar zxf typo3_src-11.5.25.tgz

# ** Am Tag vor der Migration **

# Backup anlegen. Zuerst auf TL: `mkdir /scratch/tmp/30/lean`

# 

[root@lean]

$ cd /

$ tar --exclude=var/local/prod --exclude=var/www/html -czvf /tmp/lean.system.tgz bin boot etc home initrd.img lib lib64 opt root sbin snap srv swap.img usr var vmlinuz

$ scp /tmp/lean.tgz crose@ssh.math.uzh.ch:/scratch/tmp/30/lean/

$ rsync -av /var/www/html crose@ssh.math.uzh.ch:/scratch/tmp/30/lean/

# 

# ** Am Tag der Migration **

#

# Apache anhalten

$ service apache2 stop

# Alle Cronjobs anhalten:

$ crontab -e

$ crontab -e -u www-data

# DBs - einmal lokal ablegen und dann nochmal remote sichern

$ mkdir /var/tmp/prod; cd /var/tmp/prod

$ rm *.sql

$ ALL="lean_gate_prod_t3 lean_preview_db lean_preview_t3 lean_prod_db lean_prod_t3 mysql"

$ for II in $ALL; do mysqldump $II > $II.sql ; done

$ mysqldump --no-create-info --ignore-columns=lockToDomain lean_prod_t3 fe_users > lean_prod_t3.fe_users.sql

$ scp *.sql crose@ssh.math.uzh.ch:/scratch/tmp/30/lean/

# 

# OS Migration: 

# 

# Ubuntu 20

$ do-release-upgrade 

# Login lean(u20) und Apache direkt wieder anhalten:

$ service apache2 stop

# Ubuntu 22

$ do-release-upgrade 

# Login lean(u22) und Apache direkt wieder anhalten:

$ service apache2 stop

# Check welche PHP Version installiert. Ggfs. sind noch alte installiert. Am besten vergleichen ob alle PHP Pakete von der alten Version auch in der neuen installiert sind. Bei dieser Migration fehlten nach der Migration @php-xml@, @php-json@.

$ dpkg --list | grep php

# Pakete

$ apt install wkhtmltopdf

</pre>

h2. Preparation on dev (webwork22)

<pre>

# Create dump of typo3 database:

$ mysqldump lean_dev_t3 > /var/tmp/lean_dev_t3_v11.sql

# Filestruktur 

$ cd /var/www/html

$ tar zcf /var/tmp/lean_dev_v11.tgz lean

</pre>

h2. File transfer: Dev to Production

Jump host TL

<pre>

[crose@tlX]

$ scp root@webwork22:/var/tmp/lean_dev_t3_v11.sql /scratch/tmp/30/lean/dev/

$ scp root@webwork22:/var/tmp/lean_dev_v11.tgz /scratch/tmp/30/lean/dev/

$ rsync -av  /scratch/tmp/30/lean/dev root@lean.math.uzh.ch:/var/tmp/

</pre>

h2. Setup Production

<pre>

[root@lean]

$ cd /var/www/

$ mv html html.v9

# Unpack page instance for prod and preview. Rename them to original name and check the right permissions:

# Prod

$ cd /var/www/

$ tar -xzf /var/tmp/dev/lean_dev_v11.tgz

$ mv lean html

# Preview

$ tar -xzf /var/tmp/dev/lean_dev_v11.tgz

$ mv lean html/preview

# Import T3 DBs (Prod & Peview same)

$ mysql lean_prod_t3 < /var/tmp/dev/lean_dev_t3_v11.sql

$ mysql lean_preview_t3 < /var/tmp/dev/lean_dev_t3_v11.sql

#

# ** Sync QFQ Data **

# 

[crose@tlX]

$ cd /scratch/share/system/sync/lean/

$ ./leanDevPreviewSync.sh -f

$ ./leanDevProdSync.sh -f

# 

# Restore FE_USERS

# 

$ mysql lean_prod_t3 < /var/tmp/prod/lean_prod_t3.fe_users.sql

$ mysql lean_preview_t3 < /var/tmp/prod/lean_prod_t3.fe_users.sql

# 

# Restore fileadmin

# 

$ mkdir /var/tmp/fileadmin.dev

# Keep dev version temporarily on prod for comparison in case 

$ mv /var/www/html/fileadmin/financialrequests /var/tmp/fileadmin.dev/financialrequests

$ mv /var/www/html/fileadmin/protected /var/tmp/fileadmin/.dev/protected

# Restore prod: financialrequests & protected

$ mv /var/www/html.v9/fileadmin/financialrequests /var/www/html/fileadmin/

$ mv /var/www/html.v9/fileadmin/protected /var/www/html/fileadmin/

# Restore preview: financialrequests & protected

$ \rm -R /var/www/html/preview/fileadmin/protected /var/www/html/preview/fileadmin/financialrequests

$ mv /var/www/html.v9/preview/fileadmin/financialrequests /var/www/html/preview/fileadmin/

$ mv /var/www/html.v9/preview/fileadmin/protected /var/www/html/preview/fileadmin/

</pre>

h2. Configuration: qfq.json and LocalConfiguration.php

* Prepare the final config files in `/var/tmp/dev/prev|prod`

qfq.json:

* DB_1_USER

* DB_1_PASSWORD

* DB_1_NAME

LocalConfiguration.php:

* db

* user

* password

* baseUrl

* wkhtmltopdf 

* redirect E-Mail

* sitename

* installtoolpw

Copy config files:

<pre>

$ cp /var/tmp/dev/prod/LocalConfiguration.php /var/www/html/typo3conf/

$ cp /var/tmp/dev/prev/LocalConfiguration.php /var/www/html/preview/typo3conf/

</pre>

h2. Finalize

<pre>

# Alle Cronjobs aktvieren:

$ cronjob -e

$ cronjob -e -u www-data

</pre>

* Check Backup.

* Check PDF to SMB.

* Remove old lean dev instance (webwork16).

* Update kpit PWs with webwork16/webwork22 credentials.

* Update "Project Wiki":https://project.math.uzh.ch/projects/lean/wiki summary with new dev instance (webwork22).

*

h2. Python Module

Folgende Module werden für die verwendeten Skripte benötigt:

<pre>

pandas

openpyxl

cvxpy

cvxopt

</pre>

h2. Probleme bei der Migration

* Nach dem @do-release-upgrade@ waren noch viele PHP 7.4 Pakete (Status: rc) installiert. @apt purge php7.4*@

* Es fehlten PHP Pakete: @apt install php-xml php-json@. Nach der Migration konnte Typo3 weder BE noch FE rendern. CR hat dann eine neue T3 Instanz hochgezogen, da wurde php-xml moniert. Damit war der Fehler klar. In den Apache Logs war der Fehler nicht zu erkennen.

* T3 Site-Konfiguration: T3 hat in der Site Konfiguration die URL von webwork22 gehabt (dump von webwork22). Das wird aktuell durch die I-MATH Sync Skripte noch nicht automatisch angepasst. @Sites > Site Configuration > Entry Point@

* Das FE Login Template sieht schlecht aus. Im @typo3_src@ Vezeichnis hat EN einige Anpassungen gemacht (webwork22) - darum als Quickfix die angepasste Version auf lean kopiert (geht mit dem naechsten Typo3 Update verloren (@typo3_src-11.5.25/typo3/sysext/felogin/Resources/Private/Templates/Login/Login.html@)