developer:ns7_upgrade [NethServer Wiki]

This page is read only. You can view the source, but not change it. Ask your administrator if you think this is wrong.
# NethServer 7 upgrade planning (obsolete)

**WARNING: OBSOLETE**

2017-03-21 -- The sme8 migration and ns6 upgrade features has been implemented in ns7! 

References:

* discussion: http://community.nethserver.org/t/upgrade-paths-to-ns7/4771
* ns6 upgrade: https://github.com/NethServer/dev/issues/5234
* sme8 migration: https://github.com/NethServer/dev/issues/5196

## 1. Restore from backup

_As sysadmin, I want to copy data and configuration of an existing system to a ns7 clean installation._

### Conditions Of Acceptance

* supported "existing systems" must be 

  - NethServer 6 and 7 (and derivatives), 

  - Nethesis NethService 8 / sme8 (and 9?)

* no IP conflict: source and target systems can run simultaneously on the same LAN

### Solution 1.1

Different procedures, for different scenarios:

1. From NethService 8 / sme8 (and 9?) to ns7: the "traditional" ``migration-import`` event imports data from the backup set

2. From ns7 to ns7, standard restore procedure 

3. From ns6 to ns7, migration procedure + standard restore procedure


## 2. In-place upgrade

_As sysadmin, I want to upgrade a running ns6 installation to ns7, with minimal downtime._

### Conditions Of Acceptance

* The downtime must be less than the time required to install ns7 and restore a backup from an external media on the same hardware

### Solution 2.1

1. Execute pre-config-backup events

2. Run upstream upgrade tool against upstream repositories

3. Upgrade nethserver packages (without update events)

4. Reboot the system

5. Restore config (apply "solution 1 - procedure 3" from the above section).

----

# Implementation

## Restore from ns6

### issue 1 (closed/worksforme)

``package-list`` contains obsolete packages: how to filter it? Solutions:

1. ``yum list <packages>`` output filters out non-available packages


### issue 2 - nethserver-dc

On ns7 nethserver-dc is an alternative to nethserver-directory. If it is installed on the destination system the [samba classic upgrade](https://wiki.samba.org/index.php/Migrating_a_Samba_NT4_domain_to_a_Samba_AD_domain_%28classic_upgrade%29) must be run to import existing passwords and retain old machine accounts.

The nethserver-dc requires an additional IP to run. It is not available from the backup set of ns6, so it cannot start automatically during the restore procedure!

- Solution 1 (draft)
  
  * pre-requisite 1.1: remove "Conflicts nethserver-directory" from ``nethserver-dc.spec``
  * install nethserver-directory automatically and restore ``ldap-data.ldif`` and ``ldap-config.ldif`` as usual
  * on restore procedure completion, start the DC as usual. The provision procedure must detect nethserver-directory and attempt the classic upgrade against it

- Solution 2 (draft)

  * pre-requisite 2.1: change the backup procedure in ns6, to dump the SAM DB in smbpasswd format: ``pdbedit -L -w``.

- **Solution 3** (similar to 1)
  
  * install nethserver-directory automatically and run restore-config. ``ldap-data.ldif`` and ``ldap-config.ldif`` are restored as usual
  * fix LDAP ACLs to allow samba access from nsdc IP
  * remove nethserver-directory package, but keep slapd daemon running
  * install nethserver-dc (retaining the Conflicts tag)
  * start the DC as usual. The provision procedure detects the special classic upgrade case and runs a classic upgrade instead of fresh install.


### issue 3 - mail-server alias expansion

Records of type ``pseudonym`` almost retain the same format of ns6. 

Alternative solutions:

1. Fix the The ``Account`` prop in DB, by adding the ``@<domain>`` suffix.
2. See issue 5. Fix the /etc/postfix/virtual template to support "legacy" mode by adding the ``@domain`` suffix, if missing.

System groups are no longer mail distribution lists (prop ``MailDeliveryType=copy``). Existing groups must be converted to pseudonym records with multiple recipients.


### issue 4 - hostname not applied

The old FQDN from ns6 backup is not applied.

On ns7 the config keys ``SystemName`` and ``DomainName`` come directly from the ``gethostname()`` syscall.  Existing DB values are ignored and overridden at run time.

The restore procedure must set the hostname in a preliminary step (event ``pre-restore-config``).

To support an "install-before-restore" scenario, nethserver-directory should add a ``realm leave`` during ``pre-restore-config`` event, and a ``realm join`` during ``post-restore-config``.


### issue 5 - PAM-based services (closed/#5144)

For instance for ejabberd:

- need to change the client username by adding the ``@domain`` suffix
- logon with new username and old password from backup is OK
- all users are listed as hidden (as expected on my test VM)

The username change on all clients affects almost any PAM-based service. It's a big problem if there are many clients to reconfigure.

Do we really want the ``@domain`` suffix? As workaround we can apply a backward compatible configuration to sssd.conf, like the following:

```
[sssd]
domains = dpnet.nethesis.it, legacy
config_file_version = 2
services = nss, pam

[domain/dpnet.nethesis.it]
enumerate = True
cache_credentials = True
default_shell = /usr/libexec/openssh/sftp-server
use_fully_qualified_names = True
id_provider = ldap
ldap_uri = ldap://127.0.0.1
ldap_search_base = dc=dpnet,dc=nethesis,dc=it
ldap_tls_reqcert = never

[domain/legacy]
enumerate = False
cache_credentials = True
default_shell = /usr/libexec/openssh/sftp-server
use_fully_qualified_names = False
id_provider = ldap
ldap_uri = ldap://127.0.0.1
ldap_search_base = dc=dpnet,dc=nethesis,dc=it
ldap_tls_reqcert = never


[nss]
```

With nethserver-dc "legacy" domain should look like

```
[domain/legacy]
enumerate = False
cache_credentials = True
default_shell = /usr/libexec/openssh/sftp-server
use_fully_qualified_names = False
id_provider = ad
access_provider = ad
ad_domain = nethesis.it
krb5_realm = NETHESIS.IT
krb5_store_password_if_offline = True
ldap_id_mapping = True
```

Experiments with the backward-compatible/"legacy" sssd.conf configuration:

- server manager, login OK, users and groups OK
- ssh OK
- ejabberd OK
- dovecot: in userdb remove ``home=`` override, configure ``mail_location``. TODO: check shared folders. See also [the PR attempt to fix Maildir path](https://github.com/NethServer/nethserver-mail-server/pull/15)
- postfix: fix the alias expansion in /etc/postfix/virtual template. See issue 3 above
- webtop OK

### issue 6 - Maildir path

The ns7 maildir use the long user name (with ``@domain`` suffix) form. On ns6 there was the short form. We need to ``mv`` to the new format. See also issue 3.

Implementation draft:

- https://github.com/NethServer/nethserver-mail-server/commit/7eff700ed619db0027f6c7f07290c28023163046
- https://github.com/NethServer/nethserver-mail-server/commit/4c920f1d2bfda674e007384b500110ca1a7be33b

## Migration-import modules for sme8

The migration from sme8 has the following pre-condition:

    Required modules must be installed before the migration occurs.
    
This includes the choice of the local accounts provider: Samba AD or OpenLDAP?

Checklist of existing NethServer modules with an action in **migration-import** event

 - **nethserver-base**
 - nethserver-cups
 - nethserver-directory
 - **nethserver-dnsmasq**
 - nethserver-ejabberd
 - nethserver-faxweb2
 - nethserver-fetchmail / getmail
 - **nethserver-hosts**
 - **nethserver-httpd**
 - **nethserver-httpd-admin**
 - nethserver-hylafax
 - nethserver-ibays
 - nethserver-mail-common
 - nethserver-mail-filter
 - nethserver-mail-server
 - nethserver-mysql
 - **nethserver-ntp**
 - **nethserver-openssh**
 - nethserver-samba

**BOLD**: packages in ISO

Tracked by issue https://github.com/NethServer/dev/issues/5196