Disclaimer: everything expressed in this post is my own opinion. Logos are trademarks of their respective owners.

This guide outlines how to setup a Raspberry Pi 5 with the RADXA Penta SATA HAT to run OMV (Open Media Vault) with a ZFS filesystem, and host NextCloud and Immich using Docker and Tailscale. Please read the motivation for this build guide here: https://www.ianwebster.ca/post/omv-rpi5-server!

You'll need the following hardware components for this build:

Optional components:

This guide considers the following hardware/software configuration:

Here is a diagram of the configuration at a high level:

Here's my comprehensive guide to setting up the RPi5, hardware and software.

First, let's build the Pi by installing the Cooler and RADX Penta Sata Hat.

Raspbian is the OS that our Pi will be running. OMV will be installed on top of Raspbian.

Next up, we'll install OMV. All the subsections below are based on this OMV7 guide https://wiki.omv-extras.org/doku.php?id=omv7:raspberry_pi_install which you should refer to for the most updated guidance. It is possible a newer version of OMV is released by the time you read this!

Execute the following commands to install OMV:

Enabling PCIe Gen3:

The link speed should now be 8GT/s. Note the increase from 390MB/s to 496MB/s.

You can install ZFS to protect your data from expected drive degradation over time. ZFS relies on using RAM as a filesystem cache, so I would recommend having at least an 8GB RPi5 to enable ZFS. I will make the assumption you have a 2x 2TB SSD setup attached to your SATA Hat for this subsection, and I will assume a Mirrored ZFS Pool with 2x 2TB SSDs. In this configuration, you will have 2TB of usable space, and if 1 drive fails, you may replace it to rebuild your pool.

Steps:

Click Save. Your new pool ssd-pool should now be created! The status should be OK.

Enabling TRIM on your ZFS pool generally extends the lifespan of your SSDs.

You can check the status on TRIM and run it manually with the following commands:

To set up a recurring cron job to run TRIM on your pool:

TRIM will now run every week to maintain the health of your SSDs.

If you want to access files from your server via Finder on your Mac for example, you'll need to create a SMB share. In this section, you will create a share folder, an SMB share, and as result, you will be able to access a share folder on the ssd-pool from your Mac in Finder. You can skip this section if you don't want drag-n-drop read/write compatibility with your server, but note that some these steps are required in section 2.7.

Click Save.

Click Save.

Click save.

Let's prepare for installing Docker by setting up our share folders. We'll need a backup folder, a compose folder, and an appdata folder.

Create 3 more Share Folders, using the same process as in section 2.6.2:

If you would like, you can setup the Permissions and the Access Control List in the same way before if you want to have access to your Docker container data via your Finder on your Mac, e.g. NextCloud and Immich. My recommendation is not to do so.

Now let's install Docker to host your NextCloud and Immich instances. Each instance will have its own container, and its own Tailscale container.

Tailscale is a pre-requisite to accessing your server's NextCloud and Immich instances outside your home network (although we'll also use Tailscale for home network access in this guide). As an alternative, you could setup port forwarding in your router, and use a reverse proxy like Caddy to forward external requests to either the NextCloud instance or the Immich instance. Tailscale offers a solution that avoids the hassle of port forwarding on your router.

We'll revisit Tailscale after installing NextCloud.

To install NextCloud, you'll need to create Docker compose file. You can do this by SSHing into the Pi, creating the Compose file, and using the CLI to build the compose file, but the more convenient approach for OMV is to copy and paste it into the UI.

To get started, let's build the Docker Compose file. I have adapted the compose file from this GitHub discussion from NextCloud: https://github.com/nextcloud/all-in-one/discussions/5439. I made a few improvements to make this file more OMV-friendly, such as inline-ing the Caddyfile and Tailscale Serve.json file. Ideally, you would keep the Caddyfile and Serve.json files separate, but we are limited by the OMV UI 🙂

You can optionally add a backup for your NextCloud config. Under Create a backup location, add ssd-pool/docker-backup that you created earlier in section 2.7 under OMV > Docker > Compose > Settings. Please review the nextcloud.yml to understand the implication of using CHANGE_TO_COMPOSE_DATA_PATH. Also navigate to and read this link http://10.0.1.89/#/services/compose/settings (replacing your static ip accordingly). Importantly, by setting our NEXTCLOUD_DATADIR: CHANGE_TO_COMPOSE_DATA_PATH/nextcloud_data in nextcloud.yml https://gist.github.com/iWebster28/37fb9fb7a6ae9708ad2ad461ce3d0a92#file-nextcloud-yml-L18, you are storing all user-uploaded data outside of our NextCloud container, e.g. docker-data/nextcloud_data - this means that user-uploaded data won't be backed up by default.

Note: It is possible to add the NEXTCLOUD_DATADIR from nextcloud.yml https://gist.github.com/iWebster28/37fb9fb7a6ae9708ad2ad461ce3d0a92#file-nextcloud-yml-L18 as a backup location, if you are curious:

Now that you have your NextCloud containers running, the next step is to revisit Tailscale to ensure your can access your NextCloud instance from inside and outside your network. Since we have NextCloud running behind a reverse proxy (Caddy) and Tailscale, you will need to access it from nextcloud.my-tailnet.ts.net, even on your own network.

Next, SSH into your server to get an HTTPS cert for your domain nextcloud.my-tailnet.ts.net:

(Development purposes only; do not do this otherwise) Copy your key and certificate to a temporary directory and delete them once your setup is working. This will save you the hassle of needing to regenerate keys and hitting the ACME request limit for generating HTTPS certs. You can see what HTTPS certs you have generated here: https://crt.sh/?q=nextcloud.my-tailnet.ts.net (replace your domain name).

If you want to copy these keys back to your tailscale container:

Then navigate to https://login.tailscale.com/admin/machines and remove the old machine, and any new machine e.g. nextcloud-1.

Restart the Tailscale container:

To open your server to the open internet, you can use Tailscale's beta feature called Funnel.

Setting up Immich is similar to sections 2.9 through 2.12 for NextCloud!

SSH into your server to get an HTTPS cert for your domain immich.my-tailnet.ts.net:

You have now completed building a friends-and-family shareable, internet-accessible NextCloud and Immich instance on a Raspberry Pi 5 running OMV on Raspbian with a RADXA Penta SATA HAT using SDDs in a ZFS Mirror, drawing under 15W at load, and occupying the space of a small reusable water bottle! Great job!

Clear your browser cache, restart your browser, try incognito mode, or try another browser.

Likely, Docker is the cause. The following command clears many GB of space. Another possible solution is moving the docker build cache to the SSDs instead of the SSD card and symlinking it back on the OS drive. Alternatively, get a dedicated boot drive.

No. Here's the exact thread that answers this: https://github.com/nextcloud/all-in-one/discussions/5439#discussioncomment-11894807.

Basically, Funnel is a beta feature that only supports forwarding ports 443 and 80. NextCloud Talk uses 3478 and 3479, which cannot be forwarded with Funnel. You can however use NextCloud talk if both clients (and server of course) are connected to the Tailnet.

Talk uses TCP and UDP, and currently, funnel forces TCP.

I have tried forcing TCP to see if Talk would work over funnel, but to no avail. This is the configuration I used in Tailscale if you're curious:

Regardless, the config from the main guide allows you to use Talk as long as both clients are on logged onto their Tailscale clients, which is more secure.

If your ZFS install fails when updating OMV packages, then you may see this red popover error when running docker: 500 Internal Server Error - No file system backend exists for 'zfs'. Occasionally, this happen when you update packages too.

SSH into your Pi and try loading your pool name:

If the zpool command fails, then you may need to install the zfs packages from debian (since at the time of writing this, raspbian may not fully support the dependent zfs packages)

And run again

There should be no errors. You should see your pool again at http://10.0.1.89/#/storage/zfs/pools. You may need to restart your Pi for NextCloud/Immich to pick up the ZFS pool again.

If this hangs:

Check Tailscale container logs:

https://letsencrypt.org/docs/rate-limits/#new-certificates-per-exact-set-of-hostnames

You can only request 1 cert every 34 hours. Copy the keys to an external directory as in my guide to prevent this from happening during dev!

When running this inside your Tailscale container:

If you get this error:

Then make sure you have HTTPS enabled in your Tailscale account: https://login.tailscale.com/admin/dns.

If you are having issues with Funnel, you can test from CLI:

Note: tinkering with Funnel/Tags/ACL may end up creating multiple machines in your Tailscale account. You can always delete them and Tailscale will recreate them https://login.tailscale.com/admin/machines.

Sometimes, you may end up with multiple duplicates of your service/machine e.g. nextcloud, nextcloud-1, and so on. Be sure to stop your containers in OMV UI or from the CLI over SSH. You can delete your containers from the machines page https://login.tailscale.com/admin/machines and then troubleshoot the issue. Remember that Funnel uses the ACL based on the emitted tag to route traffic, so if your machine has a name of nextcloud-1 or immich-1, then it will break, and will likely create a domain name clash with NextCloud's configuration.

If you want to start over, you can optionally delete your NextCloud DATADIR. Do not do this if you do not want to lose your account and user data. This is not a requirement to start fresh if you just want to prune your containers and verify your config works.

You can also run some CLI Docker commands to dive deep into debugging. Here are some helpful Docker commands you can run if you SSH into your Pi:

You can always find the compose file from OMV and run it from your CLI after SSHing into your Pi:

Caution: if you did not set your NEXTCLOUD_DATADIR to be outside your container, then this command will delete all your user data stored in the container! Only run this if you want a completely fresh start.)

If you run this command:

And get this error:

Fix it by running:

This may happen if you have a power outage and one of the SSDs is writing during that time period. To fix it, start a scrub by going to Storage > zfs > Pools > Select your ssd-pool > Tools > Scrub. You can check the status of the scrub from CLI:

You may have to wait a few hours. Once this is done, the drive should come back online.

This may happen if a drive completely fails. In this case, you may need to RMA your failing drive. Once you have a replacement drive, shut down your server and attach the new drive.

Replace the drive in your ZFS pool using the following commands:

Thank you for reading until the end:)