Build a PI Cluster for Local Development - Part 1
Published 5 months and 1 week ago, Categorized under: development, docker, docker-swarm, coding, pi, raspberrypi

We all know the PI device by now, but if you don't check the RaspberryPI foundation website, as this post is not to talk about the PI as a device but more about how to create your own cluster.

For Part one we will go through how to prepare our PI and install all the basic required software.

RaspberryPI 4

By the end of this three parts series you will have the following main Applications installed on your PI Cluster, so you can use them within your day to day development workflow:

  1. MariaDB.
  2. PostgreSQL 13.
  3. Redis 6.
  4. Minio Server.

My Local RaspberryPI 4 Cluster Dashboard

This is just the start, you can add more later based on your needs.

I've done this to takeoff some resources from my local machine and have them on the cluster, as this makes my development workflow simpler (and faster). Plus, I won't need to install any of these services on any computer (new or old) any more all I need to install is a PHP and web server, and I am ready to go.

Requirements:

To build a cluster you need to have:

  1. Two or more PIs (preferably PI4 2GB or 4GB).
  2. Two or more micro SD cards (check the SD card requirements).
  3. Two USB Cables.
  4. (Optional) PI cases, one for each device.

Operating System:

To be able to use the Raspberry PI in general you need to install an Operating System, and there are various distributions which you can choose from, for this tutorial we will be using the 64-Bit of the Raspberry Pi OS, which by the time of writing it is still in beta phase. You can download the beta version from here.

You can use the 32-bit without any problem, and I'll add notes about the changes you need to do later when we introduce the docker images we are going to use.

Installing operating system images:

Raspberry Pi have developed a graphical SD card writing tool that works on Mac OS, Ubuntu 18.04 and Windows, and is the easiest option for most users as it will download the image and install it automatically to the SD card.

  1. Download the latest version of Raspberry Pi Imager and install it.
  2. Connect an SD card reader with the SD card inside.
  3. Open Raspberry Pi Imager and choose the required OS from the list presented.
  4. Choose the SD card you wish to write your image to.
  5. Review your selections and click 'WRITE' to begin writing data to the SD card.

If using the Raspberry Pi Imager on Windows 10 with Controlled Folder Access enabled, you will need to explicitly allow the Raspberry Pi Imager permission to write the SD card. If this is not done, the Raspberry Pi Imager will fail with a "failed to write" error.

More options can be read from RaspberryPI manual here.

Preparing your PI:

Once you finished installing the Operating System there are two things you need to do: enable SSH access and configure your WI-FI connection, without the need to connect them to a display or a keyboard.

Once an image is created on an SD card, by inserting it into a card reader on a Linux or Windows machines the boot folder can be accessed.

  1. To enable the SSH all you need to do is to create an empty file called ssh within your boot directory.
  2. To configure your WI-FI connection you need to create a file called wpa_supplicant.conf within the boot directory, here is an example of what the file can contain:
ctrl_interface=DIR=/var/run/wpa_supplicant GROUP=netdev
update_config=1
country=<Insert 2 letter ISO 3166-1 country code here>

network={
 ssid="<Name of your wireless LAN>"
 psk="<Password for your wireless LAN>"
}

Note that some older wireless dongles don't support 5GHz networks.

More information on the wpa_supplicant.conf file can be found here. See Wikipedia for a list of 2 letter ISO 3166-1 country codes. More info about setting up a Raspberry Pi headless can be found here.

Configuration:

Connect your PIs to a power source and wait a few seconds for the PIs to start then run the following command:

$ nmap -sn 192.168.68.1/24 | grep pi

As this command will help you find the IP addresses for the PIs, remember to replace 192.168.68.1 with your router IP address.

Once you knew the IP address write them down and now it is the time to connect to your PIs.

Mine is:

  1. 192.168.68.120
  2. 192.168.68.125

I'll be using them here, but you need to always replace them with your own IPs.

Create your SSH Key:

It is advisable that you connect to your PIs using SSH Key authentication and not using the normal password authentication, to do so, you need to create an SSH key or use one you already have.

If you already have SSH key you can skip this step, if not you need to run the following command

$ ssh-keygen -o -a 100 -t ed25519 -f id_rsa

This will generate SSH key called id_rsa within your .ssh directory, be careful not to run the command if you already have SSH key called id_rsa as it will overwrite the one you already have.

For more information about SSH Usage check this link.

Copy your SSH key:

You can copy your Public SSH key to the PI for easy accessing using the following command, you need to do it for each and every PI you have:

$ ssh-copy-id -i ~/.ssh/id_rsa.pub pi@192.168.68.120

Once you run the command it will ask you to accept the SSH figureprint of your PI then it will ask you for the password, the default password for the pi user is raspberry.

I am trying to make this post short, so I am not going to go into details about securing your PI, more info can be found in this article by the Raspberry PI foundation.

Update your PI packages:

One of the first thing you need to do when you have SSH into your PI is to update the operating system software, by running the following command you will get the latest packages and install them, then clean up those which we don't need:

$ sudo apt update -y && sudo apt upgrade -y && sudo apt autoremove -y

Once you update the OS packages we need to update the Raspberry PI firmware, you can do that using the following command:

$ sudo rpi-update 

If everything is up-to-date you will get the following message

 *** Raspberry Pi firmware updater by Hexxeh, enhanced by AndrewS and Dom
 *** Performing self-update
 *** Relaunching after update
 *** Raspberry Pi firmware updater by Hexxeh, enhanced by AndrewS and Dom
 *** Your firmware is already up to date

Otherwise, It will ask you if you want to update and after that you need to reboot your PI.

Run raspi-config:

There are many configuration options that you may want to change or update, there is a tool called raspi-config, which you can use to do so, to access it you need to run the following command:

$ sudo raspi-config

Once you do, you will be presented with the following screen, take your time to check each option:

Raspberry Pi Software Configuration Tool

Make sure you to go over the localisation tools, and the Advanced Options.

Lastly, this is my PI cluster setting at my desktop 😆

My Local Cluster

This is all for this post, in next post I'll explain how to install Docker and GlusterFS on your PI.



Build a PI Cluster for Local Development:

  1. Part one: Preparing your PIs.
  2. Part two: Installing Docker, Docker swarm and Gluster.
  3. Part three: Create your Stack (MariaDB, PostgreSQL, Redis and Minio)
Share It via: