The availability of relatively cheap computing resources via Raspberry Pis, combined with the faster, flexible, and open-source version of DeltaRCM that we’d just developed (pyDeltaRCM) piqued my curiosity about networking and managing a small cluster of computing resources.
I ended up acquiring a set of Raspberry Pi 4s; my current set up is below. This became an interesting learning experience about networking and head-less ssh connections.
|4 RasberryPis (right) connected via an ethernet switch (bottom right). Powered via USB power hub (bottom left) with a shared 256GB SSD NFS drive (center left).|
Ultimately, this little project of mine has been a success. By running 64-bit Ubuntu Server on each of the Pis, I have been able to successfully install pyDeltaRCM and simulate river delta evolution on these little devices. Each Pi has 4 CPU cores, and since pyDeltaRCM enables easy parallelization, I am able to comfortably run 3 simultaneous simulations per Pi, so 12 in total. While the CPUs themselves are not wildly fast or powerful, for model domain sizes and durations similar to those in the early DeltaRCM publications, each run completes in 2 days. This means that every 2 days, a set of 12 simulations of a reasonable domain size, simulating about 150 years of time, can be completed. Extrapolating to a month, that is 180 model runs! This volume of output opens up many possibilities for model experimentation, all while not requiring CPU resources from my primary workstation.
The Pis are connected to a common
which I can access from my
desktop. This connectivity means I do not have to individually
ssh into each
Pi to transfer model outputs via
scp. Instead, I can mount the NFS drive
on my primary computer and pull the model outputs from there. At this time I
have yet to turn the set of 4 Pis into a single coherent “cluster” that accepts
jobs and distributes them. So figuring out how to do that is the next step for
me, right now I have to
ssh into each Pi individually to start any set of