From bff9d08430d9df84bf6b5a7c324a05a6a65b49a8 Mon Sep 17 00:00:00 2001
From: Richard Goodwin <github@richardgoodwin.com>
Date: Fri, 26 Oct 2018 06:58:09 -0500
Subject: [PATCH 01/29] Merge upstream master

---
 headless-scripts/Readme.md | 81 ++++++++++++++++++++++++++++++++++++++
 1 file changed, 81 insertions(+)
 create mode 100644 headless-scripts/Readme.md

diff --git a/headless-scripts/Readme.md b/headless-scripts/Readme.md
new file mode 100644
index 0000000..2eedbaa
--- /dev/null
+++ b/headless-scripts/Readme.md
@@ -0,0 +1,81 @@
+# Flashable image to get started more quickly
+
+# This is a WORK IN PROGRESS, SHOULD BE WORKING. 
+
+
+## Notes 
+
+* Assumes your Pi has access to Wifi, with internet access (during setup). (But all setup methods do currently.) USB networking still enabled for troubleshooting.
+
+
+## Configure the SD card before first boot of the Pi
+
+
+1. Flash the image from [here (10-24 dated image)](https://www.dropbox.com/s/6f8kxenvtz8pkj9/image_2018-10-24-teslausb_headless-lite.zip?dl=0) using Etcher or similar.  (Be sure to click the `...` on Dropbox and download the `.zip` file.)
+1. Mount the card again, and in the `boot` directory create a `teslausb_setup_variables.conf` file to export the same environment varibles normally needed for setup (including archive, Wifi, and push notifications (if desired).) A sample conf file is in this repo.  I.e. file should contain at a minimum (replace with your own values):
+    ```
+    export archiveserver=Nautilus
+    export sharename=SailfishCam
+    export shareuser=sailfish
+    export sharepassword=pa$$w0rd
+    export campercent=100
+    export SSID=your_ssid
+    export WIFIPASS=your_wifi_password
+    # export pushover_enabled=false
+    # export pushover_user_key=user_key
+    # export pushover_app_key=app_key
+    ```
+    (Pushover should be working but commented out by default.)
+* Boot it in your Pi, give it a bit, watching for a series of flashes (2, 3, 4, 5, maybe 6) and then a reboot and/or the CAM to become available on your PC/Mac.
+* The Pi should be available at `teslausb.local` over Wifi (if it works) or USB networking (if it doesn't). Takes about 5 minutes for me. You should see in `/boot` the TESLAUSB_SETUP_FINISHED and WIFI_ENABLED files as markers of success too.
+* Currently doesn't create the TeslaCam folder, so you'll need to do that before taking to your car.
+* If plugged into just a power source, or your car, give it a few minutes until the LED starts pulsing steadily which means the archive loop is running and you're good to go. 
+
+## What happens under the covers
+
+When the Pi boots the first time: 
+* A `/boot/teslausb-headless-setup.log` file will be created and stages logged. This takes the place of the "STOP" commands 
+* Marker files will be created in `boot` like `TESLA_USB_SETUP_STARTED` and `TESLA_USB_SETUP_FINISHED` to track progress. 
+* (Working on a progress system so the script can pick back up if needed. This is probably useful for the general/old way of setup too.)
+* Wifi is detected by looking for `/boot/WIFI_ENABLED` and if not, creates the `wpa_supplicant.conf` file in place and reboots. 
+* The Pi LED will flash patterns (2, 3, 4, 5, maybe 6) as it gets to each stage (labeled in the setup-teslausb-headless script). 
+  * 10 flashes means setup failed!
+  * After the final stage and reboot the LED will go back to normal. Remember, the step to remount the filesystem takes a few minutes.
+
+At this point the next boot should start the Dashcam/music drives like normal. If you're watching the LED it will start flashing every 1 second, which is the archive loop running. 
+
+> NOTE: Don't delete the `TESLAUSB_SETUP_FINISHED` or `WIFI_ENABLED` files. This is how the system knows setup is complete. 
+
+### Image builder source and patches
+
+For now the image creation work is at:
+* Modified pi-gen [rtgoodwin's fork of pi-gen](https://github.com/rtgoodwin/pi-gen) in (whatever current branch I'm working at the time). 
+* `headless-patch` branch of rtgoodwin fork [https://github.com/rtgoodwin/teslausb/tree/headless-patch/headless-scripts](https://github.com/rtgoodwin/teslausb/tree/headless-patch/headless-scripts)
+
+
+### Image creation TODOs
+1. Patch the hostname to teslausb
+1. Make it so if someone deletes the `TESLAUSB_SETUP_FINISHED` file it's handled gracefully. 
+1. I still see some errors during pi-gen about locale, may need to be fixed? stage0/01-locale/debconf en_US.UTF-8
+1. Cache the remount packages? Might mess with first boot like `rsyslog`
+1. Any other steps to move into the base image?
+1. Aspirational TODO: Remove more packages and set services to stopped to make the boot process faster? 
+1. NOTE: I moved all script downloads and variable creation to the initial setup. At this point, I'm designing it to pull the setup scripts dynamically, since development is still ongoing. If/when we reach a good frozen state, we can generate an image that is ready to run. I think it'll also be pretty tricky to do some of the remounting and creating the backing files etc. on the image creation side. Open to suggestions/contributions there though! At the very least we could bake in stable first stage headlessBuild scripts for Mac/Linux/Windows.
+
+
+#### Modifications to pi-gen builder from master
+
+Built image on a Raspi running Stretch, for maximum Pi-ception. 
+
+1. Add SKIP and SKIP_IMAGES files to stage3, 4, and 5 (if present). 
+1. Add a stage6. (There are stages0-5, but may be a stage5 in some cases. This will help keep a clean merge later.)
+1. Copy the prerun.sh from `stage2`. Be SURE to mark `chmod +x` it. 
+1. Remove or rename the EXPORT_NOOBS files in all stages. We don't need a NOOBS image built. 
+1. In `stage6`, create a `00-tweaks` folder, with a `00-patches` folder and patch inside to patch `cmdline.txt` to remove the resize and add the needed modules. The build process uses `quilt` for patching. Note: the path for any patching you do at this stage is `stage6/rootfs/FILEPATH` where `rootfs` represents the Pi's `/`. So, `cmdline.txt` is `stage6/rootfs/boot/cmdline.txt`.
+1. Add a patch for the `config.txt` file.
+1. Add a file called `series` in the patches directory with the name of each `.diff` file in the order you want them applied.
+1. Add a `files` folder in stage6 with modified `rc.local`. The modified `rc.local` will handle pulling down the `setup-teslausb-headless` file the first time. (Still working on build logic here.) Files are moved into final locations in a `00-run.sh` script and the `install` command. See the script for details. 
+1. (Yes at this point you could suggest that just putting the end state files in place instead of patching would be good, but why not be idiomatic? :)  )
+1. Add a script to flash LEDs
+1. Run `sudo ./build.sh` from the `pi-gen` directory.
+1. If you get a failure, it's almost certainly after stage2, so you can add SKIP files in stage2-stage5 present) and rerun `sudo CLEAN=1 ./build.sh`
\ No newline at end of file

From 6201dd91efd67e8f8f06c6c102a7a47c6f36ce12 Mon Sep 17 00:00:00 2001
From: Richard Goodwin <github@richardgoodwin.com>
Date: Tue, 30 Oct 2018 08:15:19 -0500
Subject: [PATCH 02/29] More logging in partition creation

---
 setup/pi/create-backingfiles-partition.sh | 14 ++++++++++++++
 setup/pi/setup-teslausb                   |  1 +
 2 files changed, 15 insertions(+)

diff --git a/setup/pi/create-backingfiles-partition.sh b/setup/pi/create-backingfiles-partition.sh
index 120130e..3c61b05 100644
--- a/setup/pi/create-backingfiles-partition.sh
+++ b/setup/pi/create-backingfiles-partition.sh
@@ -1,8 +1,18 @@
 #!/bin/bash -eu
 
+function setup_progress () {
+  SETUP_LOGFILE=/boot/teslausb-headless-setup.log
+  if [ $HEADLESS_SETUP = "true" ]
+  then
+    echo "$( date ) : $1" >> "$SETUP_LOGFILE"
+  fi
+    echo $1
+}
+
 BACKINGFILES_MOUNTPOINT="$1"
 MUTABLE_MOUNTPOINT="$2"
 
+setup_progress "Checking existing partitions..."
 PARTITION_TABLE=$(parted -m /dev/mmcblk0 unit B print)
 DISK_LINE=$(echo "$PARTITION_TABLE" | grep -e "^/dev/mmcblk0:")
 DISK_SIZE=$(echo "$DISK_LINE" | cut -d ":" -f 2 | sed 's/B//' )
@@ -15,19 +25,23 @@ LAST_BACKINGFILES_PARTITION_DESIRED_BYTE="$(( $DISK_SIZE - (100 * (2 ** 20)) - 1
 
 ORIGINAL_DISK_IDENTIFIER=$( fdisk -l /dev/mmcblk0 | grep -e "^Disk identifier" | sed "s/Disk identifier: 0x//" )
 
+setup_progress "Editing partition table for backing files..."
 BACKINGFILES_PARTITION_END_SPEC="$(( $LAST_BACKINGFILES_PARTITION_DESIRED_BYTE / 1000000 ))M"
 parted -a optimal -m /dev/mmcblk0 unit B mkpart primary ext4 "$FIRST_BACKINGFILES_PARTITION_BYTE" "$BACKINGFILES_PARTITION_END_SPEC"
 
 LAST_BACKINGFILES_PARTITION_BYTE=$(parted -m /dev/mmcblk0 unit B print | grep -e "^3:" | cut -d ":" -f 3 | sed 's/B//g' )
 
+setup_progress "Creating mutable (writable) partition for script usage..."
 MUTABLE_PARTITION_START_SPEC="$(( $LAST_BACKINGFILES_PARTITION_BYTE / 1000000 ))M"
 parted -a optimal -m /dev/mmcblk0 unit B mkpart primary ext4 "$MUTABLE_PARTITION_START_SPEC" 100%
 
 NEW_DISK_IDENTIFIER=$( fdisk -l /dev/mmcblk0 | grep -e "^Disk identifier" | sed "s/Disk identifier: 0x//" )
 
+setup_progress "Writing updated partitions to fstab and /boot/cmdline.txt"
 sed -i "s/${ORIGINAL_DISK_IDENTIFIER}/${NEW_DISK_IDENTIFIER}/g" /etc/fstab
 sed -i "s/${ORIGINAL_DISK_IDENTIFIER}/${NEW_DISK_IDENTIFIER}/" /boot/cmdline.txt
 
+setup_progress "Formatting new partitions..."
 mkfs.ext4 -F /dev/mmcblk0p3
 mkfs.ext4 -F /dev/mmcblk0p4
 
diff --git a/setup/pi/setup-teslausb b/setup/pi/setup-teslausb
index d92ad4b..110419e 100644
--- a/setup/pi/setup-teslausb
+++ b/setup/pi/setup-teslausb
@@ -16,6 +16,7 @@ then
 fi
 
 function setup_progress () {
+  SETUP_LOGFILE=/boot/teslausb-headless-setup.log
   if [ $HEADLESS_SETUP = "true" ]
   then
     echo "$( date ) : $1" >> "$SETUP_LOGFILE"

From 331721c742851cfe7a042c18cf827fddf6513205 Mon Sep 17 00:00:00 2001
From: Richard Goodwin <github@richardgoodwin.com>
Date: Tue, 30 Oct 2018 08:23:24 -0500
Subject: [PATCH 03/29] Pass script flag to parted

---
 setup/pi/create-backingfiles-partition.sh | 8 ++++----
 1 file changed, 4 insertions(+), 4 deletions(-)

diff --git a/setup/pi/create-backingfiles-partition.sh b/setup/pi/create-backingfiles-partition.sh
index 3c61b05..0857cca 100644
--- a/setup/pi/create-backingfiles-partition.sh
+++ b/setup/pi/create-backingfiles-partition.sh
@@ -25,15 +25,15 @@ LAST_BACKINGFILES_PARTITION_DESIRED_BYTE="$(( $DISK_SIZE - (100 * (2 ** 20)) - 1
 
 ORIGINAL_DISK_IDENTIFIER=$( fdisk -l /dev/mmcblk0 | grep -e "^Disk identifier" | sed "s/Disk identifier: 0x//" )
 
-setup_progress "Editing partition table for backing files..."
+setup_progress "Modifying partition table for backing files..."
 BACKINGFILES_PARTITION_END_SPEC="$(( $LAST_BACKINGFILES_PARTITION_DESIRED_BYTE / 1000000 ))M"
-parted -a optimal -m /dev/mmcblk0 unit B mkpart primary ext4 "$FIRST_BACKINGFILES_PARTITION_BYTE" "$BACKINGFILES_PARTITION_END_SPEC"
+parted -s -a optimal -m /dev/mmcblk0 unit B mkpart primary ext4 "$FIRST_BACKINGFILES_PARTITION_BYTE" "$BACKINGFILES_PARTITION_END_SPEC"
 
 LAST_BACKINGFILES_PARTITION_BYTE=$(parted -m /dev/mmcblk0 unit B print | grep -e "^3:" | cut -d ":" -f 3 | sed 's/B//g' )
 
-setup_progress "Creating mutable (writable) partition for script usage..."
+setup_progress "Modifying mutable (writable) partition for script usage..."
 MUTABLE_PARTITION_START_SPEC="$(( $LAST_BACKINGFILES_PARTITION_BYTE / 1000000 ))M"
-parted -a optimal -m /dev/mmcblk0 unit B mkpart primary ext4 "$MUTABLE_PARTITION_START_SPEC" 100%
+parted -s -a optimal -m /dev/mmcblk0 unit B mkpart primary ext4 "$MUTABLE_PARTITION_START_SPEC" 100%
 
 NEW_DISK_IDENTIFIER=$( fdisk -l /dev/mmcblk0 | grep -e "^Disk identifier" | sed "s/Disk identifier: 0x//" )
 

From 14d1e52a9feede5c7c07cca1989d4c1ab08df96f Mon Sep 17 00:00:00 2001
From: Richard Goodwin <github@richardgoodwin.com>
Date: Tue, 30 Oct 2018 08:38:28 -0500
Subject: [PATCH 04/29] Clearer logging message

---
 setup/pi/create-backingfiles-partition.sh | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)

diff --git a/setup/pi/create-backingfiles-partition.sh b/setup/pi/create-backingfiles-partition.sh
index 0857cca..451c4ef 100644
--- a/setup/pi/create-backingfiles-partition.sh
+++ b/setup/pi/create-backingfiles-partition.sh
@@ -25,13 +25,13 @@ LAST_BACKINGFILES_PARTITION_DESIRED_BYTE="$(( $DISK_SIZE - (100 * (2 ** 20)) - 1
 
 ORIGINAL_DISK_IDENTIFIER=$( fdisk -l /dev/mmcblk0 | grep -e "^Disk identifier" | sed "s/Disk identifier: 0x//" )
 
-setup_progress "Modifying partition table for backing files..."
+setup_progress "Modifying partition table for backing files partition..."
 BACKINGFILES_PARTITION_END_SPEC="$(( $LAST_BACKINGFILES_PARTITION_DESIRED_BYTE / 1000000 ))M"
 parted -s -a optimal -m /dev/mmcblk0 unit B mkpart primary ext4 "$FIRST_BACKINGFILES_PARTITION_BYTE" "$BACKINGFILES_PARTITION_END_SPEC"
 
 LAST_BACKINGFILES_PARTITION_BYTE=$(parted -m /dev/mmcblk0 unit B print | grep -e "^3:" | cut -d ":" -f 3 | sed 's/B//g' )
 
-setup_progress "Modifying mutable (writable) partition for script usage..."
+setup_progress "Modifying partition table for mutable (writable) partition for script usage..."
 MUTABLE_PARTITION_START_SPEC="$(( $LAST_BACKINGFILES_PARTITION_BYTE / 1000000 ))M"
 parted -s -a optimal -m /dev/mmcblk0 unit B mkpart primary ext4 "$MUTABLE_PARTITION_START_SPEC" 100%
 

From e3230483bcff0fce69d01f1b7c8209aeff60bb89 Mon Sep 17 00:00:00 2001
From: Richard Goodwin <github@richardgoodwin.com>
Date: Tue, 30 Oct 2018 08:59:57 -0500
Subject: [PATCH 05/29] Add -s to another parted line

---
 setup/pi/create-backingfiles-partition.sh | 2 +-
 setup/pi/setup-teslausb                   | 1 +
 2 files changed, 2 insertions(+), 1 deletion(-)

diff --git a/setup/pi/create-backingfiles-partition.sh b/setup/pi/create-backingfiles-partition.sh
index 451c4ef..14a81ae 100644
--- a/setup/pi/create-backingfiles-partition.sh
+++ b/setup/pi/create-backingfiles-partition.sh
@@ -13,7 +13,7 @@ BACKINGFILES_MOUNTPOINT="$1"
 MUTABLE_MOUNTPOINT="$2"
 
 setup_progress "Checking existing partitions..."
-PARTITION_TABLE=$(parted -m /dev/mmcblk0 unit B print)
+PARTITION_TABLE=$(parted -s -m /dev/mmcblk0 unit B print)
 DISK_LINE=$(echo "$PARTITION_TABLE" | grep -e "^/dev/mmcblk0:")
 DISK_SIZE=$(echo "$DISK_LINE" | cut -d ":" -f 2 | sed 's/B//' )
 
diff --git a/setup/pi/setup-teslausb b/setup/pi/setup-teslausb
index 110419e..b7561cd 100644
--- a/setup/pi/setup-teslausb
+++ b/setup/pi/setup-teslausb
@@ -170,6 +170,7 @@ function create_usb_drive_backing_files () {
   
   if [ ! -e /dev/mmcblk0p3 ]
   then
+    setup_progress "Starting to create backing files partition..."
     /tmp/create-backingfiles-partition.sh "$BACKINGFILES_MOUNTPOINT" "$MUTABLE_MOUNTPOINT"
   fi
   

From 0f8c380f54c7f8532ac5c7015f94de9ce8c85c0d Mon Sep 17 00:00:00 2001
From: Richard Goodwin <github@richardgoodwin.com>
Date: Tue, 30 Oct 2018 14:51:12 -0500
Subject: [PATCH 06/29] trying BACKINGFILES_PARTITION_END_SPEC

---
 setup/pi/create-backingfiles-partition.sh | 8 ++++----
 1 file changed, 4 insertions(+), 4 deletions(-)

diff --git a/setup/pi/create-backingfiles-partition.sh b/setup/pi/create-backingfiles-partition.sh
index 14a81ae..0849974 100644
--- a/setup/pi/create-backingfiles-partition.sh
+++ b/setup/pi/create-backingfiles-partition.sh
@@ -13,7 +13,7 @@ BACKINGFILES_MOUNTPOINT="$1"
 MUTABLE_MOUNTPOINT="$2"
 
 setup_progress "Checking existing partitions..."
-PARTITION_TABLE=$(parted -s -m /dev/mmcblk0 unit B print)
+PARTITION_TABLE=$(parted -m /dev/mmcblk0 unit B print)
 DISK_LINE=$(echo "$PARTITION_TABLE" | grep -e "^/dev/mmcblk0:")
 DISK_SIZE=$(echo "$DISK_LINE" | cut -d ":" -f 2 | sed 's/B//' )
 
@@ -27,13 +27,13 @@ ORIGINAL_DISK_IDENTIFIER=$( fdisk -l /dev/mmcblk0 | grep -e "^Disk identifier" |
 
 setup_progress "Modifying partition table for backing files partition..."
 BACKINGFILES_PARTITION_END_SPEC="$(( $LAST_BACKINGFILES_PARTITION_DESIRED_BYTE / 1000000 ))M"
-parted -s -a optimal -m /dev/mmcblk0 unit B mkpart primary ext4 "$FIRST_BACKINGFILES_PARTITION_BYTE" "$BACKINGFILES_PARTITION_END_SPEC"
+parted -a optimal -m /dev/mmcblk0 unit B mkpart primary ext4 "$FIRST_BACKINGFILES_PARTITION_BYTE" "$BACKINGFILES_PARTITION_END_SPEC"
 
 LAST_BACKINGFILES_PARTITION_BYTE=$(parted -m /dev/mmcblk0 unit B print | grep -e "^3:" | cut -d ":" -f 3 | sed 's/B//g' )
 
 setup_progress "Modifying partition table for mutable (writable) partition for script usage..."
-MUTABLE_PARTITION_START_SPEC="$(( $LAST_BACKINGFILES_PARTITION_BYTE / 1000000 ))M"
-parted -s -a optimal -m /dev/mmcblk0 unit B mkpart primary ext4 "$MUTABLE_PARTITION_START_SPEC" 100%
+MUTABLE_PARTITION_START_SPEC="$BACKINGFILES_PARTITION_END_SPEC"
+parted  -a optimal -m /dev/mmcblk0 unit B mkpart primary ext4 "$MUTABLE_PARTITION_START_SPEC" 100%
 
 NEW_DISK_IDENTIFIER=$( fdisk -l /dev/mmcblk0 | grep -e "^Disk identifier" | sed "s/Disk identifier: 0x//" )
 

From 589a16f833de9c5ceff82ac0966af18ae28b255f Mon Sep 17 00:00:00 2001
From: Richard Goodwin <github@richardgoodwin.com>
Date: Tue, 30 Oct 2018 15:15:11 -0500
Subject: [PATCH 07/29] Removed this readme since it's been moved.

---
 headless-scripts/Readme.md | 81 --------------------------------------
 1 file changed, 81 deletions(-)
 delete mode 100644 headless-scripts/Readme.md

diff --git a/headless-scripts/Readme.md b/headless-scripts/Readme.md
deleted file mode 100644
index 2eedbaa..0000000
--- a/headless-scripts/Readme.md
+++ /dev/null
@@ -1,81 +0,0 @@
-# Flashable image to get started more quickly
-
-# This is a WORK IN PROGRESS, SHOULD BE WORKING. 
-
-
-## Notes 
-
-* Assumes your Pi has access to Wifi, with internet access (during setup). (But all setup methods do currently.) USB networking still enabled for troubleshooting.
-
-
-## Configure the SD card before first boot of the Pi
-
-
-1. Flash the image from [here (10-24 dated image)](https://www.dropbox.com/s/6f8kxenvtz8pkj9/image_2018-10-24-teslausb_headless-lite.zip?dl=0) using Etcher or similar.  (Be sure to click the `...` on Dropbox and download the `.zip` file.)
-1. Mount the card again, and in the `boot` directory create a `teslausb_setup_variables.conf` file to export the same environment varibles normally needed for setup (including archive, Wifi, and push notifications (if desired).) A sample conf file is in this repo.  I.e. file should contain at a minimum (replace with your own values):
-    ```
-    export archiveserver=Nautilus
-    export sharename=SailfishCam
-    export shareuser=sailfish
-    export sharepassword=pa$$w0rd
-    export campercent=100
-    export SSID=your_ssid
-    export WIFIPASS=your_wifi_password
-    # export pushover_enabled=false
-    # export pushover_user_key=user_key
-    # export pushover_app_key=app_key
-    ```
-    (Pushover should be working but commented out by default.)
-* Boot it in your Pi, give it a bit, watching for a series of flashes (2, 3, 4, 5, maybe 6) and then a reboot and/or the CAM to become available on your PC/Mac.
-* The Pi should be available at `teslausb.local` over Wifi (if it works) or USB networking (if it doesn't). Takes about 5 minutes for me. You should see in `/boot` the TESLAUSB_SETUP_FINISHED and WIFI_ENABLED files as markers of success too.
-* Currently doesn't create the TeslaCam folder, so you'll need to do that before taking to your car.
-* If plugged into just a power source, or your car, give it a few minutes until the LED starts pulsing steadily which means the archive loop is running and you're good to go. 
-
-## What happens under the covers
-
-When the Pi boots the first time: 
-* A `/boot/teslausb-headless-setup.log` file will be created and stages logged. This takes the place of the "STOP" commands 
-* Marker files will be created in `boot` like `TESLA_USB_SETUP_STARTED` and `TESLA_USB_SETUP_FINISHED` to track progress. 
-* (Working on a progress system so the script can pick back up if needed. This is probably useful for the general/old way of setup too.)
-* Wifi is detected by looking for `/boot/WIFI_ENABLED` and if not, creates the `wpa_supplicant.conf` file in place and reboots. 
-* The Pi LED will flash patterns (2, 3, 4, 5, maybe 6) as it gets to each stage (labeled in the setup-teslausb-headless script). 
-  * 10 flashes means setup failed!
-  * After the final stage and reboot the LED will go back to normal. Remember, the step to remount the filesystem takes a few minutes.
-
-At this point the next boot should start the Dashcam/music drives like normal. If you're watching the LED it will start flashing every 1 second, which is the archive loop running. 
-
-> NOTE: Don't delete the `TESLAUSB_SETUP_FINISHED` or `WIFI_ENABLED` files. This is how the system knows setup is complete. 
-
-### Image builder source and patches
-
-For now the image creation work is at:
-* Modified pi-gen [rtgoodwin's fork of pi-gen](https://github.com/rtgoodwin/pi-gen) in (whatever current branch I'm working at the time). 
-* `headless-patch` branch of rtgoodwin fork [https://github.com/rtgoodwin/teslausb/tree/headless-patch/headless-scripts](https://github.com/rtgoodwin/teslausb/tree/headless-patch/headless-scripts)
-
-
-### Image creation TODOs
-1. Patch the hostname to teslausb
-1. Make it so if someone deletes the `TESLAUSB_SETUP_FINISHED` file it's handled gracefully. 
-1. I still see some errors during pi-gen about locale, may need to be fixed? stage0/01-locale/debconf en_US.UTF-8
-1. Cache the remount packages? Might mess with first boot like `rsyslog`
-1. Any other steps to move into the base image?
-1. Aspirational TODO: Remove more packages and set services to stopped to make the boot process faster? 
-1. NOTE: I moved all script downloads and variable creation to the initial setup. At this point, I'm designing it to pull the setup scripts dynamically, since development is still ongoing. If/when we reach a good frozen state, we can generate an image that is ready to run. I think it'll also be pretty tricky to do some of the remounting and creating the backing files etc. on the image creation side. Open to suggestions/contributions there though! At the very least we could bake in stable first stage headlessBuild scripts for Mac/Linux/Windows.
-
-
-#### Modifications to pi-gen builder from master
-
-Built image on a Raspi running Stretch, for maximum Pi-ception. 
-
-1. Add SKIP and SKIP_IMAGES files to stage3, 4, and 5 (if present). 
-1. Add a stage6. (There are stages0-5, but may be a stage5 in some cases. This will help keep a clean merge later.)
-1. Copy the prerun.sh from `stage2`. Be SURE to mark `chmod +x` it. 
-1. Remove or rename the EXPORT_NOOBS files in all stages. We don't need a NOOBS image built. 
-1. In `stage6`, create a `00-tweaks` folder, with a `00-patches` folder and patch inside to patch `cmdline.txt` to remove the resize and add the needed modules. The build process uses `quilt` for patching. Note: the path for any patching you do at this stage is `stage6/rootfs/FILEPATH` where `rootfs` represents the Pi's `/`. So, `cmdline.txt` is `stage6/rootfs/boot/cmdline.txt`.
-1. Add a patch for the `config.txt` file.
-1. Add a file called `series` in the patches directory with the name of each `.diff` file in the order you want them applied.
-1. Add a `files` folder in stage6 with modified `rc.local`. The modified `rc.local` will handle pulling down the `setup-teslausb-headless` file the first time. (Still working on build logic here.) Files are moved into final locations in a `00-run.sh` script and the `install` command. See the script for details. 
-1. (Yes at this point you could suggest that just putting the end state files in place instead of patching would be good, but why not be idiomatic? :)  )
-1. Add a script to flash LEDs
-1. Run `sudo ./build.sh` from the `pi-gen` directory.
-1. If you get a failure, it's almost certainly after stage2, so you can add SKIP files in stage2-stage5 present) and rerun `sudo CLEAN=1 ./build.sh`
\ No newline at end of file

From 9e5324f327bbeb7ef34c2f2d6a5b85e9fc748b4e Mon Sep 17 00:00:00 2001
From: Richard Goodwin <github@richardgoodwin.com>
Date: Tue, 30 Oct 2018 15:18:23 -0500
Subject: [PATCH 08/29] Missed a default variable setting

I hate this whole unbound variable protection mode, but it sure does catch bugs. :)
---
 setup/pi/create-backingfiles-partition.sh | 2 ++
 1 file changed, 2 insertions(+)

diff --git a/setup/pi/create-backingfiles-partition.sh b/setup/pi/create-backingfiles-partition.sh
index 0849974..170c873 100644
--- a/setup/pi/create-backingfiles-partition.sh
+++ b/setup/pi/create-backingfiles-partition.sh
@@ -1,5 +1,7 @@
 #!/bin/bash -eu
 
+HEADLESS_SETUP=${HEADLESS_SETUP:-false}
+
 function setup_progress () {
   SETUP_LOGFILE=/boot/teslausb-headless-setup.log
   if [ $HEADLESS_SETUP = "true" ]

From 215556acc86e88d1e9d0709ed1bfb4e5d9e046c0 Mon Sep 17 00:00:00 2001
From: Richard Goodwin <github@richardgoodwin.com>
Date: Tue, 30 Oct 2018 15:31:27 -0500
Subject: [PATCH 09/29] Make HEADLESS variable check local

---
 setup/pi/create-backingfiles-partition.sh | 3 ++-
 1 file changed, 2 insertions(+), 1 deletion(-)

diff --git a/setup/pi/create-backingfiles-partition.sh b/setup/pi/create-backingfiles-partition.sh
index 170c873..c0ae689 100644
--- a/setup/pi/create-backingfiles-partition.sh
+++ b/setup/pi/create-backingfiles-partition.sh
@@ -1,9 +1,10 @@
 #!/bin/bash -eu
 
-HEADLESS_SETUP=${HEADLESS_SETUP:-false}
+
 
 function setup_progress () {
   SETUP_LOGFILE=/boot/teslausb-headless-setup.log
+  HEADLESS_SETUP=${HEADLESS_SETUP:-false}
   if [ $HEADLESS_SETUP = "true" ]
   then
     echo "$( date ) : $1" >> "$SETUP_LOGFILE"

From 5443c0a607af5127fcefb420deb654f41aeef7e9 Mon Sep 17 00:00:00 2001
From: cimryan <cimryan@users.noreply.github.com>
Date: Tue, 30 Oct 2018 15:25:19 -0700
Subject: [PATCH 10/29] Remove vestigial LAST_BACKINGFILES_PARTITION_BYTE
 variable

---
 setup/pi/create-backingfiles-partition.sh | 2 --
 1 file changed, 2 deletions(-)

diff --git a/setup/pi/create-backingfiles-partition.sh b/setup/pi/create-backingfiles-partition.sh
index c0ae689..3b658a6 100644
--- a/setup/pi/create-backingfiles-partition.sh
+++ b/setup/pi/create-backingfiles-partition.sh
@@ -32,8 +32,6 @@ setup_progress "Modifying partition table for backing files partition..."
 BACKINGFILES_PARTITION_END_SPEC="$(( $LAST_BACKINGFILES_PARTITION_DESIRED_BYTE / 1000000 ))M"
 parted -a optimal -m /dev/mmcblk0 unit B mkpart primary ext4 "$FIRST_BACKINGFILES_PARTITION_BYTE" "$BACKINGFILES_PARTITION_END_SPEC"
 
-LAST_BACKINGFILES_PARTITION_BYTE=$(parted -m /dev/mmcblk0 unit B print | grep -e "^3:" | cut -d ":" -f 3 | sed 's/B//g' )
-
 setup_progress "Modifying partition table for mutable (writable) partition for script usage..."
 MUTABLE_PARTITION_START_SPEC="$BACKINGFILES_PARTITION_END_SPEC"
 parted  -a optimal -m /dev/mmcblk0 unit B mkpart primary ext4 "$MUTABLE_PARTITION_START_SPEC" 100%

From f8241bbbba8da91434cb56c3f6c4668a0b226df7 Mon Sep 17 00:00:00 2001
From: cimryan <cimryan@users.noreply.github.com>
Date: Tue, 30 Oct 2018 15:30:49 -0700
Subject: [PATCH 11/29] Make setup_logfile and headless_setup local to
 setup_progress

---
 setup/pi/create-backingfiles-partition.sh | 10 ++++------
 1 file changed, 4 insertions(+), 6 deletions(-)

diff --git a/setup/pi/create-backingfiles-partition.sh b/setup/pi/create-backingfiles-partition.sh
index 3b658a6..81a29b1 100644
--- a/setup/pi/create-backingfiles-partition.sh
+++ b/setup/pi/create-backingfiles-partition.sh
@@ -1,13 +1,11 @@
 #!/bin/bash -eu
 
-
-
 function setup_progress () {
-  SETUP_LOGFILE=/boot/teslausb-headless-setup.log
-  HEADLESS_SETUP=${HEADLESS_SETUP:-false}
-  if [ $HEADLESS_SETUP = "true" ]
+  local setup_logfile=/boot/teslausb-headless-setup.log
+  local headless_setup=${HEADLESS_SETUP:-false}
+  if [ $headless_setup = "true" ]
   then
-    echo "$( date ) : $1" >> "$SETUP_LOGFILE"
+    echo "$( date ) : $1" >> "$setup_logfile"
   fi
     echo $1
 }

From 0588442962a17f884e016a23ec4aa52fcfce9b4e Mon Sep 17 00:00:00 2001
From: cimryan <cimryan@users.noreply.github.com>
Date: Tue, 30 Oct 2018 15:35:00 -0700
Subject: [PATCH 12/29] Remove extra declaration and initialization of
 SETUP_LOGFILE

---
 setup/pi/setup-teslausb | 3 ---
 1 file changed, 3 deletions(-)

diff --git a/setup/pi/setup-teslausb b/setup/pi/setup-teslausb
index b7561cd..5877020 100644
--- a/setup/pi/setup-teslausb
+++ b/setup/pi/setup-teslausb
@@ -1,14 +1,11 @@
 #!/bin/bash -eu
 
 USER_ENABLED_PUSHOVER=${USER_ENABLED_PUSHOVER:-false}
-SETUP_LOGFILE=/boot/teslausb-headless-setup.log
 REPO=${REPO:-cimryan}
 BRANCH=${BRANCH:-master}
 HEADLESS_SETUP=${HEADLESS_SETUP:-false}
 USE_LED_FOR_SETUP_PROGRESS=true
 
-
-
 if ! [ $(id -u) = 0 ]
 then
   setup_progress "STOP: Run sudo -i."

From 74808c954193a4abfe1478947a4869eb7b480186 Mon Sep 17 00:00:00 2001
From: cimryan <cimryan@users.noreply.github.com>
Date: Tue, 30 Oct 2018 15:36:18 -0700
Subject: [PATCH 13/29] Make setup_logfile local to setup_progress

---
 setup/pi/setup-teslausb | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)

diff --git a/setup/pi/setup-teslausb b/setup/pi/setup-teslausb
index 5877020..4cdf3f2 100644
--- a/setup/pi/setup-teslausb
+++ b/setup/pi/setup-teslausb
@@ -13,10 +13,10 @@ then
 fi
 
 function setup_progress () {
-  SETUP_LOGFILE=/boot/teslausb-headless-setup.log
+  local setup_logfile=/boot/teslausb-headless-setup.log
   if [ $HEADLESS_SETUP = "true" ]
   then
-    echo "$( date ) : $1" >> "$SETUP_LOGFILE"
+    echo "$( date ) : $1" >> "$setup_logfile"
   fi
     echo $1
 }

From bb9347b1153de9906b4876aabbacc7a5ad6a3c98 Mon Sep 17 00:00:00 2001
From: Ray Tanaka <ray@poynt.co>
Date: Tue, 30 Oct 2018 16:44:38 -0700
Subject: [PATCH 14/29] since this is an undocumented configuration variable
 we'll move this into the main cifs config

---
 run/cifs_archive/configure-archive.sh | 2 ++
 1 file changed, 2 insertions(+)

diff --git a/run/cifs_archive/configure-archive.sh b/run/cifs_archive/configure-archive.sh
index f6c4fe4..a3e4ebd 100644
--- a/run/cifs_archive/configure-archive.sh
+++ b/run/cifs_archive/configure-archive.sh
@@ -12,6 +12,8 @@ function configure_archive () {
     mkdir "$archive_path"  
   fi
 
+  local cifs_version="${cifs_version:-3.0}"
+
   local credentials_file_path="/root/.teslaCamArchiveCredentials"
   echo "username=$shareuser" > "$credentials_file_path"
   echo "password=$sharepassword" >> "$credentials_file_path"

From 7c3a5f4b94e175acf851be09dabc181cfb99bff3 Mon Sep 17 00:00:00 2001
From: Ray Tanaka <ray@poynt.co>
Date: Tue, 30 Oct 2018 18:05:35 -0700
Subject: [PATCH 15/29] change this function to be more generic and taken in
 server name as an argument instead of a environment variable

---
 run/get-archiveserver-ip-address.sh | 3 ---
 run/lookup-ip-address.sh            | 3 +++
 2 files changed, 3 insertions(+), 3 deletions(-)
 delete mode 100644 run/get-archiveserver-ip-address.sh
 create mode 100644 run/lookup-ip-address.sh

diff --git a/run/get-archiveserver-ip-address.sh b/run/get-archiveserver-ip-address.sh
deleted file mode 100644
index 8a3936e..0000000
--- a/run/get-archiveserver-ip-address.sh
+++ /dev/null
@@ -1,3 +0,0 @@
-#!/bin/bash -eu
-
-echo "$(ping -c 1 -w 1 $archiveserver 2>/dev/null | head -n 1 | grep -o -e "(\([[:digit:]]\{1,3\}\.\)\{3\}[[:digit:]]\{1,3\})" | tr -d '()')"
\ No newline at end of file
diff --git a/run/lookup-ip-address.sh b/run/lookup-ip-address.sh
new file mode 100644
index 0000000..41b8f5a
--- /dev/null
+++ b/run/lookup-ip-address.sh
@@ -0,0 +1,3 @@
+#!/bin/bash -eu
+
+echo "$(ping -c 1 -w 1 $1 2>/dev/null | head -n 1 | grep -o -e "(\([[:digit:]]\{1,3\}\.\)\{3\}[[:digit:]]\{1,3\})" | tr -d '()')"
\ No newline at end of file

From 777db26f27785725dc0bdf32a078e5fedb584826 Mon Sep 17 00:00:00 2001
From: Ray Tanaka <ray@poynt.co>
Date: Tue, 30 Oct 2018 22:53:51 -0700
Subject: [PATCH 16/29] testing to see if verify-archive works as expected

---
 run/cifs_archive/verify-archive-configuration.sh | 13 +++++++++----
 1 file changed, 9 insertions(+), 4 deletions(-)

diff --git a/run/cifs_archive/verify-archive-configuration.sh b/run/cifs_archive/verify-archive-configuration.sh
index 684a42b..4db1830 100644
--- a/run/cifs_archive/verify-archive-configuration.sh
+++ b/run/cifs_archive/verify-archive-configuration.sh
@@ -24,9 +24,14 @@ function check_archive_mountable () {
     mkdir "$test_mount_location"
   fi
   
-  local tmp_credentials_file_path="/root/.teslaCamArchiveCredentials"
+  local cifs_version="${cifs_version:-3.0}"
+
+  local tmp_credentials_file_path="/tmp/.teslaCamArchiveCredentials"
+  echo "username=$shareuser" > "$credentials_file_path"
+  echo "password=$sharepassword" >> "$credentials_file_path"
 
   local mount_failed=false
+  echo "//$archive_server_ip_address/$sharename" "$test_mount_location" -o "vers=${cifs_version},credentials=${tmp_credentials_file_path},iocharset=utf8,file_mode=0777,dir_mode=0777"
   mount -t cifs "//$archive_server_ip_address/$sharename" "$test_mount_location" -o "vers=${cifs_version},credentials=${tmp_credentials_file_path},iocharset=utf8,file_mode=0777,dir_mode=0777" || mount_failed=true
 
   if [ "$mount_failed" = true ]
@@ -40,8 +45,8 @@ function check_archive_mountable () {
   umount "$test_mount_location"
 }
 
+echo "install dir: $INSTALL_DIR"
+ARCHIVE_SERVER_IP_ADDRESS="$( $INSTALL_DIR/lookup-ip-address.sh "$archiveserver" )"
+
 check_archive_server_reachable
-
-ARCHIVE_SERVER_IP_ADDRESS="$( /root/bin/get-archiveserver-ip-address.sh )"
-
 check_archive_mountable "$ARCHIVE_SERVER_IP_ADDRESS"

From 9c0c8ad09c0dc384ced8d5400bfa2cbb1f3d8e89 Mon Sep 17 00:00:00 2001
From: Ray Tanaka <ray@poynt.co>
Date: Tue, 30 Oct 2018 22:59:51 -0700
Subject: [PATCH 17/29] bug fix

---
 run/cifs_archive/verify-archive-configuration.sh | 5 ++---
 1 file changed, 2 insertions(+), 3 deletions(-)

diff --git a/run/cifs_archive/verify-archive-configuration.sh b/run/cifs_archive/verify-archive-configuration.sh
index 4db1830..6968f02 100644
--- a/run/cifs_archive/verify-archive-configuration.sh
+++ b/run/cifs_archive/verify-archive-configuration.sh
@@ -27,8 +27,8 @@ function check_archive_mountable () {
   local cifs_version="${cifs_version:-3.0}"
 
   local tmp_credentials_file_path="/tmp/.teslaCamArchiveCredentials"
-  echo "username=$shareuser" > "$credentials_file_path"
-  echo "password=$sharepassword" >> "$credentials_file_path"
+  echo "username=$shareuser" > "$tmp_credentials_file_path"
+  echo "password=$sharepassword" >> "$tmp_credentials_file_path"
 
   local mount_failed=false
   echo "//$archive_server_ip_address/$sharename" "$test_mount_location" -o "vers=${cifs_version},credentials=${tmp_credentials_file_path},iocharset=utf8,file_mode=0777,dir_mode=0777"
@@ -45,7 +45,6 @@ function check_archive_mountable () {
   umount "$test_mount_location"
 }
 
-echo "install dir: $INSTALL_DIR"
 ARCHIVE_SERVER_IP_ADDRESS="$( $INSTALL_DIR/lookup-ip-address.sh "$archiveserver" )"
 
 check_archive_server_reachable

From 9dd3b135c5e709197671a5db09c2b76c09217483 Mon Sep 17 00:00:00 2001
From: Ray Tanaka <ray@poynt.co>
Date: Tue, 30 Oct 2018 23:11:41 -0700
Subject: [PATCH 18/29] - removing echo - using new ip lookup function

---
 run/cifs_archive/configure-archive.sh            | 2 +-
 run/cifs_archive/verify-archive-configuration.sh | 1 -
 2 files changed, 1 insertion(+), 2 deletions(-)

diff --git a/run/cifs_archive/configure-archive.sh b/run/cifs_archive/configure-archive.sh
index a3e4ebd..f4204a9 100644
--- a/run/cifs_archive/configure-archive.sh
+++ b/run/cifs_archive/configure-archive.sh
@@ -23,6 +23,6 @@ function configure_archive () {
   echo "Configured the archive."
 }
 
-ARCHIVE_SERVER_IP_ADDRESS="$( /root/bin/get-archiveserver-ip-address.sh )"
+ARCHIVE_SERVER_IP_ADDRESS="$( $INSTALL_DIR/lookup-ip-address.sh "$archiveserver" )"
 
 configure_archive "$ARCHIVE_SERVER_IP_ADDRESS"
\ No newline at end of file
diff --git a/run/cifs_archive/verify-archive-configuration.sh b/run/cifs_archive/verify-archive-configuration.sh
index 6968f02..6b6a7a1 100644
--- a/run/cifs_archive/verify-archive-configuration.sh
+++ b/run/cifs_archive/verify-archive-configuration.sh
@@ -31,7 +31,6 @@ function check_archive_mountable () {
   echo "password=$sharepassword" >> "$tmp_credentials_file_path"
 
   local mount_failed=false
-  echo "//$archive_server_ip_address/$sharename" "$test_mount_location" -o "vers=${cifs_version},credentials=${tmp_credentials_file_path},iocharset=utf8,file_mode=0777,dir_mode=0777"
   mount -t cifs "//$archive_server_ip_address/$sharename" "$test_mount_location" -o "vers=${cifs_version},credentials=${tmp_credentials_file_path},iocharset=utf8,file_mode=0777,dir_mode=0777" || mount_failed=true
 
   if [ "$mount_failed" = true ]

From d908f7caed75b23ad8ebe7a885dff53a191f23f8 Mon Sep 17 00:00:00 2001
From: Ray Tanaka <ray@poynt.co>
Date: Wed, 31 Oct 2018 00:00:05 -0700
Subject: [PATCH 19/29] addig rclone verification step

---
 run/rclone_archive/verify-archive-configuration.sh | 13 ++++++++++++-
 1 file changed, 12 insertions(+), 1 deletion(-)

diff --git a/run/rclone_archive/verify-archive-configuration.sh b/run/rclone_archive/verify-archive-configuration.sh
index 95dbbf8..7b89e75 100644
--- a/run/rclone_archive/verify-archive-configuration.sh
+++ b/run/rclone_archive/verify-archive-configuration.sh
@@ -1 +1,12 @@
-#!/bin/bash -eu
\ No newline at end of file
+#!/bin/bash -eu
+
+function verify_configuration () {
+    echo "Verifying rlcone configuration..."
+    if ! rclone lsd "$RCLONE_DRIVE": | grep -q "$RCLONE_PATH"
+    then
+        echo "STOP: Could not find the $RCLONE_DRIVE:$RCLONE_PATH"
+        exit 1
+    fi
+}
+
+verify_configuration
\ No newline at end of file

From 5e2b0812464c6d986ad60626e330ac87745e1306 Mon Sep 17 00:00:00 2001
From: Ray Tanaka <ray@poynt.co>
Date: Wed, 31 Oct 2018 00:25:45 -0700
Subject: [PATCH 20/29] only move fake-hwclock.data if it's not already a
 symlink take out rclone specific logic out of here and move into
 rclone_archive

---
 setup/pi/make-root-fs-readonly.sh | 14 ++++----------
 1 file changed, 4 insertions(+), 10 deletions(-)

diff --git a/setup/pi/make-root-fs-readonly.sh b/setup/pi/make-root-fs-readonly.sh
index aeafeb0..ea88422 100644
--- a/setup/pi/make-root-fs-readonly.sh
+++ b/setup/pi/make-root-fs-readonly.sh
@@ -33,25 +33,19 @@ if [ ! -e "/mutable/etc" ]
 then
     mkdir -p /mutable/etc
 fi
-if [ -e "/etc/fake-hwclock.data" ]
+
+if [ ! -L "/etc/fake-hwclock.data" ] && [ -e "/etc/fake-hwclock.data" ]
 then
     echo "Moving fake-hwclock data"
-    cp /etc/fake-hwclock.data /mutable/etc/fake-hwclock.data
-    rm /etc/fake-hwclock.data
+    mv /etc/fake-hwclock.data /mutable/etc/fake-hwclock.data
     ln -s /mutable/etc/fake-hwclock.data /etc/fake-hwclock.data
 fi
 
-# Move rclone configs if it exists so we can write to it
+# Create a configs directory for others to use
 if [ ! -e "/mutable/configs" ]
 then
     mkdir -p /mutable/configs
 fi
-if [ -e "/root/.config/rclone/rclone.conf" ]
-then
-    echo "Moving rclone configs"
-    mv /root/.config/rclone /mutable/configs
-    ln -s /mutable/configs/rclone /root/.config/rclone
-fi
 
 # Move /var/spool to /tmp
 rm -rf /var/spool

From 34690ca9d408d66cb7adb48245204bc09c9954c1 Mon Sep 17 00:00:00 2001
From: Ray Tanaka <ray@poynt.co>
Date: Wed, 31 Oct 2018 00:36:27 -0700
Subject: [PATCH 21/29] add logic to move config file into /mutable as part of
 configuring the archive

---
 run/rclone_archive/configure-archive.sh | 9 ++++++++-
 1 file changed, 8 insertions(+), 1 deletion(-)

diff --git a/run/rclone_archive/configure-archive.sh b/run/rclone_archive/configure-archive.sh
index 13e291b..f204c94 100644
--- a/run/rclone_archive/configure-archive.sh
+++ b/run/rclone_archive/configure-archive.sh
@@ -1,10 +1,17 @@
 #!/bin/bash -eu
 
 function configure_archive () {
-  echo "Configuring the archive for Rclone..."
+  echo "Configuring rclone archive..."
   
   echo "drive=$RCLONE_DRIVE" > /root/.teslaCamRcloneConfig
   echo "path=$RCLONE_PATH" >> /root/.teslaCamRcloneConfig
+
+  if [ ! -L "/root/.config/rclone" ] && [ -e "/root/.config/rclone" ]
+  then
+    echo "Moving rclone configs in /mutable"
+    mv /root/.config/rclone /mutable/configs
+    ln -s /mutable/configs/rclone /root/.config/rclone
+  fi
 }
 
 configure_archive
\ No newline at end of file

From fd4d161f1a28227dd96f80175248bef1104da138 Mon Sep 17 00:00:00 2001
From: Ray Tanaka <ray@poynt.co>
Date: Wed, 31 Oct 2018 00:37:11 -0700
Subject: [PATCH 22/29] also check for configuration file as part of
 verification step

---
 run/rclone_archive/verify-archive-configuration.sh | 6 ++++++
 1 file changed, 6 insertions(+)

diff --git a/run/rclone_archive/verify-archive-configuration.sh b/run/rclone_archive/verify-archive-configuration.sh
index 7b89e75..38c5303 100644
--- a/run/rclone_archive/verify-archive-configuration.sh
+++ b/run/rclone_archive/verify-archive-configuration.sh
@@ -2,6 +2,12 @@
 
 function verify_configuration () {
     echo "Verifying rlcone configuration..."
+    if ! [ -e "/root/.config/rclone/rclone.conf" ]
+    then
+        echo "STOP: rclone config was not found. did you configure rclone correctly?"
+        exit 1
+    fi
+
     if ! rclone lsd "$RCLONE_DRIVE": | grep -q "$RCLONE_PATH"
     then
         echo "STOP: Could not find the $RCLONE_DRIVE:$RCLONE_PATH"

From 203ad8b1b5eadad4a2587c443a63e5ac4c696c4a Mon Sep 17 00:00:00 2001
From: Ray Tanaka <ray@poynt.co>
Date: Wed, 31 Oct 2018 01:11:24 -0700
Subject: [PATCH 23/29] cleaning up config functions.

---
 run/rclone_archive/configure-archive.sh | 2 ++
 run/rsync_archive/configure-archive.sh  | 9 ++-------
 2 files changed, 4 insertions(+), 7 deletions(-)

diff --git a/run/rclone_archive/configure-archive.sh b/run/rclone_archive/configure-archive.sh
index f204c94..fd9eb84 100644
--- a/run/rclone_archive/configure-archive.sh
+++ b/run/rclone_archive/configure-archive.sh
@@ -12,6 +12,8 @@ function configure_archive () {
     mv /root/.config/rclone /mutable/configs
     ln -s /mutable/configs/rclone /root/.config/rclone
   fi
+
+  echo "done"
 }
 
 configure_archive
\ No newline at end of file
diff --git a/run/rsync_archive/configure-archive.sh b/run/rsync_archive/configure-archive.sh
index 4fcb84c..9d94ff5 100644
--- a/run/rsync_archive/configure-archive.sh
+++ b/run/rsync_archive/configure-archive.sh
@@ -1,16 +1,11 @@
 #!/bin/bash -eu
 
 function configure_archive () {
-  local archive_server_ip_address="$1"
+  echo "Configuring the rsync archive..."
 
-  echo "Configuring the archive..."
-  
-  echo "Configuring for Rsync..."
   echo "user=$RSYNC_USER" > /root/.teslaCamRsyncConfig
   echo "server=$RSYNC_SERVER" >> /root/.teslaCamRsyncConfig
   echo "path=$RSYNC_PATH" >> /root/.teslaCamRsyncConfig
 }
 
-ARCHIVE_SERVER_IP_ADDRESS="$( /root/bin/get-archiveserver-ip-address.sh )"
-
-configure_archive "$ARCHIVE_SERVER_IP_ADDRESS"
\ No newline at end of file
+configure_archive
\ No newline at end of file

From aae15da2e51f031c8cc285bf5d7d01d3a5924c5c Mon Sep 17 00:00:00 2001
From: Ray Tanaka <ray@poynt.co>
Date: Wed, 31 Oct 2018 01:40:20 -0700
Subject: [PATCH 24/29] case fix because i'm ocd

---
 run/rclone_archive/configure-archive.sh | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/run/rclone_archive/configure-archive.sh b/run/rclone_archive/configure-archive.sh
index fd9eb84..6523b2c 100644
--- a/run/rclone_archive/configure-archive.sh
+++ b/run/rclone_archive/configure-archive.sh
@@ -13,7 +13,7 @@ function configure_archive () {
     ln -s /mutable/configs/rclone /root/.config/rclone
   fi
 
-  echo "done"
+  echo "Done"
 }
 
 configure_archive
\ No newline at end of file

From b93550e4dea1a01061b0faff63c6e54e0568fb9e Mon Sep 17 00:00:00 2001
From: Ray Tanaka <ray@poynt.co>
Date: Wed, 31 Oct 2018 01:42:25 -0700
Subject: [PATCH 25/29] splitting up usb configuration from script and archive
 configuration / installation. this will allow a user to run configuration
 multiple times.

---
 setup/pi/configure.sh   | 216 ++++++++++++++++++++++++++++++++++++++++
 setup/pi/setup-teslausb | 135 +------------------------
 2 files changed, 217 insertions(+), 134 deletions(-)
 create mode 100644 setup/pi/configure.sh

diff --git a/setup/pi/configure.sh b/setup/pi/configure.sh
new file mode 100644
index 0000000..e9336aa
--- /dev/null
+++ b/setup/pi/configure.sh
@@ -0,0 +1,216 @@
+#!/bin/bash -eu
+
+REPO=${REPO:-cimryan}
+BRANCH=${BRANCH:-master}
+INSTALL_DIR=${INSTALL_DIR:-/root/bin}
+
+upgrade=false   #TODO: create an option to just refresh the scripts
+debug_on=false
+
+function debug() {
+    $debug_on && echo "$1"
+    return 0
+}
+
+function check_variable () {
+    local var_name="$1"
+    if [ -z "${!var_name+x}" ]
+    then
+        echo "STOP: Define the variable $var_name like this: export $var_name=value"
+        exit 1
+    fi
+}
+
+function get_script () {
+    local local_path="$1"
+    local name="$2"
+    local remote_path="${3:-}"
+
+    echo "Starting download for $local_path/$name"
+    curl -o "$local_path/$name" https://raw.githubusercontent.com/"$REPO"/teslausb/"$BRANCH"/"$remote_path"/"$name"
+    chmod +x "$local_path/$name"
+    echo "Done"
+}
+
+function parse_command_line() {
+    echo -n "Parsing command line: "
+
+    local OPTIND=1         # Reset in case getopts has been used previously in the shell.
+    while getopts "du" opt; do
+        case "$opt" in
+        d)  debug_on=true
+            ;;
+        u)  upgrade=true
+            #TODO: just upgrade all the scripts instead building new configs
+            echo "STOP: this option is not yet implemented"
+            exit 1
+            ;;
+        esac
+    done
+    shift $((OPTIND-1))
+    [ "${1:-}" = "--" ] && shift
+
+    echo "done"
+}
+
+function install_rc_local () {
+    local install_home="$1"
+
+    if grep -q archiveloop /etc/rc.local
+    then
+        echo "Skipping rc.local installation"
+        return
+    fi
+
+    echo "Configuring /etc/rc.local to run the archive scripts at startup..."
+    echo "#!/bin/bash -eu" > ~/rc.local
+    echo "archiveserver=\"${archiveserver}\"" >> ~/rc.local
+    echo "install_home=\"${install_home}\"" >> ~/rc.local
+    cat << 'EOF' >> ~/rc.local
+LOGFILE=/tmp/rc.local.log
+
+function log () {
+  echo "$( date )" >> "$LOGFILE"
+  echo "$1" >> "$LOGFILE"
+}
+
+log "Launching archival script..."
+"$install_home"/archiveloop "$archiveserver" &
+log "All done"
+exit 0
+EOF
+
+    cat ~/rc.local > /etc/rc.local
+    rm ~/rc.local
+    echo "Installed rc.local."
+}
+
+function check_archive_configs () {
+    echo -n "Checking archive configs: "
+
+    RSYNC_ENABLE="${RSYNC_ENABLE:-false}"
+    RCLONE_ENABLE="${RCLONE_ENABLE:-false}"   
+    if [ "$RSYNC_ENABLE" = true ] && [ "$RCLONE_ENABLE" = true ]
+    then
+        echo "STOP: Can't enable rsync and rclone at the same time"
+        exit 1
+    fi
+
+    if [ "$RSYNC_ENABLE" = true ]
+    then
+        check_variable "RSYNC_USER"
+        check_variable "RSYNC_SERVER"
+        check_variable "RSYNC_PATH"
+        export archiveserver="$RSYNC_SERVER"
+        
+    elif [ "$RCLONE_ENABLE" = true ]
+    then
+        check_variable "RCLONE_DRIVE"
+        check_variable "RCLONE_PATH"
+        export archiveserver="8.8.8.8" # since it's a cloud hosted drive we'll just set this to google dns
+    else
+        # default to cifs
+        check_variable "sharename"
+        check_variable "shareuser"
+        check_variable "sharepassword"
+        check_variable "archiveserver"
+    fi
+
+    echo "done"
+}
+
+function get_archive_module () {
+
+    if [ "$RSYNC_ENABLE" = true ]
+    then
+        archive_module="run/rsync_archive"        
+    elif [ "$RCLONE_ENABLE" = true ]
+    then
+        archive_module="run/rclone_archive"
+    else
+        archive_module="run/cifs_archive"
+    fi
+
+    echo $archive_module
+}
+
+function install_archive_scripts () {
+    local install_path="$1"
+    local archive_module="$2"
+
+    echo "Installing base archive scripts into $install_path"
+    get_script $install_path archiveloop run
+    get_script $install_path remountfs_rw run
+    get_script $install_path lookup-ip-address.sh run
+
+    echo "Installing archive module scripts ($archive_module)"
+
+    get_script $install_path verify-archive-configuration.sh $archive_module
+    get_script $install_path configure-archive.sh $archive_module
+    get_script $install_path archive-clips.sh $archive_module
+    get_script $install_path connect-archive.sh $archive_module
+    get_script $install_path disconnect-archive.sh $archive_module
+}
+
+function check_and_configure_pushover () {
+    if [ ! -z "${pushover_enabled+x}" ]
+    then
+        if [ ! -n "${pushover_user_key+x}" ] || [ ! -n "${pushover_app_key+x}"  ]
+        then
+            echo "STOP: You're trying to setup Pushover but didn't provide your User and/or App key."
+            echo "Define the variables like this:"
+            echo "export pushover_user_key=put_your_userkey_here"
+            echo "export pushover_app_key=put_your_appkey_here"
+            exit 1
+        elif [ "${pushover_user_key}" = "put_your_userkey_here" ] || [  "${pushover_app_key}" = "put_your_appkey_here" ]
+        then
+            echo "STOP: You're trying to setup Pushover, but didn't replace the default User and App key values."
+            exit 1
+        else
+            echo "Enabling pushover"
+            echo "export pushover_enabled=true" > /root/.teslaCamPushoverCredentials
+            echo "export pushover_user_key=$pushover_user_key" >> /root/.teslaCamPushoverCredentials
+            echo "export pushover_app_key=$pushover_app_key" >> /root/.teslaCamPushoverCredentials
+        fi
+    else
+            echo "Pushover not configured."
+    fi
+}
+
+function install_pushover_scripts() {
+    local install_path="$1"
+    get_script $install_path send-pushover run
+}
+
+if ! [ $(id -u) = 0 ]
+then
+    echo "STOP: Run sudo -i."
+    exit 1
+fi
+
+if [ ! -e "$INSTALL_DIR" ]
+then
+    mkdir "$INSTALL_DIR"
+fi
+
+parse_command_line "$@"
+debug "Getting files from $REPO:$BRANCH"
+debug "- debug mode = $debug_on"
+debug "- upgrade only = $upgrade"
+
+check_and_configure_pushover
+install_pushover_scripts "$INSTALL_DIR"
+
+check_archive_configs
+
+archive_module="$( get_archive_module )"
+debug "Archive module: $archive_module"
+
+install_archive_scripts $INSTALL_DIR $archive_module
+"$INSTALL_DIR"/verify-archive-configuration.sh
+"$INSTALL_DIR"/configure-archive.sh
+
+install_rc_local "$INSTALL_DIR"
+
+
+
diff --git a/setup/pi/setup-teslausb b/setup/pi/setup-teslausb
index 0b52fe1..00bad1d 100644
--- a/setup/pi/setup-teslausb
+++ b/setup/pi/setup-teslausb
@@ -1,6 +1,5 @@
 #!/bin/bash -eu
 
-USER_ENABLED_PUSHOVER=${USER_ENABLED_PUSHOVER:-false}
 SETUP_LOGFILE=/boot/teslausb-headless-setup.log
 REPO=${REPO:-cimryan}
 BRANCH=${BRANCH:-master}
@@ -8,7 +7,6 @@ HEADLESS_SETUP=${HEADLESS_SETUP:-false}
 USE_LED_FOR_SETUP_PROGRESS=true
 
 
-
 if ! [ $(id -u) = 0 ]
 then
   setup_progress "STOP: Run sudo -i."
@@ -90,66 +88,6 @@ function check_variable () {
   fi
 }
 
-function check_pushover_enabled () {
-  if [ ! -z "${pushover_enabled+x}" ]
-  then
-    if [ ! -n "${pushover_user_key+x}" ] || [ ! -n "${pushover_app_key+x}"  ]
-    then
-      setup_progress "STOP: You're trying to setup Pushover but didn't provide your User and/or App key."
-      setup_progress "Define the variables like this:"
-      setup_progress "export pushover_user_key=put_your_userkey_here"
-      setup_progress "export pushover_app_key=put_your_appkey_here"
-      exit 1
-    elif [ "${pushover_user_key}" = "put_your_userkey_here" ] || [  "${pushover_app_key}" = "put_your_appkey_here" ]
-    then
-      setup_progress "STOP: You're trying to setup Pushover, but didn't replace the default User and App key values."
-      exit 1
-    else
-      USER_ENABLED_PUSHOVER=true
-      echo "export pushover_enabled=true" > /root/.teslaCamPushoverCredentials
-      echo "export pushover_user_key=$pushover_user_key" >> /root/.teslaCamPushoverCredentials
-      echo "export pushover_app_key=$pushover_app_key" >> /root/.teslaCamPushoverCredentials
-    fi
-  fi
-}
-
-function check_archive_configs () {
-
-  RSYNC_ENABLE="${RSYNC_ENABLE:-false}"
-  RCLONE_ENABLE="${RCLONE_ENABLE:-false}"
-
-  if [ "$RSYNC_ENABLE" = true ] && [ "$RCLONE_ENABLE" = true ]
-  then
-    setup_progress "STOP: Cannot enable rsync and rclone at the same time"
-    exit 1
-  fi
-
-  if [ "$RSYNC_ENABLE" = true ]
-  then
-    check_variable "RSYNC_USER"
-    check_variable "RSYNC_SERVER"
-    export archiveserver="$RSYNC_SERVER"
-    check_variable "RSYNC_PATH"
-    export archive_module="run/rsync_archive"
-  elif [ "$RCLONE_ENABLE" = true ]
-  then
-    check_variable "RCLONE_DRIVE"
-    check_variable "RCLONE_PATH"
-    # since it's a cloud hosted drive we'll just set this to google dns
-    export archiveserver="8.8.8.8"
-    export archive_module="run/rclone_archive"
-  else # Else for now, TODO allow both for more redundancy?
-    check_variable "sharename"
-    check_variable "shareuser"
-    check_variable "sharepassword"
-
-    export cifs_version="${cifs_version:-3.0}"
-    export archive_module="run/cifs_archive"
-  fi
-
-  check_variable "archiveserver"
-}
-
 function check_available_space () {
   setup_progress "Verifying that there is sufficient space available on the MicroSD card..."
 
@@ -179,6 +117,7 @@ function get_ancillary_setup_scripts () {
   get_script /tmp create-backingfiles-partition.sh setup/pi
   get_script /tmp create-backingfiles.sh setup/pi
   get_script /tmp make-root-fs-readonly.sh setup/pi
+  get_script /root configure.sh setup/pi
 }
 
 function fix_cmdline_txt_modules_load ()
@@ -223,59 +162,6 @@ function create_usb_drive_backing_files () {
   fi
 }
 
-function configure_archive_scripts () {
-  setup_progress "Configuring the archive scripts..."
-
-  get_script /root/bin archiveloop run
-  get_script /root/bin remountfs_rw run
-  
-  setup_progress "Configured the archive scripts."
-}
-
-function install_archive_scripts () {
-  check_variable "archive_module"
-  setup_progress "Installing scripts for archive module: $archive_module"
-
-  local install_path="/root/bin"
-  get_script $install_path verify-archive-configuration.sh $archive_module
-  get_script $install_path configure-archive.sh $archive_module
-  get_script $install_path archive-clips.sh $archive_module
-  get_script $install_path connect-archive.sh $archive_module
-  get_script $install_path disconnect-archive.sh $archive_module
-}
-
-function configure_pushover_scripts() {
-  get_script /root/bin send-pushover run
-}
-
-function configure_rc_local () {
-  if grep -q archiveloop /etc/rc.local
-  then
-    return
-  fi
-  
-  setup_progress "Configuring /etc/rc.local to run the archive scripts at startup..."
-  echo "#!/bin/bash -eu" > ~/rc.local
-  echo "archiveserver=\"${archiveserver}\"" >> ~/rc.local
-cat << 'EOF' >> ~/rc.local
-LOGFILE=/tmp/rc.local.log
-
-function log () {
-  echo "$( date )" >> "$LOGFILE"
-  echo "$1" >> "$LOGFILE"
-}
-
-log "Launching archival script..."
-/root/bin/archiveloop "$archiveserver" &
-log "All done"
-exit 0
-EOF
-
-  cat ~/rc.local > /etc/rc.local
-  rm ~/rc.local
-  setup_progress "Configured rc.local."
-}
-
 function configure_hostname () {
   # Headless image already has hostname set
   if [ ! $HEADLESS_SETUP = "true" ]
@@ -307,35 +193,18 @@ headless_setup_progress_flash 1
 setup_progress "Verifying environment variables..."
 
 check_variable "campercent"
-check_archive_configs
-check_pushover_enabled
 
 # Flash for Stage 3 headless (grab scripts)
 headless_setup_progress_flash 2
 
 setup_progress "Downloading additional setup scripts."
 
-if [ ! -e /root/bin ]
-then
-  mkdir /root/bin
-fi
-
-get_script /root/bin get-archiveserver-ip-address.sh run
-
-install_archive_scripts
-/root/bin/configure-archive.sh
-/root/bin/verify-archive-configuration.sh
-
 check_available_space
 
 get_ancillary_setup_scripts
 
 pushd ~
 
-configure_archive_scripts
-
-configure_pushover_scripts
-
 fix_cmdline_txt_modules_load
 
 echo "" >> /etc/fstab
@@ -345,8 +214,6 @@ headless_setup_progress_flash 3
 
 create_usb_drive_backing_files
 
-configure_rc_local
-
 configure_hostname
 
 # Flash for stage 5 headless (Mark success, FS readonly)

From 6dc1b79813784a8c062c2d14e9b024f6469c2dc1 Mon Sep 17 00:00:00 2001
From: Ray Tanaka <ray@poynt.co>
Date: Wed, 31 Oct 2018 02:07:18 -0700
Subject: [PATCH 26/29] bug fix - removing resolv.conf causes dns errors until
 you reboot. this is safer.

---
 setup/pi/make-root-fs-readonly.sh | 3 +--
 1 file changed, 1 insertion(+), 2 deletions(-)

diff --git a/setup/pi/make-root-fs-readonly.sh b/setup/pi/make-root-fs-readonly.sh
index ea88422..4ac638b 100644
--- a/setup/pi/make-root-fs-readonly.sh
+++ b/setup/pi/make-root-fs-readonly.sh
@@ -55,8 +55,7 @@ ln -s /tmp /var/spool
 sed -i "s/spool\s*0755/spool 1777/g" /usr/lib/tmpfiles.d/var.conf >/dev/null
 
 # Move dhcpd.resolv.conf to tmpfs
-touch /tmp/dhcpcd.resolv.conf
-rm /etc/resolv.conf
+mv /etc/resolv.conf /tmp/dhcpcd.resolv.conf
 ln -s /tmp/dhcpcd.resolv.conf /etc/resolv.conf
 
 # Update /etc/fstab

From 1e063e4aed1a5fc644effae69e034b60a1900604 Mon Sep 17 00:00:00 2001
From: Ray Tanaka <ray@poynt.co>
Date: Wed, 31 Oct 2018 02:28:38 -0700
Subject: [PATCH 27/29] adding new step to configure

---
 README.md | 1 +
 1 file changed, 1 insertion(+)

diff --git a/README.md b/README.md
index eabba21..4ed3acd 100644
--- a/README.md
+++ b/README.md
@@ -83,6 +83,7 @@ Follow the instructions corresponding to the technology you'd like to use to hos
     wget https://raw.githubusercontent.com/cimryan/teslausb/master/setup/pi/setup-teslausb
     chmod +x setup-teslausb
     ./setup-teslausb
+    /root/configure.sh
     ```
 1. Run this command:
     ```

From 9687859297ae1757483f680cdff2c08e244228ac Mon Sep 17 00:00:00 2001
From: Ray Tanaka <ray@poynt.co>
Date: Wed, 31 Oct 2018 03:26:39 -0700
Subject: [PATCH 28/29] typo

---
 run/rclone_archive/configure-archive.sh | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/run/rclone_archive/configure-archive.sh b/run/rclone_archive/configure-archive.sh
index 6523b2c..5740157 100644
--- a/run/rclone_archive/configure-archive.sh
+++ b/run/rclone_archive/configure-archive.sh
@@ -8,7 +8,7 @@ function configure_archive () {
 
   if [ ! -L "/root/.config/rclone" ] && [ -e "/root/.config/rclone" ]
   then
-    echo "Moving rclone configs in /mutable"
+    echo "Moving rclone configs into /mutable"
     mv /root/.config/rclone /mutable/configs
     ln -s /mutable/configs/rclone /root/.config/rclone
   fi

From 0d682af4bfa54052f62800d38da7884989974b25 Mon Sep 17 00:00:00 2001
From: Ray Tanaka <ray@poynt.co>
Date: Wed, 31 Oct 2018 03:27:26 -0700
Subject: [PATCH 29/29] let's clean up after ourselves

---
 setup/pi/setup-teslausb | 2 ++
 1 file changed, 2 insertions(+)

diff --git a/setup/pi/setup-teslausb b/setup/pi/setup-teslausb
index 23de664..460e8f2 100644
--- a/setup/pi/setup-teslausb
+++ b/setup/pi/setup-teslausb
@@ -172,10 +172,12 @@ function configure_hostname () {
     local new_host_name="teslausb"
     cp /etc/hosts ~
     sed "s/raspberrypi/$new_host_name/g" ~/hosts > /etc/hosts
+    rm ~/hosts
 
     cp /etc/hostname ~
     sed "s/raspberrypi/$new_host_name/g" ~/hostname > /etc/hostname
     setup_progress "Configured the hostname."
+    rm ~/hostname
   fi
 }