diff --git a/README.md b/README.md index ccc69cd..44f0620 100644 --- a/README.md +++ b/README.md @@ -32,13 +32,13 @@ Required: * A USB A Add-on Board if you want to plug your Pi into your Tesla like a USB drive instead of using a cable. [Amazon](https://www.amazon.com/gp/product/B07BK2BR6C/) Optional: -* A case. Don't want unprotected circuits hanging about! Official case at [Adafruit](https://www.adafruit.com/product/2885) or [Amazon](https://www.amazon.com/gp/product/B06Y593MHV). There are many others to choose from. Note that the official case won't work with the USB A Add on board. +* A case. The "Official" case: [Adafruit](https://www.adafruit.com/product/2885) or [Amazon](https://www.amazon.com/gp/product/B06Y593MHV). There are many others to choose from. Note that the official case won't work with the USB A Add on board. * USB Splitter if you don't want to lose a front USB port. [The Onvian Splitter](https://www.amazon.com/gp/product/B01KX4TKH6) has been reported working by multiple people on reddit. ### Software -Download [Raspbian Stretch Lite](https://www.raspberrypi.org/downloads/raspbian/) -Download and install: -* [Etcher](http://etcher.io) +Download: [Raspbian Stretch Lite](https://www.raspberrypi.org/downloads/raspbian/) + +Download and install: [Etcher](http://etcher.io) ## Set up the Raspberry Pi There are four phases to setting up the Pi: @@ -47,12 +47,20 @@ There are four phases to setting up the Pi: 1. Set up the archive for dashcam clips. 1. Set up the USB storage functionality. +There is a streamlined process for setting up the Pi which can currently be used if you plan to use Windows file shares, MacOS Sharing, or Samba on Linux for your video archive. [Instructions](doc/OneStepSetup.md). + +If you'd like to host the archive using another technology or would like to set the Pi up, yourself, continue these instructions. + ### Get the OS onto the MicroSD card [These instructions](https://www.raspberrypi.org/documentation/installation/installing-images/README.md) tell you how to get Raspbian onto your MicroSD card. Basically: 1. Connect your SD card to your computer. 2. Use Etcher to write the zip file you downloaded to the SD card. > Note: you don't need to uncompress the zip file you downloaded. +#### (Experimental) + +There is an experimental image that requires only a configuration file. Needs testing and issues filed _on this specific image/process_ at [this repo](https://github.com/rtgoodwin/teslausb/tree/headless-patch/headless-scripts). Issues about general functionality should still be filed on the main (cimryan) repo. + ### Get a shell on the Pi Follow the instructions corresponding to the OS you used to flash the OS onto the MicroSD card: * Windows: [Instructions](doc/GetShellWithoutMonitorOnWindows.md). diff --git a/doc/OneStepSetup.md b/doc/OneStepSetup.md new file mode 100644 index 0000000..38898ce --- /dev/null +++ b/doc/OneStepSetup.md @@ -0,0 +1,135 @@ +# One-step setup + +This is a streamlined process for setting up the Pi. You'll flash a preconfigured version of Raspbian Stretch Lite and then fill out a config file. + +## Notes + +* Assumes your Pi has access to Wifi, with internet access (during setup). (But all setup methods do currently.) USB networking is still enabled for troubleshooting or manual setup +* This image will work for either _headless_ (tested) or _manual_ (tested less) setup. +* Currently not tested with the RSYNC/SFTP method when using headless setup. + +## Configure the SD card before first boot of the Pi + +1. Flash the [latest image release](https://github.com/rtgoodwin/teslausb/releases) using Etcher or similar. + +### For headless (automatic) setup + +1. Mount the card again, and in the `boot` directory create a `teslausb_setup_variables.conf` file to export the same environment varibles normally needed for manual setup (including archive info, Wifi, and push notifications (if desired). +A sample conf file is located in the `boot` folder on the SD card. + + + The file should contain the entries below at a minimum, but **replace with your own values**: + ``` + export archiveserver=Nautilus + export sharename=SailfishCam + export shareuser=sailfish + export sharepassword=pa$$w0rd + export campercent=100 + export SSID=your_ssid + export WIFIPASS=your_wifi_password + export HEADLESS_SETUP=true + # export REPO=rtgoodwin + # export BRANCH=headless-patch + # By default will use the main repo, but if you've been asked to test the image, + # these variables should be uncommunted and updated to point to the right repo/branch + + # export pushover_enabled=false + # export pushover_user_key=user_key + # export pushover_app_key=app_key + ``` + +* Boot it in your Pi, give it a bit, watching for a series of flashes (2, 3, 4, 5) and then a reboot and/or the CAM/music drives to become available on your PC/Mac. The LED flash stages are: + +| Stage (number of flashes) | Activity | +|---|---| +| 2 | Verify setup variables | +| 3 | Grab scripts to start/continue setup | +| 4 | Create partition and files to store camera clips/music) | +| 5 | Setup completed; remounting filesystems as read-only and rebooting | + + + +* The Pi should be available for `ssh` at `pi@teslausb.local`, over Wifi (if automatic setup works) or USB networking (if it doesn't). It takes about 5 minutes, or more depending on network speed, etc. +* If plugged into just a power source, or your car, give it a few minutes until the LED starts pulsing steadily which means the archive loop is running and you're good to go. +* You should see in `/boot` the `TESLAUSB_SETUP_FINISHED` and `WIFI_ENABLED` files as markers of headless setup success as well. + +### For manual setup + +1. After flashing the image, boot it in your Pi and: + * connect via USB networking at `ssh pi@teslausb.local`. (The Pi must be connected to your PC and plugged into the port labeled USB on the Pi. Or... + * You can also just automate the Wifi portion of setup by creating the `boot/teslausb_setup_variables.conf` file and populating it with the `SSID` and `WIFIPASS` variables: + ``` + export SSID=your_ssid + export WIFIPASS=your_wifi_pass + ``` + +1. Once you have an `ssh` session, follow the steps starting at [Set up the USB storage functionality](https://github.com/cimryan/teslausb#set-up-the-usb-storage-functionality) in the main guide. + +### Troubleshooting + +#### Headless (full or Wifi) setup +* `ssh` to `pi@teslausb.local` (assuming Wifi came up, or your Pi is connected to your computer via USB) and look at the `/boot/teslausb-headless-setup.log`. +* Try `sudo -i` and then run `/etc/rc.local`. The scripts are fairly resilient to restarting and not re-running previous steps, and will tell you about progress/failure. +* If Wifi didn't come up: + * Double-check the SSID and WIFIPASS variables in `teslausb_setup_variables.conf`, and remove `/boot/WIFI_ENABLED`, then booting the SD in your Pi to retry automatic Wifi setup. + * If still no go, re-run `/etc/rc.local` + * If all else fails, copy `/boot/wpa_supplicant.conf.sample` to `/boot/wpa_supplicant.conf` and edit out the `TEMP` variables to your desired settings. +* (Note: if you get an error about `read-only filesystem`, you may have to `sudo -i` and run `/root/bin/remountfs_rw`. + + +# Background information +## What happens under the covers + +When the Pi boots the first time: +* A `/boot/teslausb-headless-setup.log` file will be created and stages logged. +* Marker files will be created in `boot` like `TESLA_USB_SETUP_STARTED` and `TESLA_USB_SETUP_FINISHED` to track progress. +* Wifi is detected by looking for `/boot/WIFI_ENABLED` and if not, creates the `wpa_supplicant.conf` file in place, using `SSID` and `WIFIPASS` from `teslausb_setup_varibles.conf` and reboots. +* The Pi LED will flash patterns (2, 3, 4, 5) as it gets to each stage (labeled in the setup-teslausb-headless script). + * ~~10 flashes means setup failed!~~ (not currently working) +* After the final stage and reboot the LED will go back to normal. Remember, the step to remount the filesystem takes a few minutes. + +At this point the next boot should start the Dashcam/music drives like normal. If you're watching the LED it will start flashing every 1 second, which is the archive loop running. + +> NOTE: Don't delete the `TESLAUSB_SETUP_FINISHED` or `WIFI_ENABLED` files. This is how the system knows setup is complete. + +### Image builder source and patches + +For now the image creation work is at: +* Modified pi-gen [rtgoodwin's fork of pi-gen](https://github.com/rtgoodwin/pi-gen) in (whatever current branch I'm working at the time). + +### Image refinement TODOs +1. ~~Patch the hostname to teslausb~~ +1. Make it so if someone deletes the `TESLAUSB_SETUP_FINISHED` file it's handled gracefully. (Right now it will try to re-run setup which should be fine.) +1. Cache the remount packages? Might mess with first boot like `rsyslog` +1. Aspirational TODO: Remove more packages and set services to stopped to make the boot process faster? +1. At this point, it's designed to pull the setup scripts _dynamically_, since development is still ongoing. If/when we reach a good frozen state, we can generate an image that is essentially ready to run. I think it'll also be pretty tricky to do some of the remounting and creating the backing files etc. on the image creation side. Open to suggestions/contributions there though! + + +#### Modifications to pi-gen builder from master + +The image is built on a Raspberry Pi 3B running Stretch, for maximum Pi-ception. + +This is the basic configuration, but it's helpful to just [look at the code itself](https://github.com/rtgoodwin/pi-gen/tree/teslausb-headless-image) and the Readme for Pi-gen which explains this all in much greater detail: + +1. Added SKIP and SKIP_IMAGES files to stage3, 4, and 5 (if present). We want to build the default image up to stage2, then add our own stage for tweaks we want. +1. Added a `stage6` (or 7, just something beyond stage5). (There are stages 0-4 in the main Raspbian pi-gen repo by default, but may be a stage5 in some cases. This will help keep a clean merge later.) +1. Copy the prerun.sh from `stage2`. Be SURE to `chmod +x` it. +1. Remove or rename the EXPORT_NOOBS files in all stages. We don't need a NOOBS image built. +1. In `stage6`, create a `00-tweaks` folder, with a `00-patches` folder and patches inside. Currently patched: + + | File | Change | + |---|---| + | `cmdline.txt`| Add the dwc2,g_ether modules | + | `config.txt`| Add the dwc2 module | + | `hosts` | Change hostname to `teslausb` | + | `hostname` | Change hostname to `teslausb` | + + * The build process uses `quilt` for patching + * The path for any patching you do at this stage is `stage6/rootfs/FILEPATH` where `rootfs` represents the Pi's `/`. So, `cmdline.txt` is `stage6/rootfs/boot/cmdline.txt`. + +1. Added a file called `series` in the patches directory with the name of each `.diff` file in the order you want them applied. +1. Added a `files` folder in stage6 with modified `rc.local`, and whatever else you want copied into the image. The modified `rc.local` will handle pulling down the `setup-teslausb-headless` file the first time and doing Wifi setup. +1. Added a script to flash the Pi LED +1. Files are moved into final locations using the `00-run.sh` script using the `install` command. See the script for details. I also `touch /boot/ssh` here so SSH is ready out of the box. +1. Run `sudo ./build.sh` from the `pi-gen` directory. +1. If you get a failure, it's almost certainly after stage2, so you can add SKIP files in all successful stages and rerun `sudo CLEAN=1 ./build.sh` \ No newline at end of file diff --git a/doc/teslausb_setup_variables.conf.sample b/doc/teslausb_setup_variables.conf.sample new file mode 100644 index 0000000..140e355 --- /dev/null +++ b/doc/teslausb_setup_variables.conf.sample @@ -0,0 +1,43 @@ +##################################################################### +# SAMPLE CONFIGURATION FILE FOR TESLAUSB Pi Setup +# +# Example config file for teslausb_setup. Lines with "#" are comments/ignored. +# Remove the "#" before "export" to activate a line. Be sure to rename this file +# to "teslausb_setup_variables.conf" and place it in the "boot" folder of your +# SD card. +# +###################################################################### + +# Default variables for CIFS (Windows/Mac file sharing) setup +export archiveserver=your_archive_name_or_ip +export sharename=your_archive_share_name +export shareuser=username +export sharepassword=password +export campercent=100 + +# Wifi setup information +export SSID=your_ssid +export WIFIPASS=your_pass + +# If doing a headless (i.e. automatic) setup +export HEADLESS_SETUP=true + +# Uncomment if setting up Pushover push notifications +# export pushover_enabled=false +# export pushover_user_key=user_key +# export pushover_app_key=app_key + +# Uncomment and change if you want setup scripts to be pulled +# from a different repo than the "main" github.com/cimryan/teslausb +# export REPO=cimryan + +# Uncomment and change if you want a different branch than master +# export BRANCH=master + +# Uncomment if you're using rsync/ssh for the archive. +# Note: RSYNC is not usable for headless setup since it currently requires a manual step. +# Note: RSYNC_ENABLE=true is going to disable the default archive server. +# export RSYNC_ENABLE=true +# export RSYNC_USER= +# export RSYNC_SERVER= +# export RSYNC_PATH= \ No newline at end of file diff --git a/run/cifs_archive/archive-clips.sh b/run/cifs_archive/archive-clips.sh index c7523a5..2816065 100644 --- a/run/cifs_archive/archive-clips.sh +++ b/run/cifs_archive/archive-clips.sh @@ -19,6 +19,9 @@ for file_name in "$CAM_MOUNT"/TeslaCam/saved*; do done log "Moved $NUM_FILES_MOVED file(s)." +if [ $NUM_FILES_MOVED -gt 0 ] +then /root/bin/send-pushover "$NUM_FILES_MOVED" +fi log "Finished moving clips to archive." diff --git a/setup/pi/setup-teslausb b/setup/pi/setup-teslausb index c2a0cdc..d92ad4b 100644 --- a/setup/pi/setup-teslausb +++ b/setup/pi/setup-teslausb @@ -1,22 +1,91 @@ #!/bin/bash -eu -USER_ENABLED_PUSHOVER=false - +USER_ENABLED_PUSHOVER=${USER_ENABLED_PUSHOVER:-false} +SETUP_LOGFILE=/boot/teslausb-headless-setup.log REPO=${REPO:-cimryan} - BRANCH=${BRANCH:-master} +HEADLESS_SETUP=${HEADLESS_SETUP:-false} +USE_LED_FOR_SETUP_PROGRESS=true + + if ! [ $(id -u) = 0 ] then - echo "STOP: Run sudo -i." + setup_progress "STOP: Run sudo -i." exit 1 fi +function setup_progress () { + if [ $HEADLESS_SETUP = "true" ] + then + echo "$( date ) : $1" >> "$SETUP_LOGFILE" + fi + echo $1 +} + +function headless_setup_populate_variables () { + # Pull in the conf file variables to make avail to this script and subscripts + if [ -e /boot/teslausb_setup_variables.conf ] && [ $HEADLESS_SETUP = "true" ] + then + source /boot/teslausb_setup_variables.conf + fi +} + + +function headless_setup_mark_setup_failed () { + if [ $HEADLESS_SETUP = "true" ] + then + setup_progress "ERROR: Setup Failed." + touch /boot/TESLAUSB_SETUP_FAILED + fi +} + +function headless_setup_mark_setup_success () { + if [ $HEADLESS_SETUP = "true" ] + then + + if [ -e /boot/TESLAUSB_SETUP_FAILED ] + then + rm /boot/TESLAUSB_SETUP_FAILED + fi + + rm /boot/TESLAUSB_SETUP_STARTED + touch /boot/TESLAUSB_SETUP_FINISHED + # This sed shouldn't be needed, but double checking just to be sure. + sed -i'.bak' -e "s/TEMPARCHIVESERVER/$archiveserver/g" /etc/rc.local + setup_progress "Main setup completed. Remounting file systems read only." + fi +} + +function headless_setup_progress_flash () { + if [ $USE_LED_FOR_SETUP_PROGRESS = "true" ] && [ $HEADLESS_SETUP = "true" ] + then + /etc/stage_flash $1 + fi +} + +function setup_led_off () { + + if [ $USE_LED_FOR_SETUP_PROGRESS = "true" ] && [ $HEADLESS_SETUP = "true" ] + then + echo "none" | sudo tee /sys/class/leds/led0/trigger > /dev/null + echo 1 | sudo tee /sys/class/leds/led0/brightness > /dev/null + fi +} + +function setup_led_on () { + + if [ $USE_LED_FOR_SETUP_PROGRESS = "true" ] && [ $HEADLESS_SETUP = "true" ] + then + echo 0 | sudo tee /sys/class/leds/led0/brightness > /dev/null + fi +} + function check_variable () { local var_name="$1" if [ -z "${!var_name+x}" ] then - echo "STOP: Define the variable $var_name like this: export $var_name=value" + setup_progress "STOP: Define the variable $var_name like this: export $var_name=value" exit 1 fi } @@ -26,14 +95,14 @@ function check_pushover_enabled () { then if [ ! -n "${pushover_user_key+x}" ] || [ ! -n "${pushover_app_key+x}" ] then - echo "STOP: You're trying to setup Pushover but didn't provide your User and/or App key." - echo "Define the variables like this:" - echo "export pushover_user_key=put_your_userkey_here" - echo "export pushover_app_key=put_your_appkey_here" + setup_progress "STOP: You're trying to setup Pushover but didn't provide your User and/or App key." + setup_progress "Define the variables like this:" + setup_progress "export pushover_user_key=put_your_userkey_here" + setup_progress "export pushover_app_key=put_your_appkey_here" exit 1 elif [ "${pushover_user_key}" = "put_your_userkey_here" ] || [ "${pushover_app_key}" = "put_your_appkey_here" ] then - echo "STOP: You're trying to setup Pushover, but didn't replace the default User and App key values." + setup_progress "STOP: You're trying to setup Pushover, but didn't replace the default User and App key values." exit 1 else USER_ENABLED_PUSHOVER=true @@ -45,17 +114,17 @@ function check_pushover_enabled () { } function check_available_space () { - echo "Verifying that there is sufficient space available on the MicroSD card..." + setup_progress "Verifying that there is sufficient space available on the MicroSD card..." local available_space="$( parted -m /dev/mmcblk0 u b print free | tail -1 | cut -d ":" -f 4 | sed 's/B//g' )" if [ "$available_space" -lt 4294967296 ] then - echo "STOP: The MicroSD card is too small." + setup_progress "STOP: The MicroSD card is too small." exit 1 fi - echo "There is sufficient space available." + setup_progress "There is sufficient space available." } function get_script () { @@ -63,8 +132,10 @@ function get_script () { local name="$2" local remote_path="${3:-}" - wget -O "$local_path/$name" https://raw.githubusercontent.com/"$REPO"/teslausb/"$BRANCH"/"$remote_path"/"$name" + curl -o "$local_path/$name" https://raw.githubusercontent.com/"$REPO"/teslausb/"$BRANCH"/"$remote_path"/"$name" + # wget -O "$local_path/$name" https://raw.githubusercontent.com/"$REPO"/teslausb/"$BRANCH"/"$remote_path"/"$name" chmod +x "$local_path/$name" + setup_progress "Downloaded $local_path/$name ..." } function get_ancillary_setup_scripts () { @@ -75,11 +146,11 @@ function get_ancillary_setup_scripts () { function fix_cmdline_txt_modules_load () { - echo "Fixing the modules-load parameter in /boot/cmdline.txt..." + setup_progress "Fixing the modules-load parameter in /boot/cmdline.txt..." cp /boot/cmdline.txt ~ cat ~/cmdline.txt | sed 's/ modules-load=dwc2,g_ether/ modules-load=dwc2/' > /boot/cmdline.txt rm ~/cmdline.txt - echo "Fixed cmdline.txt." + setup_progress "Fixed cmdline.txt." } BACKINGFILES_MOUNTPOINT=/backingfiles @@ -103,19 +174,20 @@ function create_usb_drive_backing_files () { if ! findmnt --mountpoint $BACKINGFILES_MOUNTPOINT then - echo "Mounting the partition for the backing files..." + setup_progress "Mounting the partition for the backing files..." mount $BACKINGFILES_MOUNTPOINT - echo "Mounted the partition for the backing files." + setup_progress "Mounted the partition for the backing files." fi if [ ! -e $BACKINGFILES_MOUNTPOINT/*.bin ] then + setup_progress "Creating backing disk files." /tmp/create-backingfiles.sh "$campercent" "$BACKINGFILES_MOUNTPOINT" fi } function configure_archive_scripts () { - echo "Configuring the archive scripts..." + setup_progress "Configuring the archive scripts..." get_script /root/bin archiveloop run @@ -137,7 +209,7 @@ function configure_archive_scripts () { get_script /root/bin remountfs_rw run - echo "Configured the archive scripts." + setup_progress "Configured the archive scripts." } function configure_pushover_scripts() { @@ -150,7 +222,7 @@ function configure_rc_local () { return fi - echo "Configuring /etc/rc.local to run the archive scripts at startup..." + setup_progress "Configuring /etc/rc.local to run the archive scripts at startup..." echo "#!/bin/bash -eu" > ~/rc.local echo "archiveserver=\"${archiveserver}\"" >> ~/rc.local cat << 'EOF' >> ~/rc.local @@ -169,26 +241,38 @@ EOF cat ~/rc.local > /etc/rc.local rm ~/rc.local - echo "Configured rc.local." + setup_progress "Configured rc.local." } function configure_hostname () { - echo "Configuring the hostname..." + # Headless image already has hostname set + if [ ! $HEADLESS_SETUP = "true" ] + then + setup_progress "Configuring the hostname..." - local new_host_name="teslausb" - cp /etc/hosts ~ - sed "s/raspberrypi/$new_host_name/g" ~/hosts > /etc/hosts + local new_host_name="teslausb" + cp /etc/hosts ~ + sed "s/raspberrypi/$new_host_name/g" ~/hosts > /etc/hosts - cp /etc/hostname ~ - sed "s/raspberrypi/$new_host_name/g" ~/hostname > /etc/hostname - echo "Configured the hostname." + cp /etc/hostname ~ + sed "s/raspberrypi/$new_host_name/g" ~/hostname > /etc/hostname + setup_progress "Configured the hostname." + fi } function make_root_fs_readonly () { /tmp/make-root-fs-readonly.sh } -echo "Verifying environment variables..." +headless_setup_populate_variables + +# If USE_LED_FOR_SETUP_PROGRESS = true. +setup_led_off + +# Flash for stage 2 headless (verify variables) +headless_setup_progress_flash 1 + +setup_progress "Verifying environment variables..." RSYNC_ENABLE="${RSYNC_ENABLE:-false}" RCLONE_ENABLE="${RCLONE_ENABLE:-false}" @@ -218,6 +302,11 @@ check_variable "campercent" check_pushover_enabled +# Flash for Stage 3 headless (grab scripts) +headless_setup_progress_flash 2 + +setup_progress "Downloading additional setup scripts." + if [ ! -e /root/bin ] then mkdir /root/bin @@ -247,6 +336,7 @@ get_ancillary_setup_scripts pushd ~ + configure_archive_scripts configure_pushover_scripts @@ -255,6 +345,9 @@ fix_cmdline_txt_modules_load echo "" >> /etc/fstab +# Flash for stage 4 headless (Create backing files) +headless_setup_progress_flash 3 + create_usb_drive_backing_files /root/bin/configure-archive.sh @@ -263,6 +356,14 @@ configure_rc_local configure_hostname +# Flash for stage 5 headless (Mark success, FS readonly) +headless_setup_progress_flash 4 + +headless_setup_mark_setup_success + make_root_fs_readonly -echo "All done." +# If USE_LED_FOR_SETUP_PROGRESS = true. +setup_led_on + +setup_progress "All done."