Chapter 5 Introduction to scripting

5.1 Introduction

Now that you understand how we organize our files, it is time to learn how to use our python scripts to automate the image processing steps. For many people, this will be the most confusing part of the process, so please read the instructions carefully.

5.2 Python Scripts

Notice the expanded folder called North-Country-Wild in figure 5.1. It contains the PythonScripts folder that you will need to access to run our scripts.

Brett has created a python script called upload_and_process_images.py located in the folder called PythonScripts inside of th North-Country-Wild folder in the root directory of the NoCoWild hard drive (figure 5.1). The other two scripts in the folder are used for pushing files to the zooniverse once they are processed.

Figure 5.1: Python Script files inside of the PythonScripts folder

To use the script you will need to:{#use-scripts}

Open a terminal window (See “Accessing the terminal” in Chapter 3 for a refresher).
From the terminal, cd into the PythonScripts folder by typing cd on the command line and then dragging the folder icon onto the terminal. Hit return when finished.
To access the script and learn about its subcommands, type python3 upload_and_process_images.py -h. Text with information about the script subcommands will display in the terminal window because you added -h after the name of the script (figure 5.2).

Figure 5.2: Terminal window showing location in PythonScripts folder and subcommands for the upload_and_process_images.py script

5.3 Script subcommands

You can see that there are three possible subcommands, each of which take several different arguments (Table 5.1).

Table 5.1: List of python script subcommands and their arguments
Subcommand	Arguments
change_file_size_and_copyright	This subcommand takes as arguments the path to the processed images which have already been renamed and resizes them and adds copyright information
copy_raw_images_change_file_size_and_copyright	This subcommand takes as arguments the camera number and sd card number, each as 3-digit integers, and copies, resizes, and adds copyright information to the images. If a 3-digit integer is not provided, the script will automatically add leading zeros.
completely_process_images_from_sd_card	This subcommand takes a number of different arguments (use -h after the subcommand to see them) and completely processes images from the sd card to separate directories (aka “folders”) for the raw and processed images.

Further, most of these subcommands take arguments, which are typed into the terminal as --argument-name followed the required information. For example, Table 5.2 shows the arguments that are required by the completely_process_images_from_sd_card subcommand.

Table 5.2: List of arguments required by the completely_process_images_from_sd_card subcommand
Argument.name	Description
h or help	shows the help menu
memory-card-path	indicate the full path to the SD card on the computer; you can drag the folder onto the terminal
path-to-raw-images	indicate the full path to the Raw image folder where the Raw images should be deposited; you can drag the folder onto the terminal
path-to-processed-images	indicate the full path to the Processed image folder where the processed images will be placed; you can drag the folder onto the terminal
camera-number	indicate the camera number that the memory card is associated with for this deployment as an integer not exceeding 3 digits; if you forget to include 3 digits the script will take care of it
sd-card-number	indicate the sd card number that the images come from as an integer not exceeding 3 digits; if you forget to include 3 digits the script will take care of it

5.4 Running the Scripts

Before you can begin using the scripts to process image files, you need to create empty folders for the camera-sd card combination in the Raw and Processed folders for the given year as shown in Chapter 4 Section 4.2 Hard Drive Organization. Please be careful in naming these folders so that the folder names are of the form C***_SD*** where *** represents the 3-digit camera number (including a leading zero) and the 3-digit SD card number (including a leading zero), respectively.

5.4.1 Completely process images from sd card

Most often, you will want to completely process all of the images on an SD card. As a reminder, in this process, running the python script correctly will:

Copy all of the images from the SD card to the correctly labeled folder in the Raw image directory
Copy all of the images from the Raw folder to the correct directory in the Processed folder
Rename all of the images in the correct Processed folder so that the image name includes the camera-sd card as well as the time stamp the image was made, in addition to the original image name
Resize the all of the images to just under 1MB in size for later upload to the zooniverse
Add a copyright field with the words “Barthelmess Lab and Nature Up North” to all of the image exif data.

Let’s assume for demonstration purposes that I have a number of images on an SD card in a folder called Camera and let’s further assume that the SD card was number 001 and the camera was number 001. I therefore have created a folder called C001_SD001 in both the Raw and Processed folders on the hard drive. Notice what they look like prior to processing the scripts in Figure 5.3.

Figure 5.3: Raw and Processed folders for a demonstration set of images prior to image processing

Specific instructions:

Obtain a paper copy of the data sheet associated with the specific deployment you intend to process and note the Camera number and SD card number.
Open the correct year’s folder on the NoCoWild hard drive. Create a new, empty folder named as indicated above in the Raw folder and in the Processed folder.
Place the SD card into the card reader attached to the imac.
Open a terminal session and cd to the PythonScripts folder (inside of the North-Country-Wild folder on theNoCoWild` hard drive). Refresh your memory about how here
Paste the following line of code at the cursor python3 upload_and_process_images.py completely_process_images_from_sd_card Do not press return after pasting
Paste the following line of code at the cursor --memory-card-path followed by a space and then drag the folder of images on the SD card onto the terminal Do not press return after pasting
Paste the following line of code at the cursor --path-to-raw-images followed by a space and then drag the appropriate Raw image folder onto the terminal. Do not press return after pasting
Paste the following line of code at the cursor --path-to-processed-images followed by a space and then drag the appropriate Processed image folder onto the terminal. Do not press return after pasting
Paste the following line of code at the cursor --camera-number followed by a space and then type the 3-digit camera number including a leading zero Do not press return after pasting
Paste the following line of code at the cursor --sd-card-number followed by a space and then type the 3-digit number for the SD card including a leading zero
Hit Return. If you have no typos or other errors, the script will immediately begin processing the images.

Figure 5.4 shows the terminal window for steps 5-11 above. The terminal commands are highlighted in yellow.

Terminal window showing a complete script for processing images from a SD card. The terminal commands are highlighted in yellow and the output, showing that the script ran, follows below the terminal commands.

Figure 5.4: Terminal window showing a complete script for processing images from a SD card. The terminal commands are highlighted in yellow and the output, showing that the script ran, follows below the terminal commands.

After the script ran, the resulting contents of the Raw and Processed image folders for this demonstration is shown in Figure 5.5.

Figure 5.5: The same Raw and Processed folders for a demonstration set of images after image processing

5.4.2 Running other subcommands

The process for running the other two subcommands, copy_raw_images_change_file_size_and_copyright and change_file_size_and_copyright is very similar. You will need to specify file paths and other arguments. But the overall process of running the scripts is the same:

python3 upload_and_process_images.py followed by
the appropriate subcommand, followed by
each argument for the subcommand, with “answers” (e.g. file path, camera or sd card number) specified

Remember, you can use the -h argument after a subcommand to open the help information in your terminal window to learn how to specify the arguments that go with a particular subcommand.

5.5 Summary

Managing game camera images is a complex process. Images have to be associated with specific cameras and SD cards, have to be transferred to new folders, and have to be resized and renamed. Though the steps in this chapter may seem complex, imagine if you had to go through each of these steps for every individual image! There would be so many opportunities to introduce error. Instead, we are using a series of python scripts to streamline the process and reduce the propegation of errors in the data set.