Search and Rescue operations require professionals and volunteers to work in concert in order to find the missing person as quickly as possible. Software packages such as Loc8 help facilitate this by allowing professionals to quickly search large image datasets for anything matching the clothing of the missing person. These types of software have incredible potential but, like any other complex skill, require training and experience to be used to their full potential. Below are two examples from a past blog post where I used Loc8 to discover cattle in a field. The first image was from my first attempt at finding brown cattle in areas containing dirt, as can be seen the image is full of false positives and of marginal use. The second image was created after additional training and experience was gained with the program, and false positives are minimized making the image more useful.
Figure 1: Initial cattle search
Figure 2: Follow up cattle search
While using this software in groups of five to twenty I have noticed that the majority of the image assessments resemble the first example more than the second. When time is of the essence it is impractical to provide a robust training course on the software, and allow the volunteers time to explore and gain experience. The most difficult challenge for new users to overcome seems to be determining the correct spectral ranges and software settings to use. If the search signature is incorrect, or if the settings are off, the software will be unable to find the target, even if it is present in the imagery. For this reason I assign volunteers a well trained and experienced advisor for the search.
Organization
When conducting operations I follow the Incident Command System structure defined in the National Incident Management System.
Figure 3: Intelligence branch organization
The Intelligence Lead defines the strategy, discussed in depth later, to be used and outlines the areas that the Group Leaders will function inside. The Group Leaders are then responsible for dividing up that operating area and clearly delegating portions of it to volunteers. This delegation should be clear and require no interpretation. The Intelligence Leader must be attentive to feedback provided by group leaders and adjust strategies based upon the results of the searches being conducted. When the volunteers discover potential missing persons their group leader will assess the image. If the group leader things it might be a possible hit on the missing person they will provide the image and the GPS coordinates to the intelligence leader. If the intelligence leader is provided with an image that they think is a possible they will provide the Incident Commander with the image and GPS location. By requiring the group leader to provide a second opinion, and the intelligence lead provide a third opinion on the possible hit false positives should be reduced. False negatives are a risk with increased oversight so this may need to be adapted based upon the experience of the group leaders and intelligence lead Below is an example based off of an operation that I conducted in November of 2019. In this case there were five volunteers, meaning there was only one group and that intelligence lead was also group 1 lead.
Example Operation
I was provided with the dataset created by field teams and the information that the missing person was last seen wearing red. I split the entirety of the red spectrum into segments and assigned volunteers to each segment. Initially I was also going to participate in the search but due to a software error I was unable to do so. This proved lucky as managing five volunteers and the data they generate took all of my time. During the search the volunteers would assess any images that provided alerts, and any that they thought needed a second pair of eyes were provided to the group leader. Figure 4 shows that the range of 16,0,0-56,20,20 returned the most results. This range contained many false positives, many of which were shadows found by the inclusion of +20 in green and blue.
Figure 4: First search, First Spectral Ranges
All segments of the first run were completed in around 30 minutes. During this search a potential positive was discovered and sent to the ground teams along with the GPS coordinates of the image. After the image was sent a second run was started. This run focused on splitting up the 16,0,0-56,20,20 range and reducing false positives introduced by shadows. Splitting this range into smaller segments was intended to reduce the number of images that each volunteer was required to review, and in this case was successful.
Figure 5: First Search, Second Spectral Ranges
Removing the green and blue portions of the spectrum eliminated the effect of shadows, and reduced the number of images alerted in general. The image of note that was found in the first run was again found during the second run. Ground teams were sent to investigate the object discovered
After this search was completed the number of images in the dataset was increased by field teams, and we were provided with an image of the missing person in their last known clothing. A spectral signature of the missing person's clothing was created from this image. This signature was then made the mid-point of a range, the range was then split into four segments, four volunteers were provided a segment, and one volunteer was provided the exact signature for their search.
Figure 6: Second Search, First Spectral Ranges
When reviewing figure 6 it is easy to assume that this method was ineffective since only one object was discovered. As much as I would like to say this is because that one object is the missing person, it unfortunately was not, but instead a member of the search party wearing a similar color. The next day we discovered that the missing person was not present in the dataset that we were provided, and the search was called off. While this operation was ultimately unsuccessful it did get me thinking about the most effective way to use this kind of software with volunteer groups.
Search Strategies
I have discovered 3 strategies for most effectively using this software in a group environment, and have begun using them in practice SAR operations. During manual image assessment there are two main variables, how many images are there, and how quickly can each image be accurately assessed. When using Loc8 a third variable is added, what spectral ranges were searched within each image. The strategies below are the ways I have discovered to search a large number of images
Wide Spectral Net
This strategy is the one used in the example above. The Intel leader provides the group leaders the settings to use, and a spectral range to search. The spectral range must be based off of a sample image to ensure that the range is correctly tailored to the search. The group leaders will then divide up the area provided to them and assign volunteers to each subsection. Each subsection should be tailored to the number of volunteers available, and overlap with the segments to either side. For example 18,18,18-40,40,40 should be preceded by 0,0,0-20,20,20 and followed by 38,38,38-60,60,60. The amount of overlap used will be different for each operation and depend on the color being searched for. This allows a wide range of the light spectrum to be searched simultaneously, but is more prone to false positives in certain areas. This method will require multiple runs with shifting ranges to minimize false negatives. For example an object could be 20,60,40 and would not be found by the ranges listed above. This possibility will be minimized by ensuring the spectral range is correctly constructed, as stated above
Image Subsets
One of the negatives of searching large image data-sets is that the larger the data-set the longer it will take to assess. This means that if a missing person is present in the final image of a data-set it will not be found until the final image is reached. Larger data-sets also require more time to be spent before a new spectral range can be searched. This strategy is intended to increase the assessment speed of large data-sets by reviewing them in parallel. After data is received it will be split into a number of image subsets that correlates to the number of people searching. If there are four volunteers then four or eight subsets should be made. If there are twenty people searching then ten subsets would be better. Once image subsets have been created group leaders will be assigned a spectral range and assign their volunteers an image subset to search. In this way all volunteers will be using the same spectral signature, but will be assessing the images much more rapidly. If large quantities of volunteers are available this strategy can be easily combined with either of the other strategies. This would involve the intelligence lead assigning each group leader different signatures. If twenty volunteers are available and ten image subsets are available then ten volunteers should search one range while the other ten search another. This way the number of images and signatures searched can increase rapidly.
Setting adjustment
Loc8 software allows users to adjust a number of settings for the search. The most important of these being the number of pixels required for an alert, both minimum and maximum number of pixels. The idea for this is that a missing person will not be the size of a car, maximum pixels, or smaller than a soccer ball when in the open, minimum pixels. For this reason operators will likely define their own "default settings" through experience likely depending on their view of the trade off between false positives/negatives. Certain situations can arise that will require these settings to be changed as it is uncertain how much of the missing person will be visible. In a densely forested environment it is possible that only a single pixel will be visible through the canopy, or alternatively the person could have left a trail of objects larger than a person (such as a tent or a tarp). In these situations adjusting the settings used may be the best way to assign volunteers. If intelligence lead decides to use this strategy each group will be assigned the same images and signature but different settings. In this way the strategy mirrors the "wide-net" approach with the difference that each volunteer will be searching the same signature but looking for differing numbers of pixels. The volunteer given an alert at 1 pixel (if this is used) will likely have the most alerts, and may become overwhelmed with false positives. If multiple groups are available this strategy is easily combined with image subsets. If combined with image subset then each group leader will be provided the exact settings for their group to use. Group 1 would search 1 pixel, Group 2 would search 2 pixels, Group 3 would search 3 pixels and so on. Using 1, 2, and 3 pixels is likely not a good idea as this will likely saturate the effort in false positives. Determining the maximum size for a missing person should be determined and the search should focus around that size. It must also be kept in mind that the number of pixels that a person takes up will change with altitude and sensor used. If an object is the target of interest then the maximum size of that object should be used instead.
Conclusion
Spectral Signature analysis software, such as Loc8, allows operators to rapidly assess large data-sets. As UAS increase the size of these datasets their use will be increasingly important. These software packages can be arcane to the layman and require experience to account for the large number of factors that affect image quality. By using well trained and experienced personnel to lead untrained volunteers allows them to have a much larger effect on the operation than they could have operating on their own. Even with training, experience, and manpower a methodical approach is needed to get the largest benefit from spectral assessment. Three dimensions must be searched in each dataset, the size of each image, the number of images, and the color spectrum searched. The strategies defined above are my initial assessments on how to best compromise between these three in a limited group setting. As time goes on and these strategies are employed they will be adjusted and honed to ensure that software packages like Loc8, are able to save as many lives as possible.
Comments