Image Analysis

A Picture is Worth a Thousand Words,
But What Do Those Words Mean?

by Adam Baddeley, SOTECH Correspondent

COMPLEXITY LEVELS

Change detection has four levels of complexity. The first is manual image comparison, discerning the differences that occur in a specific area of interest. The second level of complexity is automated change detection, but the challenges don’t stop here.

The next level of complexity is feature extraction. It’s not enough for analysts to see the differences between two images, they also want to know at a glance how many buildings have been destroyed by a hurricane or how many square miles are now flooded, and to do so the software has to be able to calculate that. The next level in the change detection challenge is automated object recognition, specifically looking for known types and classification of objects. Not only do users want to see that there is a building, but they want the systems to tell them whether it is a pharmaceutical production facility, weapons factory or a nuclear power plant and make and model of tank next to it; this is still some way off.

Irrespective of software algorithms used to compare images, detailed analysis is ultimately dependent on obtaining very high-resolution imagery to support that. The largest failure rates occur because of cloud cover or with dramatic look angles, according to Rob Mott, executive director for defense and intelligence solutions at Intergraph., “If you are looking directly down on rooftops, you will get very good results,” Mott said. “If your look angle is greatly skewed and you are contending with cloud cover or lower resolution imagery; then you will naturally get less accurate results. There isn’t an empirical level of failure in change detection.”

INTEGRATION PERSPECTIVE

Raytheon is currently undertaking a number of projects in support of U.S. and coalition ISR efforts in the global war on terror, integrating a range of technologies including changedetection capabilities.

“Within imagery analysis there is a metamorphosis occurring that is moving us from observing with the human eye to interpretation by computer. We are on the dawning of a different science, what I’ll call ‘low altitude forensics,’” said Frank Prautzsch, a director in the company’s Rapid Initiatives Group. “The age of high speed computing is now allowing us doing to a few different things. First, is the storing of enormous amounts of information and then pulling that information up to look for trends. Changes in vegetation, topography, or the absence or presence of man or beast are now discernable by onboard and secondary processing computer systems that can easily spot changes at the pixel level. That generates an entirely different piece of knowledge that you would not otherwise have.”

Prautzsch sees this technology as another decision tool, with trust in its effectiveness being built up by giving users information that that they would not otherwise obtain through standard methods.

“The true Rosetta Stone for this kind of program is to be able to tie it in with other sensor systems and the correlate events to give you a much clearer picture of what is going on. If you take imagery analysis and combine it with distributed sensors it will start giving you an additional piece to the intelligence mosaic and that will sometimes give you the tipping point to a decision.”

INTERGRAPH

In Intergraph’s geospatial intelligence exploitation solution, the Image Scout product fuses geospatial data for image exploitation. This technology supports the soft copy search requirement within NGA, whose goal is to move to a completely digital environment for the analysis of imagery. Intergraph also brings that solution to other elements of DoD such as the Defense Intelligence Agency’s Missile and Space Intelligence Center.

As part of an image management and exploitation continuum, Image Scout is integrated with the company’s TerraShare image management solution and an automated imagery ingest product; Auto Terra Ingest.

Mott explained. “TerraShare acts as a switchboard that sits within the enterprise architecture and the analysts representing the callers. They make requests for imagery, and TerraShare knows where the right image sets exist and sends links to that analyst to the proper image.” In addition to still imagery, Mott added that further media such as LIDAR, terrain data or even video clips are increasingly being sought.

Mott describes the relationship between the Intergraph products by using a comparison to consumer electronics. “We sometimes make a comparison to Apple and what they have done with the iPod for music and video. iTunes is what Apple uses as that main switchboard that allows users to discover, connect to, and receive digital files down to their user experience, which in that case is the iPod. TerraShare works in the same way as iTunes, except that in the case of geospatial intelligence, Image Scout represents the user experience.”

Each image is described using electronic tags containing meta-data contained in the image format’s header. Additional, customizable fields, such as the degree of cloud cover can be added to the TerraShare database, based on information inputted for example by analysts’ first pass check on the image. Auto Terra Ingest streamlines the insertion of those entries in that catalog; simplifying the systems administrator’s tasks by automatically filling the catalog entry with that information as it processes the imagery immediately upon its arrival, working around the clock without any administrator intervention unless it finds corruption in a file or an error.

Mott said, “The feedback we received from analysts at the Missile and Space Intelligence Center is that the introduction of TerraShare and Auto Terra Ingest into their existing architecture and their existing analysis applications are saving each analyst in the order of 2 to 3 hours of manual search time, each day.”

The latest version of Auto Terra Ingest supports custom upstream processing that can automate image processing that could run automated image change detections by comparing new images to a baseline image. Image Scout provides a “flip book” style of image comparison tools that allows analysts to move back through time using slider control and see the changes over time that occur in a specific area of interest. For automatic recognition, Intergraph uses the Feature Analyst for GeoMedia product sourced from Textron’s Overwatch subsidiary, which highlights new items in blue while features that are no longer there take on a red hue. That same technology also lets and operators define recipes that identify what should be interpreted as a road or a building or coastline, for example. It would then process the images and extract the corresponding information as vector features, which represents quite a time savings for the analyst who would otherwise manually count buildings or calculate square miles of flooded terrain.

In terms of automated object recognition Mott explained, “The Feature Analyst product can be used for that process as well and we have seen that the best results are produced when you have consistent high-resolution imagery. There is a lot of work ongoing with respect to improving the results under a variety of conditions but the process will not be totally automated in the near term.”

Inevitably, there is increased interest in applying the same techniques to UAV video imagery. Intergraph’s unique approach to solving this problem is to generate a georeferenced image from the UAV video in order to develop a consistent frame of reference from which comparisons can be made. Mott said, “If I have a UAV flying down a possible convoy route and I fly in the morning and then fly again in the afternoon, I then would like to have the ability to compare those two video clips. Our approach is to generate an accurately geo-referenced image by stitching together and processing and enhancing individual frames. Next, we can apply Feature Analyst for Geomedia to do this change detection across those two resulting geo-referenced images. As the tradecraft of change detection becomes more robust and reliable, we can apply that to new systems and collections and generate more definitive output.” Intergraph is currently developing a motion imagery exploitation solution to do this. Currently in prototype form, an initial releasable complete motion imagery exploitation suite is scheduled for 2009, although some of its individual components could be available sooner.

SOCET AND SEE

“Our SOCET GXP software is used just about anywhere you can open up a laptop. We have it in CONUS with many entities in the U.S. government, including all of the armed forces and specialized government agencies. It’s also forward deployed in Afghanistan and Iraq to prosecute the war on terror,” explained Rob Stout, SOCET GXP product manager for BAE Systems Geospatial eXploitation Products (GXP).

GXP has developed two products; SOCET SET is used to create digital maps, charts and terrain models, while SOCET GXP enables image and geospatial analysis. An iterative solution, SOCET GXP v2.3.1 will be updated later this year to SOCET v3.0, considered by Stout to be the point in which image analysis, photogrammetry, and geospatial production become fully combined.

There are two ways for SOCET SET and SOCET GXP to undertake change detection. For example, the Normalized Difference Vegetation Index and Tasseled Cap algorithms were developed originally for change detection in vegetation and coastlines.

Stout explained the process, “We open two images for comparison into a Multiport—where we do all of our work— and overlay the images directly on top of each other. Then the algorithms are run to establish the difference in the images. You push one button and it generates the analysis for you. It’s instantaneous.”

It looks at the pixel around the area and compares it with the other image to see if the pixels are close to the same, mostly dealing with color, saturation, moisture and area.

BAE Systems’ Visual Coverage Tool, used across DoD and government agencies, locates a designated geographic area of interest and searches for corresponding data to load for analysis. Stout said, “In this product I can type in a set of coordinates, and it will take me to that area and show me what I have in my library. I then open it up into SOCET GXP to complete numerous analysis tasks. When new information is added to the system, it will show that something is new.”

A new SOCET GXP module; Spatially Enabled Exploitation (SEE), allows users to identify an object, define, annotate and classify that object, and then store a record of its pixel composition directly in a database for future analysis. Features can be compared across multiple images quickly for accurate and efficient change detection.

“Analysts don’t have to spend time looking for things that have already been cataloged because they can update their database using information that other analysts have already processed. For example, if the object moves around, you can perform trend analysis over a period of time. Anybody connecting the same database can pull that information for use. That is helpful in a deployed situation where analysts rotate in and out frequently and different people are doing the work from one week to the next. Analysts can use someone else’s work and continue on without having to start from scratch.”

Moreover, SOCET GXP provides synchronized, 3-D viewing with Google Earth, giving analysts and decision-makers access to critical information. You can locate and preview images geographically using SOCET GXP and Google Earth as quick visual reference and discovery tools for immediate geospatial context.

ENVI

ITT Visual Information Solution’s change detection capabilities are contained within its ENVI software, current at version 4.4. ENVI is in service with a number of DoD commands and number of U.S. agencies including the DIA and NGA. Overseas it has been adopted by a number of allies including Australia, France, Germany and the U.K.

“The first thing ENVI does is register images very accurately,” explained Jim Kelley, vice president of strategic programs. “When you compare multiple images of the same location, you must ensure you are only detecting real change in the scenes, not change caused by miss-registration because an image was taken at a slightly different altitude or angle. If it is off by only 50 cm it will look like everything has changed. We have tools that do this registration in an automated fashion. The only thing ENVI shows is true change.”

ENVI also embeds complex algorithms for multi-spectral analysis to identify changes that would otherwise go undetected by an analyst’s naked eye. “For example, in Iraq if someone is digging by the side of the road, and they cover up their digging very well, it may be difficult to determine with the human eye that something has occurred. However, because of the clays and the soil, ENVI’s image processing techniques including principle component analysis, minimum noise friction or even RX anomaly detection will allow someone to detect the change that is not apparent to the human eye, but is still spectrally apparent.”

Easing pressure on a finite resource, namely analysts themselves, has also been pursued with ENVI.

Kelley said, “With new workflow-based tools to create these change detection products rapidly, we have tried to move processing capabilities out to a wider class of users. ENVI enables analysis to be rapidly produced by non-experts; users don’t need to be two-year veterans to use ENVI this effectively.”

ENVI also has the ability to create automated workflow batches to undertake change detection very rapidly according to Kelley. “If we know we are looking for certain types of targets, and we know the type of sensor we are using, we can build automated tools so that the change detection process can be done entirely in the background, rather than having to have a man in the loop for all of the processing.”

ENVI differs only very slightly between implementations from the commercial and research communities and military and national security users. In the case of the latter, two additional modules are used to support National Imagery Transmission Format data—a format used widely by U.S. government and NATO to disseminate imagery and TFRD, a classified data format typically used by U.S. government and close allies. Kelley added, “Most of ENVI functionality is included in the primary product, but one of its greatest advantages is that its programming level can easily be accessed to create customized applications. This has allowed many organizations to write their own specific add-ons, and this has produced a wide variety of additional role-specific tools.

CONCLUSION

There is a good deal of confidence in environments where the data that is coming in over time, as images coming in, that there is some level of conformity or consistency with the resolution of the data. It is a very interactive process in that initial implementations may show a level of satisfactory results, but over time the operators develop an understanding of which situations are best suited to apply this technology. There is going to be total reliance on an automated capability, but it is definitely streamlining the process. ♦