On 28th February 2022 imaging experts from academia and industry  met to discuss the topic of high resolution imaging at a range of 10KM. The morning was dedicated to identifying the problems, whilst the afternoon focused on potential solutions. Lead organiser of the workshop, Chris Greenway, MBDA, has written up the following notes taken during the day,

Imaging performance is generally considered scalable with size. To see more detail, we ideally want a bigger camera, looking at larger object features, over a shorter distance. Unfortunately for the imaging community, this is rarely the situation on offer. To address this challenge, the IET’s Vision & Imaging Technical Network held a one-day workshop to discuss the challenges associated with High Resolution Imaging, at Range. In particular, the workshop focussed the discussion on the question “is it possible to identify a person at a range of 10km … without having to carry around a camera the size of the Webb Telescope!?”

Recognising a person – or any object – is broadly defined as being able to distinguish that object from other objects, i.e. it’s a person, not a tree. Whereas identification is defined as being able to distinguish enough features that a particular person or object can be recognised.

The workshop addressed the challenges and potential solutions associated with the key aspects of the question: the imager (both hardware and processing), the object being imaged, and the environment between them.

It was noted from the start that a significant factor in identifying an object (our person) will be on how cooperative they are being. Are they lost on the savannah and looking for help, or concealing themselves in a busy venue, possibly wearing a disguise…? And it is not just the person that could vary its appearance. Scene conditions can change – such as lighting levels – and also the viewing angle; a ground level view will be different to one from above, for example.

We agreed that the principal assumption would be that our person is being cooperative, but that consideration on how to allow for a more difficult imaging task would be addressed as part of the image processing challenges (below).

For the environment, and to image a person at 10km. That’s a lot of atmosphere! A long list of challenges emerged from atmospheric absorption, and distortion, light levels and even the impact of the Earth’s curvature. Some of these factors can be mitigated based on location choice. Viewing at higher altitudes, and or in less humid, cloud-free conditions will improve imaging conisations. Also in discussion was how the imager waveband choice could influence imaging performance? Could using infrared offer improved imaging versus visible. Infrared allows the thermal signature of the person to be detected, and so is less dependent on the scene illumination; could that be a better choice for the task? And what of other wavebands or ‘signatures’ too. Could the chemical signature be utilised, or an alternative specific narrow-band that could be tuned to the features of the person, and in doing so optimise the signal detected.

The main body of discussion was given to the camera itself. Camera performance can be broadly addressed in two parts: spatial resolution, and dynamic range. And, as stated at the start, a smaller camera will generally have lower spatial resolution and (assuming a digital sensor) less dynamic range.

Spatial resolution is limited (in principle) to the geometry of the camera design, and is governed by the diffraction limit. This means that when light enters the camera, its energy spreads. Larger wavelengths passing through smaller apertures will experience most diffraction. This causes the energy to spread over more pixels and, in doing so, limits the size of feature that can be resolved. This would imply that infrared, with a wavelength approximately 10 times that of visible would diffract more, and offer mwore resolution performance for the same camera size. This prompted a discussion on whether or not high resolution was needed and whether the task could be achieved just as well with lower native spatial resolution (bigger camera pixels for example) that would be less sensitive to the ‘dynamic’ aberrations of the image as it passed through the atmosphere. Super resolution was also suggested as a technique to compensate the resolution, either with controlled sensor translations, or using the random atmospheric distortions to generate a more detailed image, and also temporal imaging was raised as having potential for application here? The conversation thread also covered the use of illumination techniques such as lidar that may provide ‘boosted’ signal and importantly range information, from which to make the identification.

Dynamic range is the measure of how sensitive a camera is to light level change. At long ranges, with much of the image energy lost to the atmosphere, then increasing the camera sensitivity will be a key factor. Advances in detector performance was discussed. In the visible wavebands, CMOS architectures (Scientific-CMOS, for example) offers the potential for very low-light level sensitivity, and coupled with novel semi-conductor materials such as ‘black silicon’, then the ability to detect over longer ranges could be increased. Infrared sensitivity is increased when cooling mechanisms are applied, typically adding cooling engines to set the detector to cryogenic temperatures. This however can increase size, weight and power (SWaP) constraints which are not desirable, particularly for a portable device. Recent developments in uncooled and ‘hot’ infrared detectors look to offer similar performance points at much reduced SWaP. In this thread, graphene detectors, single-photon avalanche diode (SPAD) arrays were considered as potential alternatives, and also quantum dot technology, where the potential to tune dots to specific wavelengths could offer improved resolution, and enable adaptive pixel binning.

Like most things in life, in imaging it is not just what you have, but how you use it that matters; and there was good discussion on how a number of these approaches and techniques could be used together. For example, could the use of multiple wavebands be combined or fused to improve image resolution? Quantum dots have already been suggested as one way to do this, but other approaches are also now emerging that could enable different camera pixels to be sensitive to different aspects of the image. Not strictly in scope, but multiple cameras were also suggested as a solution. Cameras could be used to build up a stereo image of person, to better extract information, or in a tiered approach where one camera was optimised to assess the conditions of the scene and the other then tuned to extract the maximum image detail. Being able to tune or adapt the camera was considered an important benefit, particularly over long ranges.

Conventionally, camera stabilisation can be ‘tuned’ via active stabilisation methods, either optically or electronically, to improve image quality. However, delegates discussed whether or not techniques from astronomy – such as using adaptive optics – could be implemented at a portable camera level in order to improve both camera and atmospheric stabilisation. In addition to this approach, computational imaging techniques were considered as an option for recovering image quality via post-processing restoration algorithms.

Computational Imaging (CI, and also known as Computational Optical Sensing and Imaging, COSI) covers a broad range of imaging techniques. From adding optical elements into the hardware (such as a coded aperture) that require specific restoration algorithms, through to machine-learning approaches where the camera processing can be trained to improve image quality or even extract the specific feature information directly, CI provides a real step-change in how we can approach imaging challenges. In this case, the delegates discussed the benefits of both hardware and ‘AI’ approaches, with the feeling that an ideal approach would involve both methods; for example using a coded aperture to recover wavelength defocus through the atmosphere, and then a deep-learning algorithm, such as a convolutional neural-network (CNN) to extract the person and make the recognition.

Overall, the workshop generated plenty of interesting and thought provokingthought-provoking ideas. It was great to see delegates from different industries and backgrounds sharing their experience and adding value into a common topic. It was surprising just how much in common there is – in both challenges and solutions – across the imaging community. This is a goal for the IET, to bring its members together, and it was rewarding to see business cards being exchanged and further discussions being planned.