Find Pixim Powered CamerasSide by Side

Keeping an eye on security


Wide dynamic range image sensor-processor improves security video and other applications

January 16, 2007
From: Homeland Security Asia
By: John Monti

The extended dynamic range, fast readout speed, system-on-chip integration and low power operation of Digital Pixel System (DPS) technology from Pixim represents a substantial advance over existing image capture and processing technologies. Enhancing video image capture for many applications, DPS is particularly useful for security cameras that must provide detail over a wide range of illumination conditions.

Solid-state image sensor technology dates back to the invention of the first charge coupled device (CCD) in the late 1960s. The 1990s saw the introduction of the CMOS active pixel sensor (APS), followed by the development of the DPS platform. The invention of DPS technology grew out of more than eight years of research at Stanford University. Pixim subsequently commercialized DPS three years later. Pixim's recently announced 'Orca' D2500 chipset provides all of the functions necessary to build advanced surveillance cameras and emerging imaging applications in two small Ball Grid Array devices.

The Digital Pixel Systems architecture and tightly-coupled imaging software provide superior image quality even under widely varying light conditions and wide dynamic range scenes that have both dark and bright areas. DPS provides dramatically better wide dynamic range images than charge-coupled device (CCDs) or CMOS active pixel sensors (APS) used in similar video camera applications. The highly integrated two-chip set is comprised of a digital image sensor chip and a digital image processor chip. Layout of chip interconnections and user interface is straightforward. The user interface includes a customizable menu-driven on-screen display (OSD), switches and potentiometers, in any combination. Manufacturers define which of many settings are available for user control and which are fixed at the factory. Fixed settings can be locked so that they may not be identified or changed in the field, if desired. Additionally, DPS technology offers:


DPS Image capture and processing

DPS technology converts the quantity of light striking each picture element (pixel) to a digital value at the earliest possible point: at the pixel itself. An analog-to-digital converter (ADC) is designed into each pixel, and is operated simultaneously with all other ADCs in every pixel of the sensor. This pixel-level ADC architecture permits the use of many highly parallel low-speed circuits, operating close to where the photodiode signals are generated. This is key to optimizing the signal-to-noise ratio (SNR) for each pixel.

The DPS system uses the individual ADCs in each pixel to perform non-destructive correlated double sampling (CDS) at each pixel. DPS uses this capability to sample the growing light intensity at each pixel many times during each image capture period. This allows exposure level of each pixel to be determined by the rate of change of charge collected rather than only its absolute magnitude. Each pixel is also provided with an adjustable offset cancellation gain amplifier to assure uniform response throughout the sensor array. These innovations greatly reduce noticeable fixed pattern noise problems commonly associated with the column-level ADC used on APS sensors.

Because DPS sensors are digital, pixel readout is much faster and more accurate. Each sample of the digital image is captured in on-chip RAM. The high bandwidth provided by tightly coupled local memory is used to achieve its superior high dynamic range. This approach is not practical for CCD or APS sensors because of their reliance on analog readout circuitry. This is not a problem with DPS, which greatly benefits from the digital sampling performed on each pixel.

Wide dynamic range

Wide dynamic range is essential for capturing image detail at all light levels. Today's surveillance cameras are plagued by dynamic range problems in a typical 24 hour day, due to severe reflections, glare, car headlights, and direct sunlight. The wide dynamic range achieved by DPS is realized by a patented non-destructive multi-sampling image capture capability, and advanced image-processing algorithms.

Dynamic range is the ratio of the brightest image that can be captured by the imaging system to the darkest image that can be captured. Light intensity greater than the brightest possible image will cause the sensor to saturate, while light intensity less than the darkest possible image will not register on the sensor. Both of these conditions distort the image, hiding potentially vital information that lies outside the dynamic range of the sensor.

When an exposure begins, each pixel is charged at a rate that is proportional to the intensity of the light that strikes it. A stronger light source will charge a pixel more quickly than a weaker light source. Existing analog technology typically uses a single exposure time for all pixels. At the end of the exposure, the camera will sense the total charge accumulated in each pixel. But that means some pixels (the brighter ones) may be overexposed while others (the darker ones) may be underexposed.

DPS overcomes this limitation as follows: with DPS, the light striking each pixel is sampled multiple times during the exposure period. DPS analyzes how quickly each pixel is being charged by the light striking it. This way, DPS measures light intensity by a combination of the rate at which the charge grows as well as the total charge accumulated during an entire exposure.

Specifically, the DPS system records the length of time required to nearly saturate each pixel. Pixels exposed to bright illumination will tend to saturate more quickly than other pixels. DPS determines for each pixel whether it will saturate before the next sample. If a pixel would saturate, then its elapsed exposure time is stored in memory, together with its current intensity of charge.

The advantage of this approach can be appreciated when one realizes that the entire range of each individual pixel, as well as the rate of change of the pixel charge, is used to form the resulting image, significantly increasing the dynamic range and SNR that is captured. Other technologies only measure the pixel value, not its rate of change.

DPS also provides improved color performance not available with other sensor technologies: the data recorded by each pixel is of very high quality, both in terms of accuracy and precision. High data quality allows the DPS image processing algorithms to render excellent fidelity for all colors and intensities. In surveillance applications, color accuracy is critical to forensic analysis of stored video once an incident has been recorded.

DPS provides a fast global electronic shutter to capture bright lights and produces images that do not exhibit rolling shutter artifacts common in APS sensors, or interlace artifacts common to CCD sensors. Since multi-sampling is fundamental to DPS and is included in the basic firmware, no programming is required by developers to achieve this level of quality.

Improving signal to noise ratio

The signal-to-noise ratio (SNR) is a measure of a captured image's immunity to noise interference. Peak SNR is the ratio of the strongest recordable intensity without saturation to the background noise. A camera with higher SNR typically produces better (less noisy) video in darker scenes. Photodiodes generally have better SNR when they are charged to more than half of their total capacity. But they cannot be charged beyond saturation. In other (analog) sensor technologies, high contrast portions of an image cause saturation of the photodiode and "blooming" in adjacent pixels.

The SNR in a DPS sensor is greater because each pixel is measured at its maximum value just prior to saturation. For example, for the photograph shown in Figure 2, the best exposure times are: T2 for the apple, T4 for the region below the cup, and T6 for the shadowed portion of the car because it is at these times that the photocurrent produced by these pixels is highest prior to saturation or the end of the exposure interval. Therefore, these are the times when the image is the most accurately represented by these pixels.

DPS reduces noise in the sensor in a number of ways. First, a negative feedback unity gain amplifier in each pixel eliminates any offset voltage, resulting in much greater uniformity throughout the sensor array. To minimize reset noise, each pixel value is read non-destructively at the beginning of the exposure, and this value is later subtracted from the final measured value for the pixel. This non-destructive method of correlated double sampling is unique -- most other sensors must read the CDS value, reset the sensor, then capture a new value. But the random background noise change from the initial captured CDS value introduces distortion that DPS avoids. This reduces noise that would be detectable to the human eye.

The high quality of Pixim's pixels provides high signal levels and low noise levels, resulting in excellent inherent SNR characteristics. This can provide sensitivity on par with CCDs, and an order of magnitude sensitivity gain over APS sensors, while providing extended dynamic range and excellent image quality.

Pixim DPS image sensing technology

Because DPS technology integrates the ADC into the pixel, it can be manufactured using leading edge semiconductor processes. The pixel array has significantly higher noise immunity than analog sensors because DPS technology employs a digital readout from each pixel. Additional image processing and camera functions implemented by Pixim provide a complete imaging solution in high-volume, commercially available chipsets, the latest of which is the Orca product family.

DPS image systems integrate sensing, memory and processing functions into two chips. This is especially important for imaging systems that require significant processing, where quality of output is crucial, and where small size, low power and time-to-market are important. Until now, analog-to-digital conversion could only be integrated at the chip or column level. Both approaches are common in analog sensor solutions. For the chip-level approach, a single conventional high-speed ADC is integrated with the sensor. For the column-level approach, one or more columns of the pixel array has a dedicated ADC. The ADCs are operated in parallel and, therefore, low-to-medium speed conversion techniques must be used (e.g., single-slope, algorithmic, successive approximation, or over-sampling).

By having separate ADCs for each pixel operating in parallel, the ADCs can operate at very low speed, operating at a few thousand samples per second. This lessens noise and reduces power requirements of DPS sensors.

The large bit stream used to read the ADCs is supported by on-chip RAM. These features enable much faster and more accurate readout characteristics. The DPS sensor consists of a digital pixel sensor array, ADCs and RAM. A separate image processor chip incorporates digital signal processing (DSP), a second video frame buffer, and I/O. The sensor core is powered by a single low-voltage power supply. Manufacturing is simpler than for CCDs and greater miniaturization is realized. These advantages will continue to increase in the future in accordance with Moore's Law.

Pixel-level ADC: The DPS pixel-level ADC architecture permits the use of low-speed conversion; the ADCs operate close to where the photodiode signals are generated. This optimizes signal-to-noise ratio (SNR) and power consumption beyond any sensors currently available. The large number of independent, small ADCs significantly reduces noticeable fixed pattern noise problems commonly associated with the column-level ADC variation on APS sensors. Finally, because pixel signals are available to ADCs at all times, the number and timing of images—as well as the number of bits from each image—can be freely chosen. This offers important advantages, such as the ability to optimize the image capture and processing to the scene characteristics. All of these features have been implemented in Pixim DPS systems.

DPS pixel readout: Each pixel's ADC feeds into a RAM-speed bit-serial readout. The multi-channel bit-serial design reduces the ADC output data rate, allowing it to use standard Nyquist-rate conversion instead of over-sampling. It facilitates DPS dynamic range enhancement via multiple sampling, and reduces non-uniformity by globally distributing the ADC control and clock signals, and by performing auto-zeroing. The implementation consists of a two-dimensional array of pixel blocks, a row decoder and column sense amplifiers and latches.

Each pixel block comprises one or more photo detectors sharing an ADC channel. The captured analog pixel values are digitized in parallel, one bit at a time. Each latched set of bits, using the row decoder and column sense amplifiers, forms a bit-plane that is read out in a manner similar to a standard digital memory. As depicted in Figure 6, a set of bit planes constituting an m-bit frame of the data is collected during a single exposure window. In this example, the digitized value of the uppermost right-hand corner pixel is 1001...1. Note that this image output format is quite different from the raster-scan format commonly used in CCD and APS devices.

The DPS system: Beyond the fundamental DPS technology, Pixim has integrated functionality into the silicon and software to produce a comprehensive system solution. Developers can build high-performance video surveillance cameras with straightforward hardware design and simple configuration using the Pixim Configuration Language (PCL) to control the hardware interfaces, develop menus, and control the imaging behavior of the camera. No signal processing software development is necessary. Pixim also provides complete reference designs and a variety of camera configurations for very quick time to market.

John Monti, the founding executive of Pixim, Inc., is Vice President of Sales and Marketing. He draws on experience in the fields of semiconductor applications, marketing and system-on-chip solutions, and is a published writer and speaker. For more information, e-mail: monti@pixim.com.

Language Options