See discussions, stats, and author profiles for this publication at: https://www.researchgate.

net/publication/261306459

A CMOS imager for embedded systems with integrated, real-time, motion-detection

capabilities and digital output

Conference Paper · October 2008

DOI: 10.1109/ICSENS.2008.4716434

CITATIONS READS

3 23

2 authors:

Gian nicola Angotzi M. Barbaro

Istituto Italiano di Tecnologia Università degli studi di Cagliari
28 PUBLICATIONS   185 CITATIONS    67 PUBLICATIONS   688 CITATIONS   

SEE PROFILE SEE PROFILE

Some of the authors of this publication are also working on these related projects:

use of conductive polymer composites as coatings for improved performance of implantable devices View project

Mixed-signal front-end for acquisition, elaboration and wireless transmisssion of nerural signals in BMI applications View project

All content following this page was uploaded by M. Barbaro on 01 September 2014.

The user has requested enhancement of the downloaded file.

 A CMOS imager for embedded systems with
integrated, real-time, motion-detection capabilities
and digital output
Gian Nicola Angotzi and Massimo Barbaro
Dept. of Electrical and Electronic Engineering
University of Cagliari
Cagliari, Italy)
Email: g.angotzi,barbaro@diee.unica.it
Abstract— A CMOS, smart, low-power imager with pre-
processing capabilities suitable for embedded systems, was re-
alized and successfully tested. The chip hosts an array of
64x64 active pixels for image acquisition and frame storage,
a programmable and reconfigurable analog row-processor for
parallel spatial and temporal filtering of an image row at
the time and a fully digital communication block for chip
configuration and frame grabbing. The row processor is capable
of implementing programmable and tunable 2D spatial IIR filters
and programmable temporal FIR filters with up to 8 taps. The
two computations may be cascaded on-chip in order to extract
motion information in real-time. The same row-processor can
be reconfigured into a parallel set of 64 8-bit A/D converters
to achieve fully digital read-out. The chip was fabricated in a
0.35um process by AMI Semiconductors, has a size of 6mm2 ,
hosts 120,000 transistors with a static power consumption of
4.7mW and is capable of a frame rate of 50frames/sec.
I. I NTRODUCTION
A growing interest for embedded systems integrating image
processing capabilities (i.e. face/gesture recognition, motion
detection, target tracking, etc.) is being shown in different
application fields such as surveillance, automotive, biometrics
and robotics. In such systems, the traditional approach based
on acquisition with CCD cameras and hardware/software
processing on digital platforms may be ineffective in achieving
constraints of low-power, real-time and integration. A number
of different CMOS imagers have thus been proposed, in the
last decade, since CMOS technology allows to incorporate
image acquisition capabilities as well as low-level processing
circuits, both digital or analog, on the same device [1]-[3].
In this work we present a reconfigurable CMOS imager
for real-time image processing which is intended to be a
part of a portable system, capable of performing different
kinds of spatio-temporal computations on the acquired im-
age. Spatial processing is based on the convolution of the
image with a programmable Gabor-like function (a decaying
exponential modulated by a cosine) useful for low-level tasks
needed for stereo depth estimation [4], texture analysis [5],
segmentation [6]. The temporal filter, which is a programmable
FIR filter with up to 8 taps (high-pass, band-pass, low-pass),
coupled to the Gabor filter is suited for implementation of
motion detection [7] and estimation of motion-in-depth [8].
1-4244-2581-5/08/$20.00 ©2008 IEEE 274
The chip architecture is flexible enough to allow computation
of other algorithms provided they can be described in terms
of weighted sums. In all these algorithms, the key point is
the possibility of interactively changing the parameters of the
kernel (frequency, envelope, gain, phase). Very fast output
rate is required to be able to perform multi-scale and multi-
frequency filtering of the same image.
II. C HIP A RCHITECTURE
The chip architecture shown in Figure 1 reflects the choice
of a semi-parallel processing approach, where a full portion
of the image (an entire row) is processed simultaneously,
allowing to meet both real time and low area constraints.
Acquisition of the image is implemented in the Array, made-
up of 64x64 active pixel sensors (Figure 1a) incorporating
a photodiode, 2 long term memories (LTM) needed to store
2 complete frames or 8 reduced frames, 2 output buffers
(LineCol and LineRow) and 1 input line (LineIn). Frame
storage is needed to compute temporal correlation. All the
processing is done in the analog row processor (AP), a
vectorial structure made-up of 64 identical elements (APU,
Analog Processing Units, Figure 1b). A row of the new image
or of any previously stored frame can be written in a battery
of 64 S&H (short-term memories, STM) and fed to a set of
64 programmable, switched capacitors, weighted adders which
implement a feed-forward FIR filter. The weighted adders can
also be interconnected one with the other to implement a
recursive spatial 1D IIR filter. The results of any computation
can be directly A/D converted (by reconfiguring the adders into
a successive approximation A/D converter) or stored back in
the S&H or even in the frame memories.
Notice that all the pixels of the same column as well as
all those of a same row share the same output channel so
that the Array is addressable by rows and by columns. This
is needed when 2D spatial filters are required: such filters are
implemented cascading a 1D horizontal filter and a 1D vertical
filter in such a way: a) a row image is read through LineCol
bus, b) the row is stored in the STMs, c) a 1D Gabor-like
filter is applied, d) the result is stored back in the LTMs of
IEEE SENSORS 2008 Conference
 TABLE I
I NTEGRATION TIME FOR DIFFERENT CONDITIONS OF LIGHTING

Lighting condition Lux Photocurrent Integration time

strong sunlight 100000 200 [pA] 300 [µs]
full daylight 10000 20 [pA] 150 [µs]
Normal office light 500 1 [pA] 3 [ms]

LTMm
fbCol fbRow ReadRowM writeM

rstRead
reset

SelMem

ReadColM
shutt
fbRow
ReadRowP writeP

selCol selRow

ReadColP
Acquisition outCol outRow
LTMp
Column Buffer Row Buffer

Fig. 2. Schematic of the Pixel

that affects the images during read-out is easily removed by

Fig. 1. Building blocks of the reconfigurable imager
the Array, e) the Array is accessed by columns (LineRow bus)
and finally (f) the columns are filtered again.
The Digital Control Unit (DCU) is used to produce the
correct sequence of operations, to program the shapes of the
spatial or temporal filters, to control the A/D conversion, to
rearrange the connection in the Switch Matrix to properly
reconfigure the Analog Processor and to transmit the digitally
converted output off-chip.
III. C IRCUITS
A. Pixel
The design of the pixel (whose schematic is depicted in
Figure 2) is critical since a number of constraints must be
fulfilled. In particular it is necessary to find a compromise
between the area (the Array grows with N 2 ) and the sensi-
tivity of the photodiode (which increases with its area). The
image is acquired integrating the photocurrent produced by a
photodiode (nplus on well) which is cascoded to reduce the
parasitic capacitance of the integration node. The cascode tran-
sistor acts as an electronic shutter, by modulation of its gate
(signal shutt). Since the integration time is programmable the
device can be used in different condition of lighting without
compromising the real-time of the elaborations (as reported in
Table I). The need to minimize the area of the pixel imposes
the use of minimum-size transistors for the read-out circuitry
causing a reduction of the precision. The Fixed Pattern Noise
275
fully differential processing units while short-channel effects
are reduced using two feedback signals (f bRow and f bCol).
The f bRow channel also provides an input path for signals that
must be stored in the Long Term Memories. In order to use the
same column buffer for the read-out by rows and by columns
(see section III-B), signals selRow and selCol as well as
signals outRow and outCol are shorted on the diagonal of the
Array. For the same reason the signals f bRow and f bCol are
shorted too. The Long Term Memories can be accessed in read
mode by rows (signals readRowM and readRowP ) and by
columns (signals readColM and readColP ) while in write
mode they can be accessed only by rows (signals writeM
and writeP ). Due to the non-idealities (charge redistribution
and clock feedthrough) each time a LTM is accessed in read
mode its data is affected by an error in inverse proportion to
the size of the memory capacitor. To save the area the LTM
are implemented by gate capacitors (with a saving of around
25% respect to a poly-poly2
  implementation). The area of the
Pixel is 24.7 × 24.7 um2 with a fill factor of about 30%.
B. Colum buffer
In Figure 3 is depicted the schematic of the Column Buffer.
It is made-up of a level shifter insensitive to the body effect.
To reduce the area only the portion inside the dashed square
is embedded inside the pixel, while the rest is shared by the
pixels of the same column (see section III-A). To improve
the precision the Column Buffer is locally biased (transistors
M 0 ÷ M 4) and a unitary gain buffer is used to match the
drain-source voltages of transistors M 7 and M 8. The unitary
gain buffer is a Miller OpAmp
  with a DC gain of 72dB,
V
GBW=10MHz and SR=1 µs . The transistors M 9 ÷ M 11
are used to reset the high parasitic capacitances at the nodes
 selFB selSUM selADC
VradcP
Vout
Y
VDD
preset M11 bf<6:0> b<6:0> sample
selADC In0P
M3 M2=M3=M4 topBC
GND
F20 C0 bottom hold
M4 In0P Vref
topSUM
F10
selSUM F1
A=1 fbPCA
Vref
Vin M7
F2 Vref

bottom side
M8=M7 F1a

F24 C4 Control
In4P
sel M5
F14
+ - outM

Feedback PCA
M6=M5 selHLS outP1
Vref

Program
P1

npreset M1 bf<6:0>
outM1
- +
VDD
X F20 C0
M10 In0M outP

bottom side
F10
INbf<6:0>
M1=M0
Vbias Vref
F1a
M9 Vref
GND F2
npreset
F24 C4 fbPCA
In4M selSUM F1
top
Fig. 3. Column buffer and feedback signal F14 Vref
bottom hold
Vref
topBC selADC In0M
to_STM to_FB from_STM bf<6:0> b<6:0> sample

Vin[0]
Column[i]

VradcM
STMtoLine0
Line0toSTM

Line1toFB
Line0toFB

selSUM selADC

Fig. 5. Integrated programmable adder and ADC

Topology Topology Topology Topology Topology Topology

Selection Selection Selection Selection Selection Selection
AddertoLine0

Sign
Selection
Sign
Selection
Sign
Selection
Vin[0] Vin[1] Vin[2] Vin[3] Vin[4]
fromAdder
Fig. 4.
X and Y in order to reduce the time needed for each reading.
C. Switch Matrix
The reconfigurability of the device, i.e. the capability of
performing different algorithms, is demanded to the Switch
Matrix whose schematic is depicted in Figure4. A number
of lines of signal distribution allows the interaction among
neighbouring columns, while a network of switches (Topology
selection) provides the right connection scheme depending on
the desired algorithm. Thanks to the fully differential archi-
tecture, the sign of each input can be digitally programmed
just swapping the inputs via a multiplexer (Sign selection).
Moreover, the Switch Matrix must correctly route the data
from/to the Array to/from the Analog Processor.
D. Programmable Adder
In order to save area and power consumption the pro-
grammable adder and the ADC are integrated into the same
switched capacitors circuit whose schematic is depicted in
Figure 5. The adoption of a fully differential architecture is
motivated by power-supply noise rejection. The operational
amplifier is a two-stage Miller OpAmp. The gain-bandwidth
Sign
Selection
Sign
Selection
Sign
Selection product of the OpAmp is GBW = 10 [M Hz] while the
DC open loop gain is A0 = 70 [dB]. The operating mode
of this module is set by signals selSum (adder mode) and
selADC (converter mode). When the circuit works in adder
Switch Matrix mode, two phases are needed to compute simultaneously the
weighted sum of five inputs. During the reset phase (F1 and
F1a high and F2 low) the OpAmp is connected in a unit-
gain feedback and the capacitors are put in parallel. During
the amplification phase (F1 and F1a low and F2 high) it is
computed the weighted sum of the inputs, being the weights
Ci
set by the ratios C f
(i=0. . . 4). Since each capacitor is a 7 bit
Programmable Capacitor Array (PCA) the weighted sum is
programmable too and the user can digitally set the shape of
the kernel. When the circuit works in converter mode (signal
selADC) two phases are needed to produce the whole 8 bit
digital word. During the first phase the OpAmp is connected
in a unitary gain feedback (signal F1a high) and the total
feedback capacitor (bits bf [6 : 0] all equal to one) is used
to sample (signal sample) and hold (signal hold) the input
signal that comes from the STM. During the second phase it
is implemented the well known bit cycling algorithm. In this
phase the OpAmp is in the open look configuration, working as
a comparator; the Miller capacitor is disabled and only the first
stage is used to speed-up settling of output. The bits produced
at successive approximations are stored in a bank of dynamic
flip-flops and are used to the digitally program the feedback
PCA. Once that the whole digital word has been produced it
remains stored in the bank of registers and can be read-out
while the circuit processes a new row/column.
276
 IV. E XPERIMENTAL RESULTS
The chip was designed and realized in a 0.35µm process
from AMI Semiconductors and its layout is shown in Figure 6
(due to metal fill, the microphotograph do not show many
details). The pitch of the columns is given by the size of the
pixel which is 24.7µm and determines the size of each unit
(APU) in the Analog Processor. Figure 7 shows the outputs
Sensor array and Row
scanning circuitry Processor
Digital
Control
Fig. 6. Layout of the chip
of the Analog Processor: 7(a)) original image injected for test
purposes in the STMs, 7(b)) read-out of the image through
the embedded A/D converters, 7(c)) test image filtered with
the spatial Gabor-like filter. It should be noted that there is
(a) Test image (b) Readout and A/D (c) Spatial filtering
conversion
Fig. 7. Test of the Analog Processor with an injected image
a background noise (the vertical stripes) introduced probably
by the mismatch of the two capacitors of the column S&H
(the signal is differential so 2 capacitors are needed). This is
due to the fact that the size of such capacitors is probably
too small. On the contrary, the convolution do not add further
background noise.
Figure 8 shows the output of the chip with internal acquis-
tion. In this case, all the blocks of the devices were stimulating,
from the Array, to the Analog Processor. Figures 8(a) and
8(b) show the acquisition and convolution of a simple image
with 2 vertical lines on a white background while Figures 8(c)
and 8(d) show a human face. In this case, the noise is higher
and includes the protion of noise introduced by the pixel. It
should be noted, again, that this are the preliminary results
obtained from a small number of prototypes and the device is
277
View publication stats
still under test since a number of parameters can be optimized
(and digitally set) to reduce the effect of such imperfections.
(a) Test image (b) Readout and (c) Spatial filter- (d) Spatial filter-
A/D conversion ing ing
Fig. 8. Images acquired and processed by the chip
The spatial Gabor filter is computed in less than 3µsec
while the temporal filter requires more time since the long-
term memories in the Array have to be accessed several times
to get the previous samples. Static power consumption is
6mW @3.3V for the 64 × 64 version.
V. C ONCLUSION
In conclusion, we conceived, designed and successfully
realized a reconfigurable CMOS imager for spatio-temporal
image processing with fully digital output. The versatility and
programmability of the Analog Processor allows to implement,
on the same chip, even different vision algorithms, provided
they can be accomplished by means of weighted sums of
different pixels (convolutions). The device is currently under
further tests to increase its sensitivity by proper choice of the
programmable parameters.
ACKNOWLEDGMENT
The authors would like to thank Giovanni Busonera, Daniela
Loi, Alessandra Caboni, Paolo Meloni and Giamarco Angius
for helping with the test setup and Xilinx for hardware
donation.
R EFERENCES
[1] K. Boahen and A. Andreou, “A 590000 transistor, 48000 pixel contrast
sensitive, edge-enhancing cmos imager silicon retina,” Proceedings of the
16th Conference on Advanced Research in VLSI, pp. 225–240, 1995.
[2] M. Barbaro, P.-Y. Burgi, A. Mortara, P. Nussbaum, and F. Heitger, “A
100100 pixel silicon retina for gradient extraction with steering filter
capabilities and temporal output coding,” IEEE J. Solid-State Circuits,
vol. 37, no. 2, pp. 160–172, 2002.
[3] L. Massari, M. Gottardi, L. Gonzo, D. Stoppa, and A. Simoni, “A CMOS
image sensor with programmable pixel-level analog processing,” IEEE
Trans. Neural Networks, vol. 16, no. 6, pp. 1673–1684, 2005.
[4] T. Sanger, “Stereo disparity computation using Gabor filters,” Biol.
Cybern., vol. 59, pp. 405–418, 1988.
[5] D. Dunn and W. Higgins, “Optimal gabor filters for texture segmentation,”
IEEE Transactions on Image Processing, vol. 4, pp. 947–964, July 1995.
[6] J. Van Demeeter and J. Du Buf, “Simultaneous detection of lines and
edges using compound gabor filters,” International Journal of Pattern
recognition and analysis, vol. 14, p. 757, 2000.
[7] D. Clausi and M. Jernigan, “A general model for visual motion detection,”
Proceedings of the 9th International Conference on Neural Information
Processing (ICONIP-’02), vol. 1, 2002.
[8] S. Sabatini, F. Solari, P. Cavalleri, and G. Bisio, “Phase-based binocular
perception of motion-in-depth: cortical-like operators and analog vlsi
architectures,” EURASIP Journal of Applied Signal Processing, vol. 7,
pp. 690–702, 2003.