Correlated and Uncorrelated Signals

Problem: we have two signals and . How close are they to each other?
] [n x ] [n y
Example: in a radar (or sonar) we transmit a pulse and we expect a return
0.4
0.6
0.8
1
Transmit
0 2 4 6 8 10 12 14 16 18 20
-1
-0.8
-0.6
-0.4
-0.2
0
0.2
Transmit
0 5
0
0 . 5
1
1 . 5
Receive
0 2 0 4 0 6 0 8 0 1 0 0 1 2 0 1 4 0 1 6 0 1 8 0
- 2
- 1 . 5
- 1
- 0 . 5
Example: Radar Return
Since we know what we are looking for, we keep comparing what we receive
with what we sent.
-0.6
-0.4
-0.2
0
0.2
0.4
0.6
0.8
1
-0.6
-0.4
-0.2
0
0.2
0.4
0.6
0.8
1
0 . 5
1
1 . 5
0 2 4 6 8 10 12 14 16 18 20
-1
-0.8
0 2 4 6 8 10 12 14 16 18 20
-1
-0.8
- 1 . 5
- 1
- 0 . 5
0
Receive
0 2 0 4 0 6 0 8 0 1 0 0 1 2 0 1 4 0 1 6 0 1 8 0
- 2
Similar?
NO! Think so!
Inner Product between two Signals
We need a measure of how close two signals are to each other.
This leads to the concepts of This leads to the concepts of
Inner Product
Correlation Coefficient
Inner Product
Problem: we have two signals and . How close are they to each other?
] [n x ] [n y
Define: Inner Product between two signals of the same length
1 N

=
=
0
*
] [ ] [
N
n
xy
n y n x r
Properties: Properties:
0 ] [ ] [ ] [
1
0
2
1
0
*
= =


=

=
N
n
N
n
xx
n x n x n x r
yy xx xy
r r r
2
r r r
2
if and onl if
] [ ] [ n x C n y = for some constant C
yy xx xy
r r r = if and only if
] [ ] [ n x C n y = for some constant C
How we measure similarity (correlation coefficient)
Assume: zero mean
xy
r | |
=
Compute:
yy xx
xy
r r
=
Check the value:
1 0
xy

11
xy

x,y strongly correlated x,y uncorrelated

0
xy

Back to the Example: with no return

1
1.5
2
2
3
2
3
] [n x
] [n y ] [ ] [ n y n x
-1
-0.5
0
0.5
1
-1
0
1
-1
0
1
0 100 200 300 400 500 600 700 800 900
-2
-1.5
1
0 100 200 300 400 500 600 700 800 900
-3
-2
0 100 200 300 400 500 600 700 800 900 1000
-3
-2
500
27 . 2
=
=
xy
r
r
003 . 0
982
500
=
=
=
xy
yy
xx
r
r

NO Correlation!
xy

NO Correlation!
Back to the Example: with return
1
1.5
2
2
3
2
2.5
] [n x ] [n y ] [ ] [ n y n x
-1
-0.5
0
0.5
-1
0
1
0.5
1
1.5
0 100 200 300 400 500 600 700 800 900
-2
-1.5
0 100 200 300 400 500 600 700 800 900
-3
-2
0 100 200 300 400 500 600 700 800 900 1000
-0.5
0
494 =
xy
r
754
500
=
=
yy
xx
xy
r
r
8 . 0 =
yy
Good Correlation!
Inner Product in Matlab
[ ] ) ( ) 2 ( ) 1 ( N x x x x =
Row vector
Take two signals of the same length. Each one is a vector:
[ ] ) ( ) 2 ( ) 1 ( N x x x x =
[ ] ) ( ) 2 ( ) 1 ( N y y y y =
Row vector
Row vector
[ ]
(
(
(

) 2 (
) 1 (
) ( ) 2 ( ) 1 ( ) ( ) (
*
*
*
y
y
N x x x n y n x r
N
Define: Inner Product between two vectors
[ ]
(
(
(
(

= =

=
) (
) ( ) 2 ( ) 1 ( ) ( ) (
*
1
N y
N x x x n y n x r
n
xy

x
' * y x r
xy
=
' y
conjugate,
transpose
Example
Take two signals:
2.5
y
3
0 5
0
0.5
1
1.5
2
x
y
0
1
2
0 50 100 150 200 250 300
-2.5
-2
-1.5
-1
-0.5
0 50 100 150 200 250 300
-3
-2
-1
Compute these:
Then:
0 0856 . 0
7 . 19
= =
xy
7 . 19 ' * = = y x r
xy
9 . 241 8 . 218
xy

x,y are not correlated

y
xy
8 . 218 ' * = = x x r
xx
9 241 ' * = = y y r 9 . 241 y y r
yy
Example
Take two signals:
Compute these:
3
x
9 . 230 ' * = = y x r
xy
6 . 229 ' * = = x x r
0
1
2
6 . 229 x x r
xx
3 . 234 ' * = = y y r
yy
0 50 100 150 200 250 300
-3
-2
-1
y
Then:
1 9955 . 0
3 234 6 229
9 . 230
=

=
xy

1
2
3
3 . 234 6 . 229
x,y are strongly correlated
-2
-1
0
0 50 100 150 200 250 300
-3
Example
Take two signals:
y
3
3
x
y
0
1
2
0
1
2
0 50 100 150 200 250 300
-3
-2
-1
-3
-2
-1
Compute these:
Then:
1 9955 . 0
3 234 6 229
9 . 230
=

=
xy

0 50 100 150 200 250 300

9 230 ' * = = y x r
3 . 234 6 . 229
y
x,y are strongly correlated
9 . 230 y x r
xy
6 . 229 ' * = = x x r
xx
3 234 ' * y y r 3 . 234 ' * = = y y r
yy
Typical Application: Radar
] [n s
Send a Pulse
] [
n
N
] [n y
and receive it back with noise, distortion
n
0
n
Problem: estimate the time delay , ie detect when we receive it.
0
n
Use Inner Product
Slide the pulse s[n] over the received signal and see when
the inner product is maximum:

=
+ =
1
0
*
] [ ] [ ] [
N
ys
s n y n r

0
if , 0 ] [ n n n r
ys

] [ y

n
] [ s

0
n

N
Use Inner Product
Slide the pulse x[n] over the received signal and see when
the inner product is maximum:
if
0
n n =
MAX s n y n r
N
ys
= + =

=
1
0
*
] [ ] [ ] [

] [ y

0
n =
] [ s

N
Matched Filter
Take the expression
1 N
] [ ] 0 [ ] 1 [ ] 1 [ ... ] 1 [ ] 1 [
] [ ] [ ] [
* * *
1
0
*
n y s n y s N n y N s
s n y n r
N
n
ys
+ + + + + =
+ =

=

] [ ] [ ] [ ] [ ] [ ] [ y y y
Compare this, with the output of the following FIR Filter
Then
] 1 [ ] 1 [ ] 1 [ ] 1 [ ... ] [ ] 0 [ ] [

+ + + + = N n y N h n y h n y h n r
] [n y
] [n h
*
] 1 [ ] [

+ = N n r n r
ys
1 ,..., 0 ], 1 [ ] [
*
= = N n n N s n h
Matched Filter
This Filter is called a Matched Filter
] [n y
] [ n r
] [n h
] 1 [ ] [

+ = N n r n r
ys
The output is maximum when
1 ,..., 0 ], 1 [ ] [
*
= = N n n N s n h
1 n N n = + The output is maximum when
0
1 n N n = +
1
0
+ = N n n i.e.
Example
0.4
0.6
0.8
1
] [
1 ,..., 0 ], [ = N n n s
We transmit the pulse shown below, with
length
20 = N
0 2 4 6 8 10 12 14 16 18 20
-1
-0.8
-0.6
-0.4
-0.2
0
0.2
] [n s
Max at n=119
10
12
1
1.5
] [n y
Received signal:
100 1 20 119
0
= + = n
0
2
4
6
8
-1
-0.5
0
0.5
1
] [n y
] [ n r
] [n h
0 20 40 60 80 100 120 140 160 180 200
-6
-4
-2
0 20 40 60 80 100 120 140 160 180
-2
-1.5
1 ,..., 0 ], 1 [ ] [
*
= = N n n N s n h
How do we choose a good pulse
1 ,..., 0 ], [ = N n n s
We transmit the pulse and we receive
(ignore the noise for the time being)
] [ ] [
0
n n As n y =
] [ n r
] [n h
] 1 [ ] [

+ = N n r n r
ys
1
*
N

1 ,..., 0 ], 1 [ ] [
*
= = N n n N s n h
where
] [
] [ ] [ ] [
0
0
*
0
n n r A
s n n s A n r
ss
n
ys
=
+ =

=

where
The term
is called the autocorrelation of s[n]. This characterizes

=
+ =
1
0
*
] [ ] [ ] [
N
ss
s n s n r

the pulse.
Example: a square pulse
] [n r
ss ] [n s
1
N
n
1 N
0
N N n
? ] [ ] [ ] [
1
0
*
= + =

=
N
ss
s n s n r

N s s s r
N N
ss
= = =

1
2
1
*
] [ ] [ ] [ ] 0 [
ss
= = 0 0
] [ ] [ ] [ ] [

1 1 ] [ ] 1 [ ] 1 [
2
0
2
0
*
= = + =


=

=
N s s r
N N
ss


See a few values:
k N s k s k r
k N k N
ss
= = + =


=

=
1
0
1
0
*
1 ] [ ] [ ] [


k N k k
N k N
+

1 1
*
1 ] [ ] [ ] [ k N s k s k r
k
ss
= = + =

= =0
1 ] [ ] [ ] [


Compute it in Matlab
] [n s
1
18
20
n
1 N
0
10
12
14
16
N=20; % data length
s=ones(1,N); % square pulse
4
6
8
10
( ) q p
rss=xcorr(s); % autocorr
n=-N+1:N-1; % indices for plot
-20 -15 -10 -5 0 5 10 15 20
0
2
stem(n,rss) % plot
Example: Sinusoid
15
20
25
0.6
0.8
1
0
5
10
-0.6
-0.4
-0.2
0
0.2
0.4
50 40 30 20 10 0 10 20 30 40 50
-20
-15
-10
-5
0 5 10 15 20 25 30 35 40 45 50
-1
-0.8
49 ,..., 0 ], [ = n n s
-50 -40 -30 -20 -10 0 10 20 30 40 50
49 ,..., 49 ], [ = n n r
ss
Example: Chirp
0.4
0.6
0.8
1
20
25
30
-0.6
-0.4
-0.2
0
0.2
0
5
10
15
0 5 10 15 20 25 30 35 40 45 50
-1
-0.8
-50 -40 -30 -20 -10 0 10 20 30 40 50
-10
-5
49 ,..., 49 ], [ = n n r
ss
49 ,..., 0 ], [ = n n s
s=chirp(0:49,0,49,0.1)
Example: Pseudo Noise
1.5
2
2.5
30
40
50
-2
-1.5
-1
-0.5
0
0.5
1
0
10
20
30
49 0 ] [
0 5 10 15 20 25 30 35 40 45 50
-2.5
2
-50 -40 -30 -20 -10 0 10 20 30 40 50
-20
-10
49 ,..., 49 ], [ = n n r
ss
s=randn(1,50)
49 ,..., 0 ], [ = n n s
Compare them
1 5
2
2.5
0.6
0.8
1
0.8
1
] [n s
cos
chirp pseudonoise
-1.5
-1
-0.5
0
0.5
1
1.5
-0.8
-0.6
-0.4
-0.2
0
0.2
0.4
-0.8
-0.6
-0.4
-0.2
0
0.2
0.4
0.6
] [
50
25
30
0 5 10 15 20 25 30 35 40 45 50
-2.5
-2
0 5 10 15 20 25 30 35 40 45 50
-1
0.8
0 5 10 15 20 25 30 35 40 45 50
-1
25
] [n r
0
10
20
30
40
0
5
10
15
20
25
-10
-5
0
5
10
15
20
] [n r
ss
-50 -40 -30 -20 -10 0 10 20 30 40 50
-20
-10
-50 -40 -30 -20 -10 0 10 20 30 40 50
-10
-5
-50 -40 -30 -20 -10 0 10 20 30 40 50
-20
-15
10
Two best!
Detection with Noise
Now see with added noise
] [ ] [ ] [
0
n w n n As n y + =
] [n h
] [ ] 1 [ ] [

0
n r N n n r n r
yw ys
+ + =
] [n h
1 ,..., 0 ], 1 [ ] [
*
= = N n n N s n h
White Noise
A first approximation of a disturbance is by White Noise.
White noise is such that any two different samples are
l t d ith h th uncorrelated with each other:
2
3
] [n w
1
0
1
2
] [
-3
-2
-1
0 100 200 300 400 500 600 700 800 900 1000
-4
White Noise
The autocorrelation of a white noise signal tends to be a
delta function, ie it is always zero, apart from when n=0.
] [n r
ss
n
White Noise and Filters
The output of a Filter
] [

=
1
] [ ] [ ] [
N
n w h n w
] [n w
] [n h

=0
] [ ] [ ] [ n w h n w

|
|
.
|

\
|
=

1 1 1
2 1 2 1
1
2
] [ ] [ ] [ ] [
1
] [
1
M N N M
n w n w h h
M
n w
M

|
.
|

\
|
=
|
.

\


= = = =
1 1 1
2 1 2 1
0 0 0 0
] [ ] [
1
] [ ] [
1 2
N N M
n n
n w n w
M
h h
M M


|
.
|

\
|
|
.
|

\
|
=
. \



= = =
1
2
1
2
0 0 0
2 1 2 1
] [
1
] [
1 2
M N
n
n w h
M

|
.

\
|
.

\

= = 0 0
] [ ] [
n
M

White Noise
The output of a Filter
] [

=
N
n w h n w ] [ ] [ ] [
] [n w
] [n h

=
n w h n w
0
] [ ] [ ] [

In other words the Power of the Noise at the ouput is In other words the Power of the Noise at the ouput is
related to the Power of the Noise at the input as
N
| |
1
w
N
n
W
P n h P |
.
|

\
|
=

=
1
0
2
] [
Back to the Match Filter
] [ ] 1 [ ] [ N A
] [ ] [ ] [
0
n w n n As n y + =
] [n h
] [ ] 1 [ ] [
0
n w N n n Ar n r
ss
+ + =
At the peak:
1 ,..., 0 ], 1 [ ] [
*
= = N n n N s n h
At the peak:
] 1 [ ] 0 [ ] 1 [

0 0
+ + = + N n w Ar N n r
ss
Match Filter and SNR
At the peak:
] 1 [ ] 0 [ ] 1 [

0 0
+ + = + N n r Ar N n r
sw ss
|
.
|

\
|
|
.
|

\
|
=


=

=
1
0
2
1
0
2
2
| ] [ | | ] [ | ] 0 [
N
n
N
n
ss
n s n As Ar
N
| |
1
W
N
n
W
P n s P |
.
|

\
|
=

=
1
0
2
| ] [ |
P N
N
|
|

1
2
] [
SNR N
P n s
n s P N
SNR
W
N
n
S
peak
=
|
.
|

\
|
|
.
|

\
|

=

=
1
2
0
] [
] [
W
n . \

=0
Example
Transmit a Chirp of length N=50 samples, with SNR=0dB
2 30
0
0.5
1
1.5
5
10
15
20
25
-2
-1.5
-1
-0.5
0 200 400 600 800 1000 1200
-15
-10
-5
0
5
0 50 100 150 200 250 300
2
0 200 400 600 800 1000 1200
Transmitted
Detected with
Matched Filter
Example
Transmit a Chirp of length N=100 samples, with SNR=0dB
1
1.5
2
30
40
50
-1
-0.5
0
0.5
0
10
20
0 50 100 150 200 250 300
-2
-1.5
0 200 400 600 800 1000 1200
-20
-10
Transmitted
Detected with
M t h d Filt Matched Filter
Example
Transmit a Chirp of length N=300 samples, with SNR=0dB
2 140
160
0
0.5
1
1.5
60
80
100
120
0 50 100 150 200 250 300
-2
-1.5
-1
-0.5
-20
0
20
40
0 200 400 600 800 1000 1200 1400
-40
Transmitted
Detected with
Matched Filter Matched Filter