Correlation of two variables over time: can this happen?

18 views (last 30 days)
alpedhuez on 17 Dec 2020
Edited: alpedhuez on 17 Dec 2020
I have a variable x1 and x2 between January 1 till December 31. When I calculate the correlation between Janaury and June it is positive. When I calculate the correlation between July to December it is positive. But the correlation between Janauary and December is negative. Can it happen?
Matt Gaidica on 17 Dec 2020
Yes. However, you probably want to perform a statistical test to determine if the fluctuating correlation is significant.

the cyclist on 17 Dec 2020
Yes. This is known as Simpson's Paradox. Here is an example:
rng default
x1 = [1 2 3 4 6 7 8 9]';
x2 = [1 2 3 4 -6 -5 -4 -3]' + 0.8*rand(8,1);
% Correlation of first half
corrcoef(x1(1:4),x2(1:4))
% Correlation of second half
corrcoef(x1(5:8),x2(5:8))
% Correlation of entire vector
corrcoef(x1,x2)
% Plot it
figure
scatter(x1,x2) You can see that the first half and the second half are positively correlated with each other, but if you look at the trend over the entire vector, it is negative.
alpedhuez on 17 Dec 2020
Yes I now understand confounding factor t.
x1 and x2 has a postive correlation between date 1 and date 2.
x1 and x2 has a positive correltion between date 2 and date 3.
x1 and x2 has a negative correlation betwen date 1 and date 3.
Regression of x2 on x1 between date 1 and date 2 has positive statistically sig coef.
Regression of x2 on x1 between date 2 and date 3 has positive statistically sig coef.
Regression of x2 on x1 between date 1 and date 3 has negative statistically sig coef.
Regression of x2 on x1 and time between date 1 and date 2 has positive statistically sig coef for both x1 and time.
Regression of x2 on x1 and time between date 2 and date 3 has positive statistically sig coef for both x1 and time.
Regression of x2 on x1 and time between date 1 and date 3 has positive statistically sig coef for both x1 and time.
How can it happen?

Matt Gaidica on 17 Dec 2020
Edited: Matt Gaidica on 17 Dec 2020 alpedhuez on 17 Dec 2020
I am happy to explain to get comments. But I am not sure whether I want to discuss the problem in a public bulletin board. So if there is a suggestion for a venue I will be happy to listen.