The VIMOS VLT deep survey: the build-up of the colour-density relation