It's normal for a baby's eyes to drift and wander during the first four to six months. This is especially noted when the baby is tired or is focusing on something very close to their face. What you are observing could be something called pseudoesotropia in babies, which is really an optical illusion caused by their flat nasal bridge. If you look at your baby's nose and then look at yours, you will see that yours is much larger and more defined. Babies have flat noses to facilitate breast feeding. The same is true for a baby's chin.
Since baby’s eyes are also set very wide apart, it often only looks like their eyes are crossed when they look in certain directions. The eyes should move in all directions and not stay stuck in one spot. Likewise, there shouldn't be any strange windscreen wiper like motions. If you notice either the eye staying in position or the strange motions, or observe drifting and wandering after six months of age, talk to your paediatrician.