Matthew's account may not be as full as we would like it to be. There was more than a dream, there was the prophecy, an angel appeared to Zacharias and Elizabeth who mothered John the Baptist. they were relatives to Mary. Mary also saw an angel and I am sure she would have told Joseph about that. So we have a lot more to go on than just a dream, I too would be a little less trilled to follow a dream. Also the Holy Spirit (yeah yeah I know) can prove things to us. One day I absolutely doubted God, the Bible and all that spiritual stuff and the next day I was fully convinced. I opened up to God and He did the convincing.

Matthew 1:24