It still would not have been likely that Z would have seen him come over the fence. Since there is a recreational facility of some sort in the area, it would not be unusual for someone to be walking in the direction that we can assume Martin to have been walking at that point.
That Martin was not known to Z is irrelevant, in my opinion, because it would be only remotely possible that he would know anywhere near every resident in that big an area and that many housing units. If anyone questions this, I would ask them how many of their neighbors they know over a similar area.
I cannot think of any valid reason to be so suspicious of a young person dressed appropriately for the time and the weather conditions other than that the person had some inate suspicion of the charter of any young black male whom he did not know by name.
Nothing that Z describes about Martin's behavior even seems to be probable cause to suspect him of something. It is kind of natural that, when you notice someone following you for no good reason, you would look at least a little apprehensive.
Certainly, if I were being followed, I would start looking for possible escape routes in case things got really hinky all of a sudden.