I have yet to test this, but suppose i have a spawner that i created with spawn editor and the rectangle is over lapping a mountain by lets say have the rectangle, will the monsters spawn on top of the mountain?

the algorithm for placing spawns will only select locations with tiles that are flagged as passable. Mountain tiles are not passable so spawns would not be placed there by default, even if the spawn area included it.

You can override this restriction in a couple of ways. By preceding a spawn entry with an asterisk '*', you can force the spawn to be placed on a tile even it if isnt a valid, passable surface.
You can also use the #TILES or #NOTILES spawn control keywords to explicitly control the tiles that you want to be able to spawn on.

perfect, this will allow me to have bigger rectangles and less small ones, what about water spawns? will land animals spawn on water? because i know there are various water monsters

water is considered impassable, so they will not spawn on water.

To specify water-only spawns you can use the #WET spawn control keyword. For example

#WET; waterelemental

will spawn water elementals only on water tiles.