The greeks were the first to attempt this. Anaxagoras (500 b.c.), Aristarchus (300 b.c.) used the angle to the moon to calculate relative distances and sizes. Thinking because the moon is lit by the sun they would be able to measure the angles and therewith calculate the relative distances between earth-moon and earth-sun.

What they did was the following:


Imagine watching the moon at first quarter. The angle to the sun (to light only half of the moon) would have to be at a 90o angle. The smaller the angle (a) the moon was viewed at, the closer (d) the sun would have to be. Of course the lines in this picture originate from each centres of the sun, moon and earth.

Aristarchus measured an angle of 87o between the moon and the sun which results into the sun having to be 20 further away than the moon. Of course this was done without a telescope, and it was both hard to find the exact moment of first quarter, and actually measure the angles themselves. But the effort is to be applauded. Especially since trigonometry did not exist (sine/cosine or π had not been invented yet).
Using the same methods we would now be able to do the same experiment and see that the angle is in fact 89o50', which means the sun is actually 400 times further away than the moon.

Another experiment Aristarchus did, was using the lunar eclipse. (edit...)



Now would this work on a flat earth. Yes, because nowhere in his experiment he was using the shape of the earth into his measurements or calculations.

But there are several things to keep in mind.

  1. The angular size of the moon is (about) the same as the angular size of the sun.This has to mean that the distance to each object is exactly relative to each comparative size.
    If the sun if bigger than the moon, because the have the same angular size, the sun must be further away.
  2. We need the angle of the sun at 90 degrees, but also see exactly half of the moon lit.

flat sun moon wrong 1

In the above picture, the sun has the same angular size. The angle of the moon vs the sun is 90o, but the angle of the shadow line on the moon is not (b>0)

So this picture would be better:

flat sun moon wrong 2

But, lets put in the real angle for a (between the red lines), of 87o and see how that works out.. Only that would never fit in a readable picture: We know the distance to the sun is 400x the distance to the moon. In this picture with the moon at 35mm that would place the sun at 140cm, not sure how that would fit on your screen...
So lets do some math to figure out this 400x multiplier

sunmoonangles distance2

In the above picture there are 3 important (red) lines em (earth-moon), es (earth-sun) and ms (moon-sun)
We know the angles of a (87o) that was measured by Aristarchus, b (90o) which is the angle between earth, moon and sun, and is 90o because we have a half moon.
Now In this picture the sun is set too close, therefore the dashed lines, to indicate a larger distance, the two red-dashed lines will cross at the sun's position. Also the whole system is aligned with the surface of the earth, as that will make the calculations easier to follow.

In general a line is defined by ## y = \phi * x + b ##, however em is not defined as such, we must take an arbitrary value. This does make sense, as we do not try to calculate the distance to the moon, only the relative distance between the moon and the sun.

So for es ## y_{es} = \phi_{es} * x_{es} ##, now where ##\phi## is the corresponding slope with the angle α. We now know ##\phi = \alpha * \pi / 180 = 3 * 3,14 / 180 = 0,052##, but trigonometry had not been invented yet. We will continue this line of thought for now.

For ms it will be a horizontal line (the angle must be 90o), going through both the sun, and the moon, ## y_{ms} = \alpha_{ms} * x_{ms} + b##, and because ##\alpha_{ms}=0, y_{ms}=b ##, where b is not zero, but we can substitute it with the value of ##y_{em}##.

This will lead to the following

## y_{em} = y_{es} = y_{ms}##

## y_{es} = 0,052 * x_{es} ## or answering the question on what value of x, will the lines cross ## x_{es} = y_{es} * 0,052 = y_{es} * 19,1##. This implies that for every distance to the moon (em) the distance to the sun is 19 times that.

The method used however was not based on trigonometry, but using tables en euclid calculations. And that resulted not in a specific value, but an approximation of 

$$ 18 < \frac {es}{em} < 20 $$

Aristarchus’s On the Sizes and Distances of the Sun and the Moon: Greek and Arabic Texts by JL Berggren