How far is NYC's Nautical Community?

Submitted by ub on Sun, 01/08/2012 - 18:56

If the Earth were flat, we could find the distance between any two points using the Pythagorean theorem. It would just be the square root of (latitude1 - latitude2)^2 + (longitude1 - longitude2)^2. However, since the Earth is not flat, a more complex formula is needed. If the two points are close to each other, the above formula will be very close to correct, but if they are distant, it will not.

Open Excel and enter the latitude of the first location in cell A1 and B1; enter degrees in A1 and minutes in B1. For example, New York City is at 40 degrees 47 minutes north, so enter 40 in A1 and 47 in B1.

Enter the longitude of the first location in cells A2 and B2, as above. New York City is at 73 degrees 58 minutes west, so enter 73 in A2 and 58 in B2.

Do similarly for the second location. For example, Honolulu is at 21 degrees 18 minutes north, 157 degrees 50 minutes west, so enter 21, 18, 157 and 50 in cells A3, B3, A4 and B4, respectively.

Convert all locations to decimal degrees. In cell C1 enter = a1 + a2/360, then copy this to cells C2, C3 and C4.

Convert all locations to radians. In cell D1 enter = c1/180*3.1415 and copy that to D3, D4 and D5.

Calculate the distance. In cell C6 enter =ACOS(SIN(D1)*SIN(D2)+COS(D1)*COS(D2)*COS(D4-D3))*6371.

Latitude 40 Degrees 47 Minutes N / Longitude 73 Degrees 58 Minutes West