Sine and cosinethe numbers behind rotation
Cosine and sine are a turning point's coordinates. They rotate any arrow, and a dot product feels only the gap between two turns.
Before this
Next in Calculus
Rates of changeCalculus, stop 4 of 122 minTranscript
It follows the voice. Click a sentence to jump there.
On a circle of radius one around the origin, put a point on the positive x axis. Turn it counterclockwise through an angle called theta. Its x coordinate is the cosine of theta; its y coordinate is the sine of theta. At thirty degrees, the cosine is about zero point eight seven; the sine is exactly one half.
Keep the point turning steadily, and graph its height, the sine, over time. It rises and falls between one and minus one: a wave, one cycle per turn. Double the speed, and the wave's frequency, how often it repeats, doubles.
Cosine and sine also rotate arrows. Turn the arrows of length one, along x and y, by theta. The x arrow lands where the point did: cosine theta, sine theta. The y arrow, a quarter turn ahead, gets the x arrow's triangle, turned on its side. The legs swap, and the sine leg points the negative way, so the tip lands at minus sine theta, cosine theta.
The arrow ending at two, one, is a chain: two x arrows, then one y arrow. Rotate the chain; each step turns with it. Add the turned steps. Along x: two cosines, minus one sine. Along y: two sines, plus one cosine. With x and y in place of two and one, the same sums turn any arrow.
Take the arrow two, one, and the arrow one, two. Compare them with a dot product: multiply matching coordinates and add. Two times one, plus one times two: four. Rotate both by the same angle. Every coordinate changes; the angle between them doesn't, and the dot product stays four.
Turn one arrow by twenty degrees and the other by fifty. The angle between them opens by thirty; the dot product drops to about two. At fifty and eighty degrees, still thirty apart, it's exactly the same.
Language models with rotary position embeddings turn the arrows used to compare words by an angle proportional to each word's position. So in the comparison, their positions count only through the gap between them.