Every GIF carries a colour palette — a table of up to 256 colours that its frames are drawn from. Because a photo contains thousands of colours, converting it to GIF means picking the best 256 (a step called palette generation) and mapping everything to them.
Reducing the palette further — say to 128 or 64 colours — makes the file much smaller but can cause banding in gradients. Good GIF tools generate an optimised palette per clip so the result looks as close to the original as possible.
Shrink a GIF by lowering its palette with Compress GIF.