Although the question is typically used metaphorically, evolutionary biology provides literal answers, made possible by the Darwinian principle that species evolve over time, and thus that chickens had ancestors that were not chickens, similar to a view expressed by the Greek philosopher Anaximander when addressing the paradox.
If the question refers to eggs in general, the egg came first. The first amniote egg—that is, a hard-shelled egg that could be laid on land, rather than remaining in water like the eggs of fish or amphibians—appeared around 312 million years ago. In contrast, chickens are domesticated descendants of red junglefowl and probably arose little more than eight thousand years ago, at most.
If the question refers to chicken eggs specifically, the answer is still the egg, but the explanation is more complicated. The process by which the chicken arose through the interbreeding and domestication of multiple species of wild jungle fowl is poorly understood, and the point at which this evolving organism became a chicken is a somewhat arbitrary distinction. Whatever criteria one chooses, an animal nearly identical to the modern chicken (i.e., a proto-chicken) laid a fertilized egg that had DNA identical to the modern chicken (due to mutations in the mother's ovum, the father's sperm, or the fertilised zygote)..
It has been suggested that the actions of a protein found in modern chicken eggs may make the answer different. In the uterus, chickens produce ovocleidin-17 (OC-17), which causes the formation of the thickened calcium carbonate shell around their eggs. Because OC-17 is expressed by the hen and not the egg, the bird in which the protein first arose, though having hatched from a non-reinforced egg, would then have laid the first egg having such a reinforced shell: the chicken would have preceded this first 'modern' chicken egg. However, the presence of OC-17 or a homolog in other species, such as turkeys, and finches suggests that such eggshell-reinforcing proteins are common to all birds, and thus long predate the first chickens.