Published in 1937, this is a very short story resonating with urban legend that the pyramids were designed by aliens if not actually constructed by them. In this story the alien pyramids are the Central American ones rather than Egyptian, which were being rediscovered and excavated in this time frame. Miller's character attributes these theories to someone named Kobal, but they are quite similar to those of Immanuel Velikovsky who published the popular Worlds in Collision in 1950.
The story has a lot of consistency problems: One character is overly hasty; the other is overly coy. The pyramid is shining white in a remote, obscure valley, not jungle covered. And curiously, no mention of Mayan calander end of world. I guess that's real new urban legend.
The end is the theme of the Star Trek IV: The Voyage Home movie with alien spacefaring whales: Don't mess with this, the aliens may come back real soon now and they'll be really pissed if you do.