I remixed an old openscad christmas tree design to include space for a coin cell and an LED. Coincidentally I have a bunch of green LEDs lying around — their light gets through the plastic a lot better than the white ones.
I made one with space for a regular 2032 coin cell, and one with space for a rechargeable 2450 (which doesn’t last nearly as long). You could remix again for AA or AAA, but then you’d need a real circuit instead of just sliding the cell into the space between the two leads of the LED.
(Caution: the space between the LED and the battery has to be longer than you think, or else the negative lead will bend so that it creates a short between the negative face of the coin cell and the rim of the positive face. Good thing coin cells can’t deliver much current.)
You’ll also need to put sticky tape on the back side of the coin cell to pull it out or use pliers with some kind of insulation, because it turns out those little so-and-so’s are slick and hard to pull out by hand. Or maybe I’ll fab some nonconductive tweezers…