Are Alexa-controlled night lights worth it if I don't already own an Echo?

Generally no. The Echo Glow specifically requires a compatible Alexa device for voice control — pcmag.com points out it has no built-in microphone or speaker. If you don't already have an Echo in the household, a self-contained smart light like the Olight Sphere (app-controlled) or the sensor-based Casper Glow is a better fit.