Is offering to work for free a bad idea for an undergrad researcher? I’ve read that PhDs are almost always funded, and being accepted into a PhD program without funding is equivalent to a polite rejection.

Is this also true for undergraduate research? For example, many universities have a fund for undergrads to do one or two semesters of paid research, but students must look for other funding sources after that.

Is it wise for undergrads to offer to do research work for free, or does it devalue the undergrad’s work in the eyes of a PI?

This question assumes the setting as an R1 university in the United States, for a student in a STEM field like computer science, mathematics, or electrical engineering.
If you are a student who is getting something of (educational) value out of your research, and don't need the money, it's not necessarily a bad idea to do undergrad research without getting paid for it. (If you weren't a student, this would probably be a very bad idea, though.)

Personally, however, I will only supervise students who are either getting paid or getting some kind of independent study credit. (I'm not the only supervisor I know with this policy.) I don't want to put time and effort into supervising a student who is only very loosely committed to the research project.

I encourage you to consider that option - independent study or thesis credit - for undergraduate research.