Q:

Creepy YA short story about children living in walls to hide from a monstrous man

This story scared me so badly when I was a little kid that my father threw the book away! I would have read it in English in the US prior to 1989, and it was almost certainly a Scholastic Book Club book.  I would most likely have been in fourth or fifth grade at the time, so that might help narrow down which Scholastic flyer it had been in.
If it was a book, it was a very short book, but I think it's more likely that it was a short story in a compilation of creepy stories. I think the cover may have been blue.
The main thing that I recall about the story is the house.  This house was huge and had twisting corridors and stairs everywhere - for some reason children had been transported to the house and had to hide inside the walls.  Some of the children had been hiding in the house for years, possibly longer.
I think that it was a neighbourhood 'haunted house', and that's how the main kid in the story got trapped there - he decided to explore the house, and somehow got trapped there, stuck out of time or in a pocket dimension or whatever. 
The windows of the house all showed views of different seasons - one window might show winter with snow on the ground, the next window would look out to spring and green grass, etc.  I have the impression that the house was some sort of dimensional nexus, with the windows opening to different alternate universes or even different eras in time. 
The children had to hide in the walls because a monstrous man was roaming the house, looking for the kids.  I remember imagining that he looked like a gaunt Frankenstein's monster (he's what I was so scared of & the reason my dad tossed the book out!), but I am not at all certain that that's how he was described in the book - that's just how I imagined him looking.  Tall, ominous, he never spoke as far as I can recall.  He would snatch the children up if he could catch them, and take them away to an unknown but horrible fate.

A:

"The House on Pearl Street" from "The Haunted Planet" by D.J. Arneson and Tony Tallarico 

Taking a shortcut through the woods, a boy comes across an old house. He goes in to find an ugly giant holding a group of ghostly children from different era’s of time captive inside the house. The children tell the boy he will be one of them too, and that there is no escape.

Horror Delve, The Haunted Planet Review
This review mentions different seasons windows and children in hidden panel in the wall.

He looks out the window and it is winter out. Only it was SUMMER when he came in! Then a super-tall ghoul-thing (see above) shows up and shambles after him. These children pull him into a hidden panel in the wall and tell him they are ghosts who live there because the ghoul-thing killed them, too, and he’ll be joining them soon, because there’s no escape. They kick him out of the safe room (I assume because they want to PLAY with him, Danny, and in order for them to do that, he has to be eaten by the ghoul-thing) and he runs around (outside of another window, it’s fall, this is the house of all the seasons, although it’s not really explained why that’s the case)

