The Jews has some freedom in their building plans.
But by 539 b.c. the Jews (under the rule of Cyrus of Persia) were allowed to return from Babylonia and to build a modest Second Temple in about 515 b.c.