She said Australia with a population of 22 million should not "hurtle down" towards a big population but opt for skilled migration.
Gillard, the country's first woman prime minister who ousted Rudd, took a different stand on one of his chief policy issues and said she supports a population that the nation's environment, infrastructure and services can sustain.
The nation should not "hurtle down the track towards a big population," she said according to an ABC report.
"I don't support the idea of a big Australia with arbitrary targets of, say, a 40 million-strong Australia or a 36 million-strong Australia. We need to stop, take a breath and develop policies for a sustainable Australia," she said.
According to Treasury's Intergenerational Report earlier this year Australian population was projected to rise from about 22 million to 35.9 million in 2050 if the current trends in overseas migration and fertility continued, with immigration by far the biggest contributor.
Gillard said: "If you spoke to the people of Western Sydney, for example, about a big Australia... they would laugh at you and ask you a very simple question: where will these 40 million people go?"
However, Gillard said it does not mean putting a stop on immigration all together. "I don't want business to be held back because they couldn't find the right workers... That's why skilled migration is so important.
"But also I don't want areas of Australia with 25 per cent youth unemployment because there are no jobs," she said.
Melbourne was predicted to hit a figure of 7 million people, and Sydney would grow to more than 7.5 million by 2050. The report prompted the country to rethink its migration policy over whether big cities, now straining under inadequate infrastructure, could cope with the growth.
Former Prime Minister Rudd, who was an advocate of a "big Australia" had appointed Tony Burke as Population Minister to develop a strategy.
Gillard, who immigrated as a child from Wales in 1966 when Australia's population was 11.5 million, said Burke's job description would now change to "send a very clear message about this new direction". He would now be known as the Minister for Sustainable Population.
Though Gillard stressed that her belief that population growth should be limited was "not about bringing down the shutters in immigration," any move to lower current rates would involve taking in significantly fewer immigrants.
Last year, overseas migration added almost 300,000 people majorly from the developing countries like Philippines, Malaysia, India, Indonesia and Vietnam.