Police said two bodies were recovered late on Thursday while two injured people were shifted to hospital. The work to clear the debris was still on.
The mishap occurred Thursday evening at Chimakurthi, near Ongole in Prakasam district.
Over 20 people were working 60 to 80 feet deep in the Hamsa granite quarry when the landslip occurred. Three bodies were completely buried in the mishap.
According to eyewitnesses huge boulders from the edge of the quarry fell on the workers. Authorities face a tough task in clearing the debris as heavy cranes could not reach the scene due to lack of proper roads. District officials said the work to clear the debris may continue for the next 24 hours.
Prakasam district Superintendent of Police Navin Chand told reporters that the exact toll would be known only after clearing the debris.
The police official said action would be taken against those responsible for the mishap after investigations.