A person who studies rocks is called a geologist.
In general a geologist and in particular a petrologist. Petro means rock(s) and petrographer is a geologist who specialises the study of rocks in hand specimen and under the microscope.
Geologist or there is another word we sometimes use as someone who studies minerals and rocks and the earth. Petrography.
Petrologist