Should i pick my rats up or leave them?

I got my rats exactly a week ago and I gave them a day to settle in until I decided to try to pick them up. I was a bit hesitant at first and still am, sometimes flinching and pulling my hand out before picking them up and i think this may have scared them. however i have managed to pick them up a few times and my family and friends have picked them up once or twice aswell. my problem is when i do pick them up they try to run away and dont seem to enjoy it very much, they also run into their little igloo whenever i sit down to watch them! i dont feel like im making any progress! should i keep picking them up or let them be?? i want them to like me and be comfortable around me but i dont want to stress them out and scare them even more!!? they dont seem to like me at all and try to get away from me all the time! please help :(