So. I loved reading the Bible as a child (I started at 7). I was a bookworm, and the Bible was just another book.
I am not religious at all
If it's just reading, I think it's great he's getting a bit of culture. He should read other religious texts from other religions as well, to compare all the wonderful and nonsensical things they say
If Grandma is trying to convince him that all that stuff is real and he should believe, then that's a breach of trust and I wouldn't leave them alone very frequently. I would also talk about religions with my kid. There are books for children that compare religions - you can get him one of those and start a Humanist discussion.