I am currently in the US as a visiting grad student on a J-1 visa. My advisor wants to keep working with me, but for various personal reasons I need to go back to my home country after my time here is done. The professor I am working with is totally fine with me working remotely, but the university requires me to hold a J-1 visiting scholar status in order to be employed. So my question is if I could hold a J-1 visa while not physically being present in the US for the time of the visa and also not have my main point of residency in the US?
You can certainly hold a J-1 visa without using it to travel to the US. You can also use it to travel to the US and then leave the US, which is probably what you'll have to do.
The school probably requires you to have a visa because they are probably not set up to pay people who work as employees but reside outside the US -- and I don't mean only in terms of making the payments, but more importantly in terms of taxation and labor law. So if their lawyers catch wind of it, there might be trouble.
Of course, in order to set you up as an employee in the US, they need to document that you have authorization to work in the US (using the I-9 form), which is why they say you need a visa. To complete the form, you'll actually need an I-94 form, which is only issued when you enter the US. After you've submitted the I-9 form, though, you can leave the US again.