Canada normally won't care too much about your H-4 visa, though they might care about it if they think you'll have trouble getting back to the US when you leave Canada. Your question suggests there may be some confusion about whether you have a valid H-4 visa or H-4 status. This answer assumes that your H-4 visa is expired, and that you have valid H-4 status until September 2020, which should be reflected on your I-94.
If you go to Canada for 30 days or less, you can be readmitted to the US without a valid H-4 visa under automatic visa revalidation. The US government has pages describing this at the websites of both the state department and CBP. There are some restrictions, including that you may not have a pending visa application.
If your trip to Canada is longer than 30 days, you will need a new visa in your passport. You should be able to get that while you're in Canada. I don't know whether Canadian border officers are going to like that, but as far as I understand it, traveling to Canada for the purpose of visiting a US consulate to get a new visa is fairly common, so you should presumably have a fairly good chance of being admitted.
Because one of the conditions of automatic visa revalidation is that you not have a pending visa application, if you go to Canada and apply for a new H-4 visa, you will not be able to return to the US until it is granted and placed in your passport.