Is there any inexpensive, decent Jewish neighborhood in Jerusalem? I've learned that the east side tends to be Arabic and the west Jewish. We are trying to stay in a Jewish neighborhood to learn the language and culture.
Any Jewish neighborhood is decent, and if you're non-Jewish then most of the Arabic neighborhoods are decent as well (for Jews it may be hazardous, especially if you're there on ideological grounds).
You're right about the West/East division (originated in the split of the city between Jordan and Israel between 1948 and 1967, Jordanians banished Jews from the city in 1948, so the Eastern part was "ethnically cleansed"). However nowadays there are also Jewish neighborhoods in the Eastern parts (Pisgat Zeev, Neve Yakov, Maale Adumim, just to name a few).
If you're not looking for a fancy/expensive place, then you shouldn't be looking at the neighborhoods in the city center (Rehavia, Mamilla, Moshava Germanit etc), most of the neighborhhods on the outskirts are relatively inexpensive (Ramot, Gilo, Kiryat Yovel, etc). Note the "relatively". Israel is pretty expensive in general, and lodging in the capital will cost you.
Note that many neighborhoods are "religious" (Jerusalem in general is very conservative, on both sides), and in some you won't be able to drive on Saturdays and holidays. Public transportation is off on Saturdays (which includes Friday afternoon) in the city.
Depends on what you mean by inexpensive? If you're looking for guest houses and rental apartments compared to hotels then there are ranges and neighborhoods vary - You can get cheaper and more expensive in the same neighborhood: Maalot Dafna as example of rentals. Guest houses are closer to the City Center and Ben Yehuda / King George area and Hotels - Prima Palace is a cheaper Hotel, compared to Ramada or King David.
I would set yourself a budget and work out which area you want to be close to and then work out from there areas you would like to visit and pick a place that's central to that.