Is there any inexpensive, decent Jewish neighborhood in Jerusalem? I've learned that the east side tends to be Arabic and the west Jewish. We are trying to stay in a Jewish neighborhood to learn the language and culture.
Any Jewish neighborhood is decent, and if you're non-Jewish then most of the Arabic neighborhoods are decent as well (for Jews it may be hazardous, especially if you're there on ideological grounds).
You're right about the West/East division (originated in the split of the city between Jordan and Israel between 1948 and 1967, Jordanians banished Jews from the city in 1948, so the Eastern part was "ethnically cleansed"). However nowadays there are also Jewish neighborhoods in the Eastern parts (Pisgat Zeev, Neve Yakov, Maale Adumim, just to name a few).
If you're not looking for a fancy/expensive place, then you shouldn't be looking at the neighborhoods in the city center (Rehavia, Mamilla, Moshava Germanit etc), most of the neighborhhods on the outskirts are relatively inexpensive (Ramot, Gilo, Kiryat Yovel, etc). Note the "relatively". Israel is pretty expensive in general, and lodging in the capital will cost you.
Note that many neighborhoods are "religious" (Jerusalem in general is very conservative, on both sides), and in some you won't be able to drive on Saturdays and holidays. Public transportation is off on Saturdays (which includes Friday afternoon) in the city.