Ulpanim are usually sponsored by the government or the Jewish Agency. They're free to olim because someone else pays.
Most of the ulpanim require signing up through the immigration office which you won't be able to do. There are some open to the general public, however, where you can pay on your own.
Quick Google search shows some private options (example, not affiliated), and universities also offer it to foreign students (example 1, example 2[he]).
As to visa type - inquire with the consulate. For A/2 I believe you have to be enrolled into accredited institution, I don't know if ulpan counts.