This is the temporary shack on the Cambodian side. You can see the elaborate roof of the bigger newer building hiding behind the welcome sign. This was January 2015. The new building may be open by now.
The Lao side at the time looked more substantial. I don't recall an ATM or a bank. Maybe but I kinda don't think there was.
I would get US dollars before heading to the border. And I would get extra. I had $100 or more that I exchanged over a month earlier when I found a decent exchange house in Thailand. I basically carry USD just for visa-on-arrival fees.
I got at least two each Laos and Cambodia visas on that trip and for me as an Aussie citizen they were $30 USD each. I do recall at some crossings there was a $1 or $2 extra fee. They were less common than I expected from reading traveller reports on the Internet etc. In each case I decided the fee looked official enough and was probably not a bribe in my opinion.
Now I was travelling with two Norwegian friends. Norwegians are usually even more welcome in even more places than Aussies. But in my Googling I had read that their visa fee may be $5 more than mine. I was handling that bit of the money for all three of us. I made sure that I was going to have at least $30 for me and $35 for each of them plus $2 extra each minimum. It turned out to be wise because Norwegians did in fact have to pay more than Aussies.
The moral of that story is that when you're changing money for visas fees it's better to err in favour of the maximum amount you think it could possibly be.
I believe there were banks in Stung Treng yet oddly I have no recollection of how I got money there. I stayed for about a week which makes me think I must've got money from an ATM.
Between Stung Treng and the border there is very little. Rough bush. A police compound. Probably some huts here and there with some selling a few supplies. Certainly nothing like a bank or ATM until Stung Treng itself.
In Don Det, the main backpacker island at Si Phan Don there were many businesses with signs saying "ATM" but they didn't actually have an ATM. What they did was charge you a fee do do a cash advance on a Visa or Mastercard. Be aware that you will pay higher bank fees and/or interest on cash advances. The same places also changed money so you would be able to get your US dollars there.
But there are 4,000 islands there. I wonder if the biggest island that tourists don't usually go to because it's not special might have banks with ATMs. A lot more people live there. I think it's called Don Kong. But I also recall that two islands had very similar names so be aware of that.
I seem to recall walking from where I got off the boat from Si Phan Don to the border. Too short to be any banks or ATMs as I recall it.
I'm sorry I don't have many of the precise answers. I hope somebody provides another answer that does. But the tips here should help you at least.
It turns out I wrote a Facebook update at the time and it does contain some extra details.
I did have to pay an extra $2 USD fee to depart Laos. There was no extra fee to enter Cambodia, just the $30 USD I expected for an Australian citizen.
There was one town along the road to Stung Treng. It had one guesthouse that was double the price I was expecting in Stung Treng so I kept going even though it was dark. I doubt that town had a bank or ATM but I'll see what I can find...
The town between the border and Stung Treng is so small I can't find it on Google Maps.
I can find at least two ATMs in Stung Treng on Google Maps.
Ban Nakasang has an ATM, according to my Googling, plus two other banks. This is the village you would normally get of the boat back from Don Det.
That's a lot handier than Don Khong, which doesn't seem to have an ATM I can find.