Each country has 3 mobile operators and all of them offer mobile internet (though not all offer 4G). They also all operate on the same frequency meaning you can use the same phone everywhere (make sure your phone is operator unlocked though !)
According to their respective Wikipedia pages, coverage seems to be quite good: 98% of populated territory (2010) for Georgia, 95% (2014) for Azerbaijan and 90% (2012) for Armenia (that last number needs to be verified)
Beeline (3G only) covers both Armenia and Georgia and they have offices in both countries, and according to their website there doesn't seem to be extra charges within their network, meaning you can buy one sim card for both countries.
The "local" prepaid plan is 1GB valid for a month, and their roaming ones can be daily, weekly or longer, in chunks of 500MB.
GeoCell (3G/4G) is another Georgian operator that has data plans ranging from 500MB to 15GB (they also have roaming agreements with Armenia and Azerbaijan)
VivaCell MTS (3G/4G) is an Armenian operator, they have prepaid 3G plans that charge per MB of usage (7 to 15 AMD/MB)
AzerCell (3G/4G) have their SimSim plan which seems to be the prepaid option with different options (0.2 to 0.4 AZN/MB)
BakCell (3G/4G) is another Azeri operator that offers prepaid internet. They have packages ranging from 40-600MB between 1-4AZN, valid for a month.
All of the above operators seem to have roaming agreements with their neighbours and beyond, so you shouldn't have much trouble using the same sim in more than one place.