I would recommend Kumano in southern Mie Prefecture, located about four hours south of Nagoya (Location in Google Maps). I spent five years living very close to here and, based on familiarity with the area and its offerings, can recommend it to you!
It satisfies all of your criteria:
We love rocky beaches, nature hikes, history, the smell of ocean air, the scenery of water, and we want to experience Japan in a smaller town way
Kumano is located at the bottom of Japan's Kii Peninsula and is one of the most isolated areas of the country, a region referred to as "Nanki". It has a famous historical hiking path, called the "Kumano Kodo", which winds through the mountains. It is rather famous in Japan for the smooth black rocks that makes up its beaches, which stretch from the city southward.
I want to avoid bears and lethal animals.
Ironically, the meaning of the name "Kumano" in Chinese characters (熊野) literally translates to "Bear Field", but don't let that fool you it is not a bear-heavy area. All hiking trails in the mountains in Japan will offer the possibility of some encounters with wildlife, however lethal animals attacks on mountain hikers in this region are exceedingly rare.
Due to possible radiation, I'd like to shun Fukushima and its surroundings.
Kumano is approximately 800km away from the reactor in Fukushima prefecture so no worries
I won't drive, so must rely on public transport and walking (over reasonable distances).
You can get to Kumano by express train. From the main train station you'll be using (Kumanoshi Station), you can get to both the ocean and mountain hiking trails on foot. The main area of the city is also very walkable. Public transportation for getting to nearby areas (such as Atashika Beach) is available. However it runs rather infrequently, so you will probably find it easiest to stay in Kumano and its walkable surroundings.
I want to evade temples and all buildings related to religion.
Though the Kumano Kodo hiking trail, like many hiking trails in Japan, was originally laid out centuries ago as a pilgrimage route between a major template and a major shrine, the trail itself is not religious in the present day. Though there are some forks in the trail that will take you to temples, you can easily avoid them as they are far away from the portion of the trail in Kumano. I should mention that though you can easily avoid visiting temples and shrines while visiting Japan, there is basically no area of the country you can visit where there will be no existing temples or shrines present or nearby.
No crowds please.
As long as you don't visit on Aug 17 (the day of the Fireworks Festival), you'll have no trouble with this point. It is one of the least-populated and least-visited areas of Japan's main island, but has the advantage of being accessible from a major station (Nagoya).
Further reading and info in English about Kumano and the hiking trails there: