Is it possible to travel the country (by public transportation) as
tourist (including Stockholm, Gothenburg) WITHOUT a credit and debit
card (and no app as well)
In Stockholm it is possible. You will need to purchase either a single ticket, an SL card containing pay as you go credit, or an SL card containing a period ticket (24 hours, 72 hours, 3 days 7 days etc). You can buy these from train & metro stations and some retailers like Pressbyrån or 7-eleven. Buy the card with cash, then use it on public transport.
You cannot "buy a ticket" on a bus/ferry - using any payment method - you must have a single ticket already, use the SL card, or use the app.
More information from SL: https://sl.se/en/fares--tickets/
Gothenburg has a similar system: https://www.vasttrafik.se/en/tickets/
For other regions, you'll need to do some research.
Booking long distance trains will be more difficult. You will need to buy tickets from an SJ ticket office and they will charge a booking fee for this.
As others have mentioned, buying a prepaid debit card using cash will make life a lot easier.