Officially, only Mercosur nationals do not need to get their passports stamped at this border crossing if they are not traveling too far into the other country, while people from further away are required to do so. As a practical matter, though, (at least in non-covid times) checks are very loose and probably nothing bad will happen to you if you are consistent about not stopping to chat with immigration officials.
One way to get into trouble, would be to get inspected only in one direction. If you are recorded as entering Brazil without leaving it, or leaving Brazil without ever having entered it, this could cause problems for future visits.
My recommendation is to just follow the official rules and get stamped each time you cross the border (assuming you have the necessary visas or other permissions to cross); it's not hard to do and causes the least amount of risk. But, if you really don't want to, it'll probably be fine.
I don't know if enforcement has been made stricter—or if borders have been closed entirely—during the covid-19 pandemic or not; you would have to ask around locally for the latest information. In any event it is smarter for both you and the communities you propose to visit to delay your trip until the situation is safer.
(You don't ask, but, for completeness: the Argentine border guards at Puerto Iguazú actually do look at the passports of everyone who enters or leaves Argentina.)