Firstly, be aware that there are three possible lavatory placings - at the front, midway in the plane, and at the back. The lavatory row at the back is 'special' as it's something people don't always realise - this row usually doesn't have reclining seats. Well worth noting. I regularly choose the back of the plane to get away from the kid-aisle with the bassinets further forward, but each flight has its own areas with their own merits. As a result of where I tend to choose, I am often near a bathroom.
However, it sounds like you're asking more about the mid-way lavatories, so I'll just describe lavatory seats in general.
Personally, I have anosmia (no sense of smell), so that side of things isn't a problem. Obviously the stronger your sense of smell, the worse it's going to get if there are issues with the bathroom.
Next, there's the inevitable queue. Although most planes have those handy signs to tell you when the toilet is in use, people still like to get up and queue - either because it's urgent for them, or even just to stand for a while and stretch those legs. What this means as a person close to the bathroom is that you constantly have people standing next to you, bumping you accidentally, watching your tv, chatting to each other right next to you, or potentially (horror!) even trying to make smalltalk with you! If you're trying to sleep, this can be very frustrating.
Nwxt there's the noise from the bathroom itself. You'll rarely hear actual ... 'functions' inside, but you will hear the flush, and some people have a tendency to slam the door open and make a racket.
It's also worth noting, as I alluded to before earlier, that midway bathrooms can sometimes be near the aisle with the bassinets and babies. This can be noisy and distracting as well.
So, long story short - if, like me, you don't even try to sleep on planes and have no sense of smell, I prefer the back of the plane (but not the very back row) and the aisle seat so that I can get up and wander a bit. If you like to sleep on a plane, don't watch movies, and have a stronger sense of smell, you may find it a very problematic and frustrating flight.