In order to check in, do I have to claim my baggage first then find the operating airlines to check in, or, is it something like notifying them that I am there to claim my seat?
You should find out when you check in for your first leg. Usually you will be checked in for any unconfirmed legs at that point, though you may get a boarding pass that still shows an unconfirmed seat. This has happened to me a few times in the past.
The bag will normally be checked to the final destination, and you'll normally get a seat assignment at the gate or at transfer desk in the airport from which the unconfirmed leg departs. It's possible, however, that your bag will be checked only to that airport, in which case you may have to retrieve it. The agent who checks you in at your point of origin should clarify exactly what you need to do.
Do I need transit visa, layover is around hours, to change terminal (3 to 5) at Manchester. MAN to ORD is operated by American Airlines.
Whether you need a visa for the UK, and if so what type of visa you need, depends on your nationality (which you have not disclosed) and on certain other facts. You can learn about the requirements elsewhere on this site at Is there a way to find out if I need a transit visa for a layover in the UK?
If you need to retrieve your bags, you will need to go through passport control. Note that most people who would normally require a visa may be able to qualify for the "transit without visa" concession if (among other conditions) they have a visa for the US. Another condition that must be met is that your departure must be either on the same day as your arrival or before midnight on the following day.