There are two parts to an itinerary: an entry in a reservation manager (a Global Distribution System or GDS such as Sabre or Galileo) which contains all reserved flights and is called a PNR (Passenger Name Record) or locator, and a ticket, which has a set of coupons, each good for travel from one place to another. Typically, there will be one coupon per flight (but there are cases where there are extra coupons, called open jaws, for which no flights are purchased). In the old days, all tickets were paper, but for many years most tickets are electronic. Each flight coupon is for purchased air travel from one airport to another, and in most cases lists the date, departure time, and flight code of the flight.
An open ticket is a ticket where some coupons only list the two airports flown from and to, they do not list a specific time, date, or flight code. On some airlines, open tickets must be paper tickets, not electronic tickets.
Open tickets used to be fairly common in the days when all tickets were paper, all fares were flexible and permitted changes, and more people traveled without fixed dates in mind. However, now that almost all tickets are electronic and most fares are highly restricted, they are rare.
I'll answer the three specific questions:
What exactly is an open ticket?
An open ticket is a ticket with at least one coupon that does not list a specific flight on a specific date.
How do you get an actual flight from one?
You have the airline or a travel agent make a reservation for the specific flight you want, and in the PNR list the ticket number you have. The corresponding coupon in the ticket is then validated by filling in the reserved flight.
What are typical restrictions?
Depending on the airline, it may need to be a paper ticket, which is a big restriction. You have to purchase the ticket using a fare that permits open tickets, which these days may also be a big issue.
Open tickets used to be used for multi-segment trips such as around-the-world fares, as well as simple round-trips on fully-flexible fares.
One case where they were especially useful is when a ticket is purchased but some intended flights are too far in the future to be loaded into the reservation system and hence are not bookable. People could get around this by leaving the last few flights specified only by from airport and to airport. Later, when the desired flights were bookable, they reserved them and used corresponding coupons of the existing ticket. Because of the limitations of open tickets, it's becoming more common for people who want to achieve the same thing to book their itinerary with flights on dates they have no intention of flying ("dummy dates"). Then, when the real dates are within the booking window (which varies but is commonly around 330-365 days), they change the flights from the dummy dates to the real dates. This avoids ever having an open ticket, but it also reserves seats on flights that aren't intended to be used, which airlines do not like.