No you can't turn up late to check-in on a delayed flight, unless the check-in closing time is delayed along with the departure time. Note that usually this is not the case.
The check-in closing time is set relative to the scheduled departure, and usually remains the same, regardless of whether the flight is delayed or not. Hence if you show up at the airport late for check-in, even though you still have plenty of time before boarding due to the flight being delayed, you will be considered as a no-show. This happened to me a few years back on a snowy day in which all inbound and outbound flights to/from LHR were delayed.
Several websites and forums mention this (see here and here). However I believe this webpage from Air Berlin to be more authoritative:
We are unable to check in passengers arriving at the check-in desk after the check-in deadline. These passengers lose their entitlement to be carried on the flight and do not have any right of reimbursement, issue of a credit note or free rebooking. This check-in deadline will remain the same even if the departure of the flight is delayed.
Other airlines, including Emirates and British Airways, carry no mention of whether the check-in deadline can be modified, probably because they assume that when flights are delayed the check-in closing time does not vary. However they do mention that arriving late for check-in will cause you to be denied boarding in their Terms of Carriage. Emirates (in PDF) says:
6.3 You must comply with the applicable Check-in Deadline. We have the right to cancel your
reservation if you do not comply with the Check-in Deadline or, if no Check-in Deadline has been
indicated, you fail to check-in prior to closing of check-in for your flight. See also Article 5.7.
British Airways says:
6c) You must check in by the check-in deadline
If you do not complete the check-in process by the check-in deadline, we may decide to cancel your reservation and not carry you. By completing the check-in process we mean that you have received your boarding pass for your flight.