You'll want to look around what's known as the Old Quarter of Hanoi. You'll find hotels and hostels of various levels of quality throughout the area. However, Ma May and the surrounding streets are probably a good place to start for backpacker hostels (and bars), as is the little maze of streets just to the north of St Joseph's Cathedral. The latter area is probably a bit quieter.
The Old Quarter is roughly the area in the black line on the map. Ma May is on the right in the middle, the cathedral is at the bottom. The airport is 30km or so to the north, and the main station just off the bottom left of the map.
It's all walkable, so long as you mind the scooters, but bear in mind that Hanoi can get pretty hot and sweaty for most of the year. You might find it best therefore to get your walking done early in the day.
Map © OpenStreetMap contributors