I usually go a little closer to Albuquerque so spend more time up in the Jemez Mountains like the Jemez Springs area up to around Cuba and over to Los Alamos exploring Forest Service roads west of Santa Fe, but also some good places up in the Taos and Eagle Nest area as well.
For the route your are taking, I agree that Chama is a great location. Try the Los Brazos area near Chama too as you can camp nearby, or stay at Corkins Lodge or Los Brazos cabins to do some fishing and hiking. If you can stay for a couple of days, you can take a ride on the Cumbres and Toltec Scenic Railroad into southern CO, or just drive into the southern CO area like over to Conejos to the northeast or towards Pagosa Springs to the northwest if you have time to kill. But you can also get some fly fishing in just staying along the Chama River too if you stay below the confluence with the Rio Brazos River.
Here are some shots I took a couple years ago when I was up around Chama. Sorry for the shot of one of my old F150s, but you get the idea...lol:
Looking upstream along the Los Brazos:
Chama area:
Up towards Conejos: