American English retroflex 'd' in the word 'guardian'?

I believe the sound you're hearing there is not a retroflex plosive but a retroflex tap or flap, transcribed as [ɽ]. The /d/ in guardian satisfies the American English flapping rules (see Wiki), since the stress is on the first syllable of that word. I doubt that many American speakers have an actual plosive there; I certainly don't. Instead, it's the same sound as the voiced /t/ in party. The end of the first syllable of guardian may sound different from that of party, since the latter undergoes prefortis clipping, but the consonant sound there is certainly the same (at least when I say them).

If so, you're absolutely correct that an [ɽ] can occur there, in place of either /d/ or /t/. A Course in Phonetics (Ladefodeg & Johnson 2015, 7th ed.) say as much, though their terminology obscures things a bit. They distinguish between taps, which are never retroflex, and flaps, which generally are. They then state that in American English, a (retroflex) flap can be used instead of a (non-retroflex) tap when the /t/ or /d/ is preceded by /ɹ/.

Here's the complete excerpt, from p. 186-187, with the most relevant sentence emboldened:

It is useful to distinguish between taps and flaps. In a tap, the tip of the tongue simply moves up to contact the roof of the mouth in the dental or alveolar region, and then moves back to the floor of the mouth along the same path. In a flap, the tip of the tongue is first curled up and back in a retroflex gesture, and then strikes the roof of the mouth in the post-alveolar region as it returns to its position behind the lower front teeth. The distinction between taps and flaps is thus to some extent bound up with what might be called a distinction in place of articulation. Flaps are typically retroflex articulations, but it is possible to make the articulatory gesture required for a flap at other places of articulation. The tongue can be pulled back and then, as it is flapped forward, made to strike the alveolar ridge or even the teeth, making alveolar or dental flaps. Flaps are distinguished from taps by the direction of the movement—from back to front for flaps, up and down for taps—rather than by the exact point of contact.

Some forms of American English have both taps and flaps. Taps occur as the regular pronunciation of /t, d, n/ in words such as latter, ladder, tanner. The flap occurs in words that have an r-colored vowel in the stressed syllable. In dirty and sorting, speakers who have the tongue bunched or retracted for the r-colored vowel will produce a flap as they move the tongue forward for the non-r-colored vowel.

As far as transcription goes, they say later on the same page that:

An alveolar tap may be symbolized by the special symbol [ɾ], and the post-alveolar (retroflex) flap by [ɽ].

They refer to this again on p. 188:

You should also be able to produce a retroflex flap. As we have seen, many speakers of American English use this type of articulation in sequences such as herding /hɚɽɪŋ/, in which the tongue is curled up and back after the r-colored vowel, and then strikes the back part of the alveolar ridge as it moves down during the consonant.