Was I driving more than 5 mph under the speed limit, or less than 5 mph under the speed limit?

If this were programming, parentheses would help to disambiguate:

"I am driving (more than 5 mph) under the posted speed limit."
"I am driving more than (5 mph under the posted speed limit)."

Thus the same sentence can be parsed such that it means both what you're intending, and the opposite. Same goes for the second phrasing, with the same grouping logic.

That being said, English is not programming. Your first statement ("more than 5 mph under") sounds more intuitively appropriate for the meaning that you're trying to convey.

You could also try to avoid "more than" or "less than" entirely, in favor of a disambiguating word choice:

"I am driving slower than 5 mph under the posted speed limit."

You were driving more than 5 mph under the speed limit. To get there, you subtracted your speed from the limit, giving you 7 mph under. This is more than 5, so there you go.

While one of the two could be considered correct (I would say the first one), there is as you observe a potential for misunderstanding. You can remove that by slightly permuting the first phrase:

I was driving under the posted speed limit, by more than 5 mph.

This also has the advantage of first stating the main point (your speed was under the posted limit) and then quantifying that statement by an estimate of how much you remained below the limit, rather than starting with a quantification for a statement that has not yet been made.

I think the other answers have missed a key factor in how this would be interpreted. And that's that it's a lot more common to be talking about your speed in relation to the national speed limit than to be talking about your speed in relation to 5mph less than the speed limit.

In other words it is unusual to say that your speed was "less than (national limit - 5)", whereas it's fairly common to quantify how much less than the national speed limit you were.

So I think that

I was driving more than 5 mph under the posted speed limit

is very clear that you mean that's how much you were under the limit by. In fact it took me a little while to find your other meaning at all.

Edit: David Richerby makes another excellent point: if you were indeed saying that your speed was "less than (national limit - 5)" you would say "I was driving at less than (national limit - 5)". Not "I was driving less than (national limit - 5)". Further evidence for the ambiguity being negligable in this instance. Do read his answer.