Nearly a thousand years ago, the Norman Conquest had a profound effect not only on the English nation but also on the English language. One of the manifestations of this event is the survival of the postpositive adjective.
In many languages, including French, a modifying word follows the word it modifies, such as in the phrase ressource naturelle (“natural resources”). Because of Norman French’s influence on law, politics, and other matters sovereign, we still sometimes use this form in the mongrel melange that is the English language.
Thus “attorney general” (as well as “secretary general” and “postmaster general”), which refers not to a military rank but to the office holder’s generic scope of responsibility. Thus court-martial, which literally pertains to a court of a martial, or warlike, nature but practically applies to a military court in wartime or peacetime. Thus “heir apparent” and knight-errant, artifacts of feudal system. (Note that compound form is inconsistent: Open compounds prevail, but some hyphenated forms persist. When in doubt, look the term up. If certain, look the term up anyway.)
This form reaches even into the quotidian vocabulary of business, with “accounts payable” and “accounts receivable,” as well as “notary public,” and in terms that apply to government but have entered general use, such as “body politic.” There’s even a pair of ordinary words that sometimes take postpositive adjectives in some contexts; I used one earlier in this post, in the phrase “matters sovereign.” Another is things, as in “things unsaid.”
And how are such terms pluralized? Generally — as shown in the first two examples in the paragraph above — the noun, not the adjective, logically takes the plural form: for example, “attorneys general” (but attorney-generals in British English), courts-martial, and “notaries public.”
The same is true of mother-in-law and like terms, the plural form of which is rendered mothers-in-law, and similar constructions such as “editor in chief” (sometimes hyphenated, though that style is outmoded), right-of-way, and sergeant-at-arms (pluralized as “editors in chief,” rights-of-way, and sergeants-at-arms, respectively).