There's more infrastructure under urban roads. Crews come in to fix some utility, shred a section of a lane, patch it poorly with dissimilar materials, and leave.
Though given his definition of quality I expect he is actually ignoring all the real rural roads and only talking about major roads which while they get less traffic than urban roads are maintained to similar standards.
Like the other replies have indicated, I'm not so sure this is the case? I live in very rural northwest Iowa, and while there are certainly plenty of gravel roads around here, I'm only driving on them if I'm intentionally trying to go "off the beaten path." You'll take a gravel road if you live on a farm, or you're trying to get to somewhere secluded such as a lake, campground or maybe a county park; but (imo) it's rare for the average person to drive down a gravel road just going from Point A to Point B on their daily commute.
"Santa Fe has a higher percentage of dirt roads than any other state capital in the nation. Unless they are well graded and graveled, avoid these unpaved roads when they are wet. The soil contains a lot of caliche, or clay, which gets very slick when mixed with water. During winter storms roads may be shut down entirely." - https://archive.nytimes.com/www.nytimes.com/fodors/top/featu...
With Google Maps, the dirt road closest to the center of town that I found is Del Norte Lane, at about 1/2 mile, with more dirt roads just north of it.
Santa Fe also has a lot of multi-million dollar homes on dirt roads.
Santa Fe is a special place, and not indicative of "average".
[1] "He and his surveyor laid out the town, including the districts, house and garden plots and the Santa Fe Plaza for the government buildings. These included the governor's headquarters, government offices, a jail, arsenal and a chapel." - https://en.wikipedia.org/wiki/Pedro_de_Peralta
[2] "In 1513 the monarchs wrote out a set of guidelines that ordained the conduct of Spaniards in the New World as well as that of the Indians that they found there." https://en.wikipedia.org/wiki/Spanish_Colonial_architecture#...
[3] "These structures had been laid out around a street grid and series of plazas—a practice that had become a standard for new Spanish settlements in the Americas and Asia—yet their irregular rather than orthogonal alignment seemed disorderly to Domínguez [in 1776]. ... The employment of the grid in town layouts remained in use even when New Mexico became part of Mexico and the United States, only to be replaced by the cul-de-sac and other American suburban models of development since the 1950s." - https://sah-archipedia.org/essays/PF-01-ART004
[4] "The Tanoans and other Pueblo peoples settled along the Santa Fe River from the mid 11th to mid 12th centuries,[20] but had abandoned the site for at least 200 years by the time Spanish arrived in the early 17th century." - https://en.wikipedia.org/wiki/Santa_Fe,_New_Mexico