Standards Michigan Group
2723 South State Street | Suite 150
Ann Arbor, MI | 48104
888-748-3670
To: Ms. Rachel Trello, National Institute of Standards and Technology
RE: NIST 2023-0309: Bias in Training Data. Recommend metrics.
February 2, 2024
Dear Ms.Trello:
Thank you for the opportunity to comment. Standards Michigan has been involved in standards of all types – ad hoc, consortia, open, voluntary, de facto – for over 30 years. See our ABOUT.
Our resources limit our recommendations to only one–a peak concern– which originates from our work in academic literature. Because so much of the consulting information (training data) comes from the written word–and much of that in academic precincts–we expect that algorithmic bias we already see in AI output will accelerate. A great deal of intelligence happens with action, not words.
Perhaps NIST will need to develop new “units of measurement” to rise to its challenge. AI metrics could be a simple number, for example, or a vector space reflecting the bias to the user– where appropriate –if the bias (or one-sidedness) cannot be eliminated.
A reference to some relatively new numerical indices will enlighten my suggestion. In my profession, for example, we have reliability indices (See: IEEE 1366 Guide for Electrical Power Distribution Reliability Indices). These have proven helpful in the fullness of time.
AI will have many applications in its journey along the classic “Rogers Curve” Metrics could be formulated to indicate the sources, even if it means slowing down the output in non-critical queries Much of the success in the American experiment in democracy is owed to constructive debate on all sides of an issue.
This is a worthy project and I wish you every conceivable success..
Very truly yours.
Michael A Anthony, P.E.
“Language is the only homeland.” — Czesław Miłosz (Nobel Laureate, 1980)
www.standardsmichigan.com
