Avoiding discrimination in the use of AI: new Office for Civil Rights guidance

 Just released this week from the U.S. Department of Education's Office for Civil Rights is new guidance on avoiding discrimination in the use of AI. The document offers 22 scenarios in which an investigation would result. Reading them, they sound a lot like what is not being thought enough about in schools.
Here, for example, is the third one: 

 Teachers at a middle school often rely on non-contracted, online third-party applications and websites to translate and interpret for parents who have LEP. Some of these applications leverage AI to translate or interpret. A parent complains to the school’s principal that they have been unable to communicate with their child’s teacher regarding the child’s academic and behavioral progress in class due to incoherent translations. The school does not investigate or attempt to resolve the parent’s concern. OCR would likely have reason to open an investigation based on this complaint. Based on the facts, as alleged, the school may not be ensuring meaningful communication with parents who have LEP in a language the parents can understand.  

Or how about this one: 

 A school district allows schools in the district to use a generative AI tool to write Section 504 Plans for students with disabilities. The school district does not have any policies regarding how to use the tool or how to ensure that the group of knowledgeable people responsible for evaluating a student review what the AI produces to determine whether it meets the individual needs of each student. One school begins using the tool to create Section 504 Plans for all students with diabetes. School staff do not review or modify the generated Section 504 Plans and begin implementing them, and they inform parents that they believe AI tools make more effective choices than people. A local group of parents of students with diabetes at that school files a complaint with the school district stating that their students’ Section 504 Plans’ provisions look almost identical and, in some cases, do not match the specific needs of their children. The school district states that they defer to the school’s decision on how to utilize AI tools and does not investigate further. OCR would have reason to open this complaint for investigation. Based on the facts, as alleged, students may not have been provided with FAPE because their 504 plans may not have been designed to meet their individual educational needs.  

You can read K-12 Dive on this here.

As I have said, we are not thinking enough about what we are doing here.