Prosecutor Cites Imaginary Case After AI Search Error

A senior Crown prosecutor has cited a non-existent legal precedent in a high court case, adding to a growing list of AI-related errors impacting the justice system. Ravinder Dhillon, representing the Crown Prosecution Service (CPS), referenced a fabricated case during a dispute over whether a child rape case should be heard in the high court. This incident follows recent revelations about hundreds of AI applications secretly capable of generating abusive deepfakes and concerns over faulty AI facial recognition systems leading to false accusations of shoplifting.

These developments underscore a broader trend of AI technology producing unreliable or harmful outputs across various sectors. Earlier this week, it was reported that nearly 400 apps and websites, primarily linked to Chinese developers, were found to contain hidden features for creating violent and sexually abusive deepfakes. Separately, a significant portion of NHS trusts listed as benefiting from Palantir's AI data platform have admitted to using little or none of the technology, raising questions about the accuracy of public claims regarding AI's impact in healthcare.

In another instance of AI misuse, Brazilian lawyers were fined $16,000 for attempting to manipulate a court's AI system through prompt injection hidden within a legal filing. The court's AI flagged the hidden text, which demanded superficial legal scrutiny. These varied incidents highlight the urgent need for greater oversight and ethical considerations in the development and deployment of artificial intelligence, particularly in sensitive areas like law, healthcare, and content generation.