In the new era of digital warfare, artificial intelligence has become a rising star on the battlefield. The U.S. Department of Defense recently announced that it will collaborate with startup Scale AI to test the potential of generative AI models for military applications.
According to Hong Kong-based IDC, the San Francisco-based company will build a set of tools and dataset frameworks tailored to the Pentagon to help them evaluate and refine the capabilities of large language models. The framework will include measuring model performance, providing real-time feedback to fighters on the battlefield, and creating a dedicated set of public sector assessments to test how well AI models perform in military support applications, such as collating findings from operational reports.
The ability of large language models to analyze and generate text is expected to improve the Pentagon's ability to gather intelligence and plan combat operations, thereby helping to make decisions on the battlefield.
Imagine a battle commander being able to get all the strategic information they need in real time. "Imagine those battle commanders no longer need to access information via PowerPoint or e-mail within the organization – situational response time has been reduced from one or two days to ten minutes. "Defense Department Chief Digital and Artificial Intelligence Craig Martell said this at the Defense Data and AI Symposium.
However, while AI can quickly process large amounts of information, it also has security risks. Military data is often extremely sensitive, and there are concerns that once this information gets into large language models, it could lead to data breaches due to just-in-time injection attacks or API abuse.
The biggest obstacle is that these models tend to produce inaccurate or false information, a phenomenon known as "hallucinations." With the introduction of Scale AI, the Pentagon hopes to test the performance of different models to identify potential risks, before considering using them to support combat or intelligence gathering.
The startup will reportedly compile a "retention dataset" with examples of effective responses that would be useful for military use. The Department of Defense can then compare the responses of different models to the same prompt and assess their usefulness.
Last year, the Department of Defense established Task Force Lima — a team led by Martell, who previously led machine learning at rideshare company Lyft, to work on military applications of generative AI.
"It is incumbent upon the Department of Defense to responsibly adopt generative AI models while identifying appropriate safeguards and mitigating risks that may arise from issues such as poor management of training data," Martell explained at the time. We must also take into account how our adversaries will take advantage of this technology and try to disrupt our own AI solutions. ”
Still, tools like ChatGPT are temporarily disabled internally for fear that military secrets may be leaked or extracted. The U.S. Space Force told employees not to use the software.
For this matter, Scale AI chose to remain silent.