DeepSeek’s success learning from bigger AI models raises questions about the billions being spent on the most advanced technology.

The Medium post goes over various flavors of distillation, including response-based distillation, feature-based distillation and relation-based distillation. It also covers two fundamentally different modes of distillation – off-line and online distillation.

Since Chinese artificial intelligence (AI) start-up DeepSeek rattled Silicon Valley and Wall Street with its cost-effective models, the company has been accused of data theft through a practice that is common across the industry.

Whether it's ChatGPT since the past couple of years or DeepSeek more recently, the field of artificial intelligence (AI) has seen rapid advancements, with models becoming increasingly large and complex.

OpenAI accuses Chinese AI firm DeepSeek of stealing its content through "knowledge distillation," sparking concerns over security, ethics, and national interests.

One possible answer being floated in tech circles is distillation, an AI training method that uses bigger "teacher" models to train smaller but faster-operating "student" models.

Microsoft and OpenAI are investigating whether DeepSeek, a Chinese artificial intelligence startup, illegally copying proprietary American technology, sources told Bloomberg

If there are elements that we want a smaller AI model to have, and the larger models contain it, a kind of transference can be undertaken, formally known as knowledge distillation since you ...

DeepSeek: After US Navy, Congressional offices have been warned not to use DeepSeek, an upstart Chinese chatbot that is roiling the American AI market. Prior to this, the US Navy instructed its members to avoid using DeepSeek over national security concerns.

The Nasdaq Composite fell 3.1% on Monday, while AI leader Nvidia tumbled 17%. But the reality was, and is, far more complicated. DeepSeek didn’t replicate OpenAI’s ability by spending a few million dollars.