Doraemon's language translator... Honyaku Konnyaku

There is a moment in almost every Doraemon episode where Nobita can’t communicate with a foreigner, or a dog, or a Martian, and Doraemon reaches into his pocket and hands him a piece of jelly. The jelly is called honyaku konnyaku (翻訳蒟蒻), translation konnyaku. You eat it. Now you can understand and speak any language. Animal, alien, dialect, dead. Done.

It’s a kid’s gag, but I keep coming back to it because the spec is unusually clean for a piece of fiction. The way I tend to read the gadget, as a biomedical engineer, is as a signal-processing pipeline with three stages. A sensor stage to pick up the source language. A decoder stage to extract meaning from a noisy biological signal. An actuator stage to produce the target language at the output. Every medical device I have ever worked on has a version of those three stages. The konnyaku is just the version where all three live inside the user.

Two of the three are mostly solved, on engineering grounds. Just not inside you yet.

The sensor and decoder, joined together, are what the field now calls end-to-end speech translation. Meta’s SeamlessM4T does this across roughly a hundred languages without going through a transcription step in the middle. Apple’s Live Translate runs inside your AirPods at around two seconds of latency, the same order of magnitude a cochlear implant has to hit to stay synchronized with a wearer’s own speech. Google has been shipping versions of this since 2017. They are not perfect, but a fluent French conversation through a pair of earbuds is shipping product, not a demo.

The interesting thing, once you have actually tried to build a piece of something like this, is that the headline accuracy number is the easy part. The hard part is the latency budget. A natural conversation needs an end-to-end loop of under roughly 400 milliseconds. Below that threshold, people stop noticing the lag. Current open speech-translation pipelines sit at 1.5 to 2 seconds, mostly because they emit only after they think they have heard a complete utterance. The next step, the thing every serious lab is working on, is streaming translation: emit partial output every few hundred milliseconds, revise it as more audio arrives. That alone shaves about a second. After that comes speculative decoding, where the model predicts a few tokens ahead and only re-runs the inference if it was wrong. That shaves another two or three hundred. Stack those, run them on a quantized on-device model in the half-billion-parameter range, and you are inside the konnyaku’s latency target. None of this is theoretical. The pieces exist. They have not all been packed into a shippable earbud at the same time yet.

The actuator side, producing the translated speech in your own voice, is where the field moved faster than most people expected. Speaker identity, the timbre and prosody that make you sound like you, is structurally entangled with linguistic content in a way that loosely mirrors how the cortex separates them. Voice-cloning models can now condition on a few seconds of reference audio and produce an output in that same timbre. Nobita opening his mouth and speaking fluent Martian in Nobita’s own voice is, by my read of the curve, about five years out as a consumer product.

The part that gets genuinely interesting, once those two are done, is what comes after. Doraemon’s konjac is auditory: speak, translate, hear. The engineering frontier, though, is moving upstream. At UCSF and a handful of other labs, intracortical electrode arrays placed over the speech-motor cortex have decoded attempted speech directly in ALS patients whose mouths can no longer form words. The neural signal is the same signal a healthy person would have generated. The translation can in principle happen before air ever leaves the lungs. Read this way, the konnyaku may turn out not to be a jelly at all. It may be a thin film of platinum-iridium microelectrodes about the size of a postage stamp, plus a downstream model the size of a credit card.

If you wanted to draw a credible block diagram of that version of the gadget today, it would look like this. A high-density ECoG or microelectrode array with a couple of thousand channels, sampling at maybe 30 kHz, sitting subdurally over the ventral sensorimotor cortex. A low-power neuromorphic chip doing first-pass phoneme decoding inside the body, so raw neural data never has to leave the user. A short-range link to a paired earbud running the same kind of streaming translation pipeline I described above, plus a voice-cloning head trained on the wearer’s own pre-implant recordings. A return path for incoming audio, vocoded into the user’s native phonetic inventory and pushed back through bone conduction. Three of those subsystems exist as research prototypes today. The fourth, integrating them into something with a power budget measured in milliwatts, is the actual engineering problem.

The piece that is still science fiction, even at that level of ambition, is the long tail. Translation systems trained on millions of hours of speech work because the data exists. Most human languages, by count, almost none of any model has heard. You cannot translate from a language you have no recordings of, and you cannot fix that with a bigger model. You have to go out and record. The engineering path here looks completely different from the konnyaku’s main loop: a small, robust mobile capture rig that works offline, a self-supervised pretraining stage (wav2vec or HuBERT class) on whatever unlabeled audio you can scrape together, phonetic transfer learning from related languages, and active learning to choose which recording session most reduces the model’s uncertainty. Federated training lets the model improve without the corpus ever leaving the village it was recorded in. None of this is futuristic. It is just slow.

This is the part I keep coming back to. The Tani languages I grew up around have, between them, a few hundred thousand speakers and at most a few dozen hours of carefully recorded audio. The training corpora behind the major translation models are measured in millions of hours. The gap is roughly six orders of magnitude. Structurally, it is the same kind of gap you see when you try to apply a model trained on healthy ECG to a rare arrhythmia. You do not close that gap with scale. You close it by going where the data is, and engineering carefully for the constraints the environment actually gives you.

Here is the thing about the konjac, though. It covers the whole list. The hardest part of that gadget is not the part that works for the languages where the engineering compounds in your favor. It is the part that works for the languages where it compounds against you. And that part still has to be built by hand, one corpus at a time, in places with patchy internet and patient elders.

The translation jelly is mostly here. The pieces are visible, even the upstream ones. The way it gets the rest of the way there is the same way the long tail of any technology gets there. Slowly, at the unglamorous edges, by the people who care.