Attention mechanisms can only improve the performance of models when the input and output sequences are of similar lengths.

Attention mechanisms are not limited to sequences of the same length; they actually shine when the input and output differ in size. By assigning a weight to every input token for each output token, the model can focus on the most relevant parts, regardless of how many tokens are present on either side. For example, translating “I love you” (three words) into “Je t’aime” (three words) still benefits from attention, because the model learns to align “love” with “t’aime” even though the words are not identical. Thus, attention improves performance whenever the model needs to relate parts of the input to parts of the output, no matter how the lengths compare.