Subject
Computer Science
Topic
Artificial Intelligence
What is the main role of self-attention in a transformer?
What is the main role of self-attention in a transformer?
a)
Relate each token to relevant tokens in the contextb)
Assign fuzzy memberships onlyc)
Mutate model weights randomlyd)
Remove token order from training dataExplanation
Self-attention computes context-dependent mixtures so token representations incorporate relevant information from other positions.Related questions
A support bot must answer from a company’s current policy documents and cite them. Which design best reduces unsupported answers?A text generator should produce more varied outputs from the same prompt. Which decoding change usually helps?A generated medical answer sounds confident but has no supporting evidence. What is the safest response?In an autoregressive language model, how is a sequence generated?A pretrained model is further trained on labelled examples for one task. What process is this?Untrusted retrieved text tells a support bot to ignore its system rules. What security problem is this?Why should generated citations be checked before publication?ChatGPT, released to the public in November 2022, was developed by ____.