Quiz 10 - Muti-Agent System [11/17]

JavaScript isn't enabled in your browser, so this file can't be opened. Enable and reload.

Email *

1. According to Oriol Vinyals, why was Imitation Learning (IL) or pre-training a necessary first step before applying Reinforcement Learning (RL) in the AlphaStar project?

1 point

(A) The RL algorithms available at the time were too slow to run without an initial model.

(B) The IL step was required to teach the neural network the grammatical structure of the StarCraft API calls.

(C) Pure RL failed because random policy could not discover any useful knowledge, making IL necessary to bootstrap the agent with human prior knowledge to solve the exploration problem.

(D) IL was only used to generate the final, high-performance policy after RL was complete.structed the agents to use nanobodies.

2. What primary issue with standard self-play reinforcement learning did the AlphaStar team solve by introducing the 'League of Exploiters'? *

1 point

(A) The main agents struggled to manage the long time horizon and delayed rewards of the game.

(B) The main agents were too susceptible to adversarial examples and pixel-level attacks.

(C) The main policy would over-specialize, losing the diversity of strategies and becoming fragile against strategies that the exploiters were forced to create.

(D) The ELO scoring system was found to be highly unreliable for measuring true agent strength.

3. Vinyals notes that the compute envelope for modern LLMs is heavily skewed toward pre-training (IL). What does he argue this imbalance prevents in the current LLM training paradigm?

1 point

(A) It makes the model unable to perform high-quality multi-language translation and synthesis.

(B) It restricts the use of complex Transformer architectures, forcing researchers to use simpler RNNs.

(C) It hinders the full maturity of post-training (RL) steps, where the model gains experiential knowledge and robustness, limiting the overall strength and generalization compared to the AlphaStar system.

(D) It forces developers to rely heavily on structured, fixed-format API calls rather than natural language prompting.

4. What is the key difference when applying multi-agent reinforcement learning techniques (like the League) from a competitive game like StarCraft to collaborative real-world LLM agent tasks (e.g., a math tutor)?

1 point

(A) The real-world agents are trained without any reward signal, as success is too ambiguous.

(B) The system cannot use LLM-based users and must rely entirely on human testers. that could bind to both COVID and the biomarkers for ADHD.

(D) The League's adversarial concept is adapted to expose knowledge gaps and weaknesses of the main agent by training diverse synthetic user populations.

5. How did the architecture of AlphaStar's action space relate to modern LLM function calling?

1 point

(A) AlphaStar used a highly flexible, text-based sequence-to-sequence model, while LLMs use only fixed, binary vector outputs.

(B) AlphaStar and LLMs both use identical, fixed RNN architectures, making the comparison straightforward.

(C) AlphaStar's output was an auto-regressive sequence of commands and arguments, establishing a core sequence-generation pattern that is now implemented via text-based Transformer outputs for LLM function calling.

(D) AlphaStar was able to write new Python code on the fly to call APIs, a capability LLMs are only now developing.

A copy of your responses will be emailed to the address you provided.

Submit

Clear form

reCAPTCHA

Privacy Terms

This form was created inside of UC Berkeley.

Does this form look suspicious? Report

Forms