Robot soccer, where teams of autonomous agents compete against each other, has long been regarded as a grand challenge in artificial ntelligence. Despite recent successes of learned policies over heuristics and handcrafted rules in other domains, current teams in the RoboCup soccer simulation leagues still rely on handcrafted strategies and apply reinforcement learning only on small subcomponents. This limits a learning agent’s ability to find strong, high-level strategies for the game in its entirety. In this work, we investigate whether it is possible for agents to learn competent soccer strategies in a full 22 player soccer game using limited computational resources (one CPU and one GPU), from tabula rasa and entirely through self-play.