The World Health Organization (WHO) Global Action Plan on Physical Activity recommends adopting a systems approach to implementing and tailoring actions according to local contexts. We held group model-building workshops with key stakeholders in the Caribbean region to develop a causal loop diagram to describe the system driving the increasing physical inactivity in the region and envision the most effective ways of intervening in that system to encourage and promote physical activity. We used the causal loop diagram to inform how the WHO Global Action Plan on Physical Activity might be adapted to a local context. Although the WHO recommendations aligned well with our causal loop diagram, the diagram also illustrates the importance of local context in determining how interventions should be coordinated and implemented. Some interventions included creating safe physical activity spaces for both sexes, tackling negative attitudes to physical activity in certain contexts, including in schools and workplaces, and improving infrastructure for active transport. The causal loop diagram may also help understand how policies may be undermined or supported by key actors or where policies should be coordinated. We demonstrate how, in a region with a high level of physical inactivity and low resources, applying systems thinking with relevant stakeholders can help the targeted adaptation of global recommendations to local contexts.