Small Language Models (SLMs) are gaining significant traction for use in AI agents as they allow privacy-preserving, low-latency, and on-premise deployment using local and resource-constrained devices. However, they face significant limitations in tool-calling scenarios, where they must select appropriate tools, generate valid tool invocations, or reason over dynamically changing sets of available actions. Their limited context capacity and reduced reasoning capabilities make them particularly sensitive to long prompts, extensive tool descriptions, and prompt organization, resulting in decreased tool-selection accuracy and unreliable structured outputs. While existing research has proposed various prompt and context management techniques, their effectiveness for tool-calling in dynamic environments could benefit from systematic investigation.

This thesis aims to assess the impact of various context and prompt management strategies on SLM-powered AI agents. To explore this, a modular agent will be implemented that performs different tasks using different strategies powered by different SLMs. The proposed strategies will be evaluated through controlled experiments against a defined baseline to assess their effectiveness.

The objective of this thesis is to investigate how prompt and context management strategies can improve tool-calling performance in SLM-based agents that work with limited resources. This includes the analysis of the state of the art on Small Language Models, AI agents, tool-calling, prompt engineering, context management, and other relevant work, followed by the design and implementation of a modular SLM-based agent as described above. Various prompt and context management strategies will be systematically explored, combined, and refined, and their effectiveness evaluated using a suitable systematic evaluation.