Large language models (LLMs) have ushered in a multitude of new language generation capabilities, bringing AI-guided storytelling much closer to reality. Authors can now meaningfully engage with AI writing assistants to help during the story writing process. These same capabilities have also enabled the ability to add more diverse and complex interactions for narrative-driven role-playing games. Though due to the memory and processing that LLMs require, efficient inference techniques are needed as video games are real-time systems with demanding performance requirements. In this thesis I explore two overarching lines of inquiry: (1) What is the user experience with systems designed for language generation in storytelling and video games? and (2) How can we improve these systems to address limitations that adversely affect user experience? I investigate these questions through the lens of AI story writing assistants and video game dialogue systems. In Chapter 2 I discuss my explorations using LLMs as AI story writing assistants on the online collaborative writing platform Storium, where real authors on the Storium platform can query a model for suggested story continuations. Then in Chapter 3, I extract dialogue from the widely-acclaimed role-playing game Disco Elysium: The Final Cut, which contains 1.1M words of dialogue spread across a complex graph of utterances where node reachability depends on game state. Using a reusable research artifact — a web app that recreatse the dialogue system from the game — I explore real players’ experiences interacting with the game augmented by LLM-generated dialogue. In a natural follow-up, in Chapter 4 I examine how to enhance the player experience, while maintaining game’s existing structure, by introducing a virtual game master (GM) that allows players to type their desired response in a conversation, rather than choose from a set of pre-written options. To address the response time considerations of these real-time systems, in Chapter 5 I investigate efficient inference for the Transformer architecture by incorporating linguistic features into the decoding process. I conclude with Chapter 6, where I consider promising future directions for improving the virtual GM and ways for integrating LLMs into the video game dialogue writing process.