{"id":1381,"date":"2026-01-12T11:10:57","date_gmt":"2026-01-12T10:10:57","guid":{"rendered":"https:\/\/www.ideas.edu.pl\/?post_type=publikacje&#038;p=1381"},"modified":"2026-01-12T11:22:02","modified_gmt":"2026-01-12T10:22:02","slug":"methodology-for-systematic-security-testing-of-llm-based-applications","status":"publish","type":"publikacje","link":"https:\/\/www.ideas.edu.pl\/en\/publikacje\/methodology-for-systematic-security-testing-of-llm-based-applications\/","title":{"rendered":"Methodology for Systematic Security Testing of LLM-based Applications"},"content":{"rendered":"<p>Large Language Models (LLMs) have emerged as a groundbreaking technology due to their remarkable natural language processing capabilities. They offer efficient and user-friendly solutions to complex problems, such as enabling interactive chatbots within web applications. As a result, many web applications are being developed to harness LLMs, catering to specific user needs. However, with the rapid proliferation of these applications, the importance of securing them is also growing.<br>This paper presents a systematic, easily extensible methodology for the security testing of LLM-based applications. The proposed approach consists of four phases designed to identify vulnerabilities in applications integrated with LLMs. It also discusses common attack vectors and outlines potential defense techniques. To demonstrate the effectiveness of this methodology, an experimental evaluation was conducted on popular applications from the GitHub repository. From the most widely used applications, fourteen were selected and analyzed, resulting in the identification of 24 vulnerabilities. The problems found were reported and subsequently fixed, resulting in the assignment of 16 Common Vulnerabilities and Exposures (CVEs) identifiers.<\/p>\n\n\n\n<p>Authors: Dawid Nastaj, Wojciech Mazurczyk<\/p>\n\n\n\n<div style=\"height:64px\" aria-hidden=\"true\" class=\"wp-block-spacer\"><\/div>\n\n\n\n<div class=\"wp-block-group is-layout-grid wp-container-core-group-is-layout-e2bd5cb0 wp-block-group-is-layout-grid\"><\/div>\n\n\n\n<div class=\"wp-block-group is-layout-grid wp-container-core-group-is-layout-e2bd5cb0 wp-block-group-is-layout-grid\"><\/div>\n\n\n\n<p><\/p>","protected":false},"template":"","nazwa-konferencji":[35,36],"rodzaj-publikacji":[13],"rok-publikacji":[14],"class_list":["post-1381","publikacje","type-publikacje","status-publish","hentry","nazwa-konferencji-esorics","nazwa-konferencji-konferencja-esorics","rodzaj-publikacji-artykul-konferencyjny","rok-publikacji-14"],"yoast_head":"<!-- This site is optimized with the Yoast SEO plugin v27.4 - https:\/\/yoast.com\/product\/yoast-seo-wordpress\/ -->\n<title>Methodology for Systematic Security Testing of LLM-based Applications &#8226; IDEAS Instytut Badawczy<\/title>\n<meta name=\"robots\" content=\"index, follow, max-snippet:-1, max-image-preview:large, max-video-preview:-1\" \/>\n<link rel=\"canonical\" href=\"https:\/\/www.ideas.edu.pl\/en\/publikacje\/methodology-for-systematic-security-testing-of-llm-based-applications\/\" \/>\n<meta property=\"og:locale\" content=\"en_US\" \/>\n<meta property=\"og:type\" content=\"article\" \/>\n<meta property=\"og:title\" content=\"Methodology for Systematic Security Testing of LLM-based Applications &#8226; IDEAS Instytut Badawczy\" \/>\n<meta property=\"og:description\" content=\"Large Language Models (LLMs) have emerged as a groundbreaking technology due to their remarkable natural language processing capabilities. They offer efficient and user-friendly solutions to complex problems, such as enabling interactive chatbots within web applications. As a result, many web applications are being developed to harness LLMs, catering to specific user needs. However, with the [&hellip;]\" \/>\n<meta property=\"og:url\" content=\"https:\/\/www.ideas.edu.pl\/en\/publikacje\/methodology-for-systematic-security-testing-of-llm-based-applications\/\" \/>\n<meta property=\"og:site_name\" content=\"IDEAS Instytut Badawczy\" \/>\n<meta property=\"article:modified_time\" content=\"2026-01-12T10:22:02+00:00\" \/>\n<meta property=\"og:image\" content=\"https:\/\/www.ideas.edu.pl\/wp-content\/uploads\/feature-image-home.webp\" \/>\n\t<meta property=\"og:image:width\" content=\"1800\" \/>\n\t<meta property=\"og:image:height\" content=\"945\" \/>\n\t<meta property=\"og:image:type\" content=\"image\/webp\" \/>\n<meta name=\"twitter:card\" content=\"summary_large_image\" \/>\n<meta name=\"twitter:label1\" content=\"Est. reading time\" \/>\n\t<meta name=\"twitter:data1\" content=\"1 minute\" \/>\n<script type=\"application\/ld+json\" class=\"yoast-schema-graph\">{\"@context\":\"https:\\\/\\\/schema.org\",\"@graph\":[{\"@type\":\"WebPage\",\"@id\":\"https:\\\/\\\/www.ideas.edu.pl\\\/publikacje\\\/methodology-for-systematic-security-testing-of-llm-based-applications\\\/\",\"url\":\"https:\\\/\\\/www.ideas.edu.pl\\\/publikacje\\\/methodology-for-systematic-security-testing-of-llm-based-applications\\\/\",\"name\":\"Methodology for Systematic Security Testing of LLM-based Applications &#8226; IDEAS Instytut Badawczy\",\"isPartOf\":{\"@id\":\"https:\\\/\\\/www.ideas.edu.pl\\\/#website\"},\"datePublished\":\"2026-01-12T10:10:57+00:00\",\"dateModified\":\"2026-01-12T10:22:02+00:00\",\"breadcrumb\":{\"@id\":\"https:\\\/\\\/www.ideas.edu.pl\\\/publikacje\\\/methodology-for-systematic-security-testing-of-llm-based-applications\\\/#breadcrumb\"},\"inLanguage\":\"en-US\",\"potentialAction\":[{\"@type\":\"ReadAction\",\"target\":[\"https:\\\/\\\/www.ideas.edu.pl\\\/publikacje\\\/methodology-for-systematic-security-testing-of-llm-based-applications\\\/\"]}]},{\"@type\":\"BreadcrumbList\",\"@id\":\"https:\\\/\\\/www.ideas.edu.pl\\\/publikacje\\\/methodology-for-systematic-security-testing-of-llm-based-applications\\\/#breadcrumb\",\"itemListElement\":[{\"@type\":\"ListItem\",\"position\":1,\"name\":\"Strona g\u0142\u00f3wna\",\"item\":\"https:\\\/\\\/www.ideas.edu.pl\\\/\"},{\"@type\":\"ListItem\",\"position\":2,\"name\":\"Publikacje\",\"item\":\"https:\\\/\\\/www.ideas.edu.pl\\\/publikacje\\\/\"},{\"@type\":\"ListItem\",\"position\":3,\"name\":\"Methodology for Systematic Security Testing of LLM-based Applications\"}]},{\"@type\":\"WebSite\",\"@id\":\"https:\\\/\\\/www.ideas.edu.pl\\\/#website\",\"url\":\"https:\\\/\\\/www.ideas.edu.pl\\\/\",\"name\":\"IDEAS Instytut Badawczy\",\"description\":\"Pa\u0144stwowa jednostka badawczo-naukowa\",\"publisher\":{\"@id\":\"https:\\\/\\\/www.ideas.edu.pl\\\/#organization\"},\"alternateName\":\"IDEAS\",\"potentialAction\":[{\"@type\":\"SearchAction\",\"target\":{\"@type\":\"EntryPoint\",\"urlTemplate\":\"https:\\\/\\\/www.ideas.edu.pl\\\/?s={search_term_string}\"},\"query-input\":{\"@type\":\"PropertyValueSpecification\",\"valueRequired\":true,\"valueName\":\"search_term_string\"}}],\"inLanguage\":\"en-US\"},{\"@type\":\"Organization\",\"@id\":\"https:\\\/\\\/www.ideas.edu.pl\\\/#organization\",\"name\":\"IDEAS Instytut Badawczy\",\"url\":\"https:\\\/\\\/www.ideas.edu.pl\\\/\",\"logo\":{\"@type\":\"ImageObject\",\"inLanguage\":\"en-US\",\"@id\":\"https:\\\/\\\/www.ideas.edu.pl\\\/#\\\/schema\\\/logo\\\/image\\\/\",\"url\":\"https:\\\/\\\/www.ideas.edu.pl\\\/wp-content\\\/uploads\\\/e90241d6e73025d0d829abc28d67cb84.svg\",\"contentUrl\":\"https:\\\/\\\/www.ideas.edu.pl\\\/wp-content\\\/uploads\\\/e90241d6e73025d0d829abc28d67cb84.svg\",\"width\":152,\"height\":43,\"caption\":\"IDEAS Instytut Badawczy\"},\"image\":{\"@id\":\"https:\\\/\\\/www.ideas.edu.pl\\\/#\\\/schema\\\/logo\\\/image\\\/\"}}]}<\/script>\n<!-- \/ Yoast SEO plugin. -->","yoast_head_json":{"title":"Methodology for Systematic Security Testing of LLM-based Applications &#8226; IDEAS Instytut Badawczy","robots":{"index":"index","follow":"follow","max-snippet":"max-snippet:-1","max-image-preview":"max-image-preview:large","max-video-preview":"max-video-preview:-1"},"canonical":"https:\/\/www.ideas.edu.pl\/en\/publikacje\/methodology-for-systematic-security-testing-of-llm-based-applications\/","og_locale":"en_US","og_type":"article","og_title":"Methodology for Systematic Security Testing of LLM-based Applications &#8226; IDEAS Instytut Badawczy","og_description":"Large Language Models (LLMs) have emerged as a groundbreaking technology due to their remarkable natural language processing capabilities. They offer efficient and user-friendly solutions to complex problems, such as enabling interactive chatbots within web applications. As a result, many web applications are being developed to harness LLMs, catering to specific user needs. However, with the [&hellip;]","og_url":"https:\/\/www.ideas.edu.pl\/en\/publikacje\/methodology-for-systematic-security-testing-of-llm-based-applications\/","og_site_name":"IDEAS Instytut Badawczy","article_modified_time":"2026-01-12T10:22:02+00:00","og_image":[{"width":1800,"height":945,"url":"https:\/\/www.ideas.edu.pl\/wp-content\/uploads\/feature-image-home.webp","type":"image\/webp"}],"twitter_card":"summary_large_image","twitter_misc":{"Est. reading time":"1 minute"},"schema":{"@context":"https:\/\/schema.org","@graph":[{"@type":"WebPage","@id":"https:\/\/www.ideas.edu.pl\/publikacje\/methodology-for-systematic-security-testing-of-llm-based-applications\/","url":"https:\/\/www.ideas.edu.pl\/publikacje\/methodology-for-systematic-security-testing-of-llm-based-applications\/","name":"Methodology for Systematic Security Testing of LLM-based Applications &#8226; IDEAS Instytut Badawczy","isPartOf":{"@id":"https:\/\/www.ideas.edu.pl\/#website"},"datePublished":"2026-01-12T10:10:57+00:00","dateModified":"2026-01-12T10:22:02+00:00","breadcrumb":{"@id":"https:\/\/www.ideas.edu.pl\/publikacje\/methodology-for-systematic-security-testing-of-llm-based-applications\/#breadcrumb"},"inLanguage":"en-US","potentialAction":[{"@type":"ReadAction","target":["https:\/\/www.ideas.edu.pl\/publikacje\/methodology-for-systematic-security-testing-of-llm-based-applications\/"]}]},{"@type":"BreadcrumbList","@id":"https:\/\/www.ideas.edu.pl\/publikacje\/methodology-for-systematic-security-testing-of-llm-based-applications\/#breadcrumb","itemListElement":[{"@type":"ListItem","position":1,"name":"Strona g\u0142\u00f3wna","item":"https:\/\/www.ideas.edu.pl\/"},{"@type":"ListItem","position":2,"name":"Publikacje","item":"https:\/\/www.ideas.edu.pl\/publikacje\/"},{"@type":"ListItem","position":3,"name":"Methodology for Systematic Security Testing of LLM-based Applications"}]},{"@type":"WebSite","@id":"https:\/\/www.ideas.edu.pl\/#website","url":"https:\/\/www.ideas.edu.pl\/","name":"IDEAS Research Institute","description":"State research and scientific unit","publisher":{"@id":"https:\/\/www.ideas.edu.pl\/#organization"},"alternateName":"IDEAS","potentialAction":[{"@type":"SearchAction","target":{"@type":"EntryPoint","urlTemplate":"https:\/\/www.ideas.edu.pl\/?s={search_term_string}"},"query-input":{"@type":"PropertyValueSpecification","valueRequired":true,"valueName":"search_term_string"}}],"inLanguage":"en-US"},{"@type":"Organization","@id":"https:\/\/www.ideas.edu.pl\/#organization","name":"IDEAS Research Institute","url":"https:\/\/www.ideas.edu.pl\/","logo":{"@type":"ImageObject","inLanguage":"en-US","@id":"https:\/\/www.ideas.edu.pl\/#\/schema\/logo\/image\/","url":"https:\/\/www.ideas.edu.pl\/wp-content\/uploads\/e90241d6e73025d0d829abc28d67cb84.svg","contentUrl":"https:\/\/www.ideas.edu.pl\/wp-content\/uploads\/e90241d6e73025d0d829abc28d67cb84.svg","width":152,"height":43,"caption":"IDEAS Instytut Badawczy"},"image":{"@id":"https:\/\/www.ideas.edu.pl\/#\/schema\/logo\/image\/"}}]}},"_links":{"self":[{"href":"https:\/\/www.ideas.edu.pl\/en\/wp-json\/wp\/v2\/publikacje\/1381","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/www.ideas.edu.pl\/en\/wp-json\/wp\/v2\/publikacje"}],"about":[{"href":"https:\/\/www.ideas.edu.pl\/en\/wp-json\/wp\/v2\/types\/publikacje"}],"wp:attachment":[{"href":"https:\/\/www.ideas.edu.pl\/en\/wp-json\/wp\/v2\/media?parent=1381"}],"wp:term":[{"taxonomy":"nazwa-konferencji","embeddable":true,"href":"https:\/\/www.ideas.edu.pl\/en\/wp-json\/wp\/v2\/nazwa-konferencji?post=1381"},{"taxonomy":"rodzaj-publikacji","embeddable":true,"href":"https:\/\/www.ideas.edu.pl\/en\/wp-json\/wp\/v2\/rodzaj-publikacji?post=1381"},{"taxonomy":"rok-publikacji","embeddable":true,"href":"https:\/\/www.ideas.edu.pl\/en\/wp-json\/wp\/v2\/rok-publikacji?post=1381"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}