{"id":8748,"date":"2026-01-20T18:27:58","date_gmt":"2026-01-20T12:57:58","guid":{"rendered":"https:\/\/www.testleaf.com\/blog\/?p=8748"},"modified":"2026-01-20T18:28:37","modified_gmt":"2026-01-20T12:58:37","slug":"playwright-ai-session-preservation-failure-triage-ci","status":"publish","type":"post","link":"https:\/\/www.testleaf.com\/blog\/playwright-ai-session-preservation-failure-triage-ci\/","title":{"rendered":"Playwright + AI: AI-Driven Session Preservation &#038; Failure Triage (Built for Real CI)"},"content":{"rendered":"<div style=\"margin-top: 0px; margin-bottom: 0px;\" class=\"sharethis-inline-share-buttons\" ><\/div><!--[if lt IE 9]><script>document.createElement('audio');<\/script><![endif]-->\n<audio class=\"wp-audio-shortcode\" id=\"audio-8748-1\" preload=\"none\" style=\"width: 100%;\" controls=\"controls\"><source type=\"audio\/mpeg\" src=\"https:\/\/www.testleaf.com\/blog\/wp-content\/uploads\/2026\/01\/Playwright-AI-AI-Driven-Session-Preservation-Failure-Triage.mp3?_=1\" \/><a href=\"https:\/\/www.testleaf.com\/blog\/wp-content\/uploads\/2026\/01\/Playwright-AI-AI-Driven-Session-Preservation-Failure-Triage.mp3\">https:\/\/www.testleaf.com\/blog\/wp-content\/uploads\/2026\/01\/Playwright-AI-AI-Driven-Session-Preservation-Failure-Triage.mp3<\/a><\/audio>\n<p>&nbsp;<\/p>\n<p>If you run Playwright in CI at scale, you\u2019ve seen this movie.<\/p>\n<p>\u2705 Test fails<br \/>\n\u2705 You open the screenshot<br \/>\n\u2705 Instead of the <a href=\"https:\/\/www.testleaf.com\/blog\/from-manual-qa-reports-to-automated-dashboards\/\">dashboard<\/a>\u2026 you\u2019re staring at the login page<\/p>\n<p>And suddenly everyone wonders: <strong>\u201cDid we break prod?\u201d<\/strong><\/p>\n<p>Most of the time, you didn\u2019t.<\/p>\n<p>What you hit is an <strong>auth\/session artifact<\/strong>\u2014a flaky failure that <em>looks<\/em> like a regression, but isn\u2019t. This post is a practical guide to turning those \u201cmystery failures\u201d into <strong>fast, repeatable diagnosis<\/strong> using a simple combo:<\/p>\n<p><strong>Rules-first triage + AI summaries + safe smart retries.<\/strong><\/p>\n<h2><span class=\"ez-toc-section\" id=\"Why_this_keeps_happening_even_with_storageState\"><\/span><strong>Why this keeps happening (even with <\/strong><strong>storageState<\/strong><strong>)<\/strong><span class=\"ez-toc-section-end\"><\/span><\/h2><div id=\"ez-toc-container\" class=\"ez-toc-v2_0_82_2 counter-hierarchy ez-toc-counter ez-toc-grey ez-toc-container-direction\">\n<div class=\"ez-toc-title-container\">\n<p class=\"ez-toc-title\" style=\"cursor:inherit\">Table of Contents<\/p>\n<span class=\"ez-toc-title-toggle\"><a href=\"#\" class=\"ez-toc-pull-right ez-toc-btn ez-toc-btn-xs ez-toc-btn-default ez-toc-toggle\" aria-label=\"Toggle Table of Content\"><span class=\"ez-toc-js-icon-con\"><span class=\"\"><span class=\"eztoc-hide\" style=\"display:none;\">Toggle<\/span><span class=\"ez-toc-icon-toggle-span\"><svg style=\"fill: #999;color:#999\" xmlns=\"http:\/\/www.w3.org\/2000\/svg\" class=\"list-377408\" width=\"20px\" height=\"20px\" viewBox=\"0 0 24 24\" fill=\"none\"><path d=\"M6 6H4v2h2V6zm14 0H8v2h12V6zM4 11h2v2H4v-2zm16 0H8v2h12v-2zM4 16h2v2H4v-2zm16 0H8v2h12v-2z\" fill=\"currentColor\"><\/path><\/svg><svg style=\"fill: #999;color:#999\" class=\"arrow-unsorted-368013\" xmlns=\"http:\/\/www.w3.org\/2000\/svg\" width=\"10px\" height=\"10px\" viewBox=\"0 0 24 24\" version=\"1.2\" baseProfile=\"tiny\"><path d=\"M18.2 9.3l-6.2-6.3-6.2 6.3c-.2.2-.3.4-.3.7s.1.5.3.7c.2.2.4.3.7.3h11c.3 0 .5-.1.7-.3.2-.2.3-.5.3-.7s-.1-.5-.3-.7zM5.8 14.7l6.2 6.3 6.2-6.3c.2-.2.3-.5.3-.7s-.1-.5-.3-.7c-.2-.2-.4-.3-.7-.3h-11c-.3 0-.5.1-.7.3-.2.2-.3.5-.3.7s.1.5.3.7z\"\/><\/svg><\/span><\/span><\/span><\/a><\/span><\/div>\n<nav><ul class='ez-toc-list ez-toc-list-level-1 ' ><li class='ez-toc-page-1 ez-toc-heading-level-2'><a class=\"ez-toc-link ez-toc-heading-1\" href=\"https:\/\/www.testleaf.com\/blog\/playwright-ai-session-preservation-failure-triage-ci\/#Why_this_keeps_happening_even_with_storageState\" >Why this keeps happening (even with storageState)<\/a><\/li><li class='ez-toc-page-1 ez-toc-heading-level-2'><a class=\"ez-toc-link ez-toc-heading-2\" href=\"https:\/\/www.testleaf.com\/blog\/playwright-ai-session-preservation-failure-triage-ci\/#1_What_%E2%80%9Csession_preservation%E2%80%9D_actually_means\" >1) What \u201csession preservation\u201d actually means<\/a><\/li><li class='ez-toc-page-1 ez-toc-heading-level-2'><a class=\"ez-toc-link ez-toc-heading-3\" href=\"https:\/\/www.testleaf.com\/blog\/playwright-ai-session-preservation-failure-triage-ci\/#2_What_%E2%80%9Cfailure_triage%E2%80%9D_means_and_why_it_matters\" >2) What \u201cfailure triage\u201d means (and why it matters)<\/a><\/li><li class='ez-toc-page-1 ez-toc-heading-level-2'><a class=\"ez-toc-link ez-toc-heading-4\" href=\"https:\/\/www.testleaf.com\/blog\/playwright-ai-session-preservation-failure-triage-ci\/#3_The_Signal_Map_the_backbone_of_reliable_triage\" >3) The Signal Map (the backbone of reliable triage)<\/a><\/li><li class='ez-toc-page-1 ez-toc-heading-level-2'><a class=\"ez-toc-link ez-toc-heading-5\" href=\"https:\/\/www.testleaf.com\/blog\/playwright-ai-session-preservation-failure-triage-ci\/#4_Practical_implementation_Rules-first_AI_second\" >4) Practical implementation: Rules-first + AI second<\/a><\/li><li class='ez-toc-page-1 ez-toc-heading-level-2'><a class=\"ez-toc-link ez-toc-heading-6\" href=\"https:\/\/www.testleaf.com\/blog\/playwright-ai-session-preservation-failure-triage-ci\/#5_Where_AI_fits_without_becoming_risky\" >5) Where AI fits (without becoming risky)<\/a><\/li><li class='ez-toc-page-1 ez-toc-heading-level-2'><a class=\"ez-toc-link ez-toc-heading-7\" href=\"https:\/\/www.testleaf.com\/blog\/playwright-ai-session-preservation-failure-triage-ci\/#6_The_safe_%E2%80%9CSmart_Retry%E2%80%9D_loop_the_real_win\" >6) The safe \u201cSmart Retry\u201d loop (the real win)<\/a><\/li><li class='ez-toc-page-1 ez-toc-heading-level-2'><a class=\"ez-toc-link ez-toc-heading-8\" href=\"https:\/\/www.testleaf.com\/blog\/playwright-ai-session-preservation-failure-triage-ci\/#7_About_Playwright_traces_use_them_properly\" >7) About Playwright traces (use them properly)<\/a><\/li><li class='ez-toc-page-1 ez-toc-heading-level-2'><a class=\"ez-toc-link ez-toc-heading-9\" href=\"https:\/\/www.testleaf.com\/blog\/playwright-ai-session-preservation-failure-triage-ci\/#FAQs\" >FAQs<\/a><\/li><\/ul><\/nav><\/div>\n\n<p>Even with clean login design and storageState, Playwright suites still fail for reasons like:<\/p>\n<ul>\n<li><strong>Session expired earlier than expected<\/strong><\/li>\n<li><strong>Storage state not applied<\/strong> (wrong context, stale state, wrong worker)<\/li>\n<li><strong>Redirect loops back to <\/strong><strong>\/login<\/strong><\/li>\n<li><strong>403 \u201cAccess denied\u201d<\/strong> (role\/permission mismatch)<\/li>\n<li><strong>Environment instability<\/strong> (auth service hiccups, timeouts that look like auth)<\/li>\n<\/ul>\n<p><img fetchpriority=\"high\" decoding=\"async\" class=\"aligncenter size-full wp-image-8753\" src=\"https:\/\/www.testleaf.com\/blog\/wp-content\/uploads\/2026\/01\/Why-Auth-Session-Failures-Keep-Breaking-CI.jpg\" alt=\"Why Auth &amp; Session Failures Keep Breaking CI\" width=\"2048\" height=\"1143\" srcset=\"https:\/\/www.testleaf.com\/blog\/wp-content\/uploads\/2026\/01\/Why-Auth-Session-Failures-Keep-Breaking-CI.jpg 2048w, https:\/\/www.testleaf.com\/blog\/wp-content\/uploads\/2026\/01\/Why-Auth-Session-Failures-Keep-Breaking-CI-300x167.jpg 300w, https:\/\/www.testleaf.com\/blog\/wp-content\/uploads\/2026\/01\/Why-Auth-Session-Failures-Keep-Breaking-CI-1024x572.jpg 1024w, https:\/\/www.testleaf.com\/blog\/wp-content\/uploads\/2026\/01\/Why-Auth-Session-Failures-Keep-Breaking-CI-768x429.jpg 768w, https:\/\/www.testleaf.com\/blog\/wp-content\/uploads\/2026\/01\/Why-Auth-Session-Failures-Keep-Breaking-CI-1536x857.jpg 1536w, https:\/\/www.testleaf.com\/blog\/wp-content\/uploads\/2026\/01\/Why-Auth-Session-Failures-Keep-Breaking-CI-150x84.jpg 150w\" sizes=\"(max-width: 2048px) 100vw, 2048px\" \/><\/p>\n<p>Traditionally, a human has to:<\/p>\n<ul>\n<li>open logs and <a href=\"https:\/\/www.testleaf.com\/blog\/screenshots-to-videos-the-evolution-of-qa-evidence\/\">screenshots<\/a>,<\/li>\n<li>replay the trace,<\/li>\n<li>guess the root cause,<\/li>\n<li>re-run the pipeline \u201cjust to confirm.\u201d<\/li>\n<\/ul>\n<p>That\u2019s expensive. And it\u2019s repeated\u2026 every day.<\/p>\n<p>So here\u2019s the better approach:<\/p>\n<p>Don\u2019t make every failure a detective story.<br \/>\nMake failures fall into <strong>known buckets<\/strong> with <strong>known fixes<\/strong>.<\/p>\n<p><strong>Popular Articles:<\/strong> <a href=\"https:\/\/www.testleaf.com\/blog\/top-30-playwright-interview-questions-and-answers-2025-updated-guide\/\">playwright interview questions<\/a><\/p>\n<h2><span class=\"ez-toc-section\" id=\"1_What_%E2%80%9Csession_preservation%E2%80%9D_actually_means\"><\/span><strong>1) What \u201csession preservation\u201d actually means<\/strong><span class=\"ez-toc-section-end\"><\/span><\/h2>\n<p>In this context, session preservation means:<\/p>\n<p>Keeping a test run consistently authenticated so tests don\u2019t accidentally run as an anonymous user.<\/p>\n<p>Important nuance:<\/p>\n<p><strong>storageState<\/strong><strong> doesn\u2019t keep sessions alive.<\/strong><br \/>\nIt\u2019s a snapshot (cookies + localStorage) captured at a moment in time.<\/p>\n<p>It becomes stale when:<\/p>\n<ul>\n<li>cookies expire,<\/li>\n<li>tokens rotate or refresh fails,<\/li>\n<li>sessions get invalidated after deployments,<\/li>\n<li>long <a href=\"https:\/\/www.testleaf.com\/blog\/parallel-test-execution-with-selenium-grid-azure-kubernetes-scaling-qa-without-sacrificing-time\/\">parallel runs<\/a> reuse outdated state files.<\/li>\n<\/ul>\n<p>Where AI helps is not \u201clogging in for you,\u201d but <strong>watching your runs<\/strong> and pointing to patterns like:<\/p>\n<ul>\n<li>\u201cThese 15 failures match auth\/session behavior, not regression bugs.\u201d<\/li>\n<li>\u201cThis suite\u2019s auth.json expires mid-run\u2014refresh cadence is too low.\u201d<\/li>\n<\/ul>\n<h2><span class=\"ez-toc-section\" id=\"2_What_%E2%80%9Cfailure_triage%E2%80%9D_means_and_why_it_matters\"><\/span><strong>2) What \u201cfailure triage\u201d means (and why it matters)<\/strong><span class=\"ez-toc-section-end\"><\/span><\/h2>\n<p>Failure triage is the art of classifying failures into <strong>actionable buckets<\/strong> instead of staring at a stack trace.<\/p>\n<p>Instead of \u201cTimeout 30s\u201d you want:<\/p>\n<ul>\n<li><strong>Auth issue<\/strong><\/li>\n<li><strong>Permission issue<\/strong><\/li>\n<li><strong><a href=\"https:\/\/www.testleaf.com\/blog\/ai-powered-element-locators-selenium-automation\/\">Locator<\/a> issue<\/strong><\/li>\n<li><strong>Perf\/timeout<\/strong><\/li>\n<li><strong>App bug \/ environment<\/strong><\/li>\n<\/ul>\n<p><img decoding=\"async\" class=\"aligncenter size-full wp-image-8754\" src=\"https:\/\/www.testleaf.com\/blog\/wp-content\/uploads\/2026\/01\/Failure-Triage-with-a-Signal-Map.jpg\" alt=\"Failure Triage with a Signal Map\" width=\"2048\" height=\"1143\" srcset=\"https:\/\/www.testleaf.com\/blog\/wp-content\/uploads\/2026\/01\/Failure-Triage-with-a-Signal-Map.jpg 2048w, https:\/\/www.testleaf.com\/blog\/wp-content\/uploads\/2026\/01\/Failure-Triage-with-a-Signal-Map-300x167.jpg 300w, https:\/\/www.testleaf.com\/blog\/wp-content\/uploads\/2026\/01\/Failure-Triage-with-a-Signal-Map-1024x572.jpg 1024w, https:\/\/www.testleaf.com\/blog\/wp-content\/uploads\/2026\/01\/Failure-Triage-with-a-Signal-Map-768x429.jpg 768w, https:\/\/www.testleaf.com\/blog\/wp-content\/uploads\/2026\/01\/Failure-Triage-with-a-Signal-Map-1536x857.jpg 1536w, https:\/\/www.testleaf.com\/blog\/wp-content\/uploads\/2026\/01\/Failure-Triage-with-a-Signal-Map-150x84.jpg 150w\" sizes=\"(max-width: 2048px) 100vw, 2048px\" \/><\/p>\n<p>The goal is a single high-signal line:<\/p>\n<p>\u201cUser session was invalid \u2192 \/orders redirected to \/login after a 401.\u201d<\/p>\n<p>That one sentence can save <strong>10\u201315 minutes per failure<\/strong>.<\/p>\n<p><strong>Other Helpful Articles:<\/strong> <a href=\"https:\/\/www.testleaf.com\/blog\/ai-ml-engineer-salary-in-india-2026-freshers-to-senior-level\/\">AI and ML engineer salary in india 2026<\/a><\/p>\n<h2><span class=\"ez-toc-section\" id=\"3_The_Signal_Map_the_backbone_of_reliable_triage\"><\/span><strong>3) The Signal Map (the backbone of reliable triage)<\/strong><span class=\"ez-toc-section-end\"><\/span><\/h2>\n<p>Before you go \u201cfull AI,\u201d start with a simple signal map. It gives you fast wins and makes AI more accurate.<\/p>\n<table>\n<thead>\n<tr>\n<td><strong>Signal observed<\/strong><\/td>\n<td><strong>What it usually means<\/strong><\/td>\n<td><strong>Bucket<\/strong><\/td>\n<\/tr>\n<\/thead>\n<tbody>\n<tr>\n<td>Final URL is \/login or contains returnUrl=<\/td>\n<td>Session invalid or not applied<\/td>\n<td>AUTH_ISSUE<\/td>\n<\/tr>\n<tr>\n<td>Redirect chain includes 302 \u2192 \/login<\/td>\n<td>Session expired on protected route<\/td>\n<td>AUTH_ISSUE<\/td>\n<\/tr>\n<tr>\n<td>API returns <strong>401<\/strong> on key endpoints (\/me, \/session)<\/td>\n<td>Token expired \/ state stale<\/td>\n<td>AUTH_ISSUE<\/td>\n<\/tr>\n<tr>\n<td>API returns <strong>403<\/strong> (especially on admin routes)<\/td>\n<td>Role\/permission mismatch<\/td>\n<td>PERMISSION_ISSUE<\/td>\n<\/tr>\n<tr>\n<td>UI text: \u201cSign in again\u201d \/ \u201cSession expired\u201d<\/td>\n<td>App logged user out<\/td>\n<td>AUTH_ISSUE<\/td>\n<\/tr>\n<tr>\n<td>500\/503 spikes<\/td>\n<td>Real app issue or outage<\/td>\n<td>APP_OR_ENV<\/td>\n<\/tr>\n<\/tbody>\n<\/table>\n<p><strong>Key idea:<\/strong> You don\u2019t need an LLM to discover obvious signals.<br \/>\nUse rules for speed. Use AI for summarizing and guidance.<\/p>\n<h2><span class=\"ez-toc-section\" id=\"4_Practical_implementation_Rules-first_AI_second\"><\/span><strong>4) Practical implementation: Rules-first + AI second<\/strong><span class=\"ez-toc-section-end\"><\/span><\/h2>\n<p>A 10\/10 CI approach looks like this:<\/p>\n<ol>\n<li>Collect a small set of <strong>high-signal context<\/strong> when a test fails<\/li>\n<li>Classify quickly using rules<\/li>\n<li>Ask AI for a <strong>structured summary + recommendation<\/strong><\/li>\n<li>Attach it to reports \/ Slack \/ CI logs<\/li>\n<li>Optionally <a href=\"https:\/\/www.testleaf.com\/blog\/self-healing-playwright-login-session-failures\/\">self-heal<\/a> with guardrails (retry once)<\/li>\n<\/ol>\n<p><img decoding=\"async\" class=\"aligncenter size-full wp-image-8752\" src=\"https:\/\/www.testleaf.com\/blog\/wp-content\/uploads\/2026\/01\/Rules-First-AI-Safe-Smart-Retry.jpg\" alt=\"Rules-First + AI + Safe Smart Retry\" width=\"2048\" height=\"1143\" srcset=\"https:\/\/www.testleaf.com\/blog\/wp-content\/uploads\/2026\/01\/Rules-First-AI-Safe-Smart-Retry.jpg 2048w, https:\/\/www.testleaf.com\/blog\/wp-content\/uploads\/2026\/01\/Rules-First-AI-Safe-Smart-Retry-300x167.jpg 300w, https:\/\/www.testleaf.com\/blog\/wp-content\/uploads\/2026\/01\/Rules-First-AI-Safe-Smart-Retry-1024x572.jpg 1024w, https:\/\/www.testleaf.com\/blog\/wp-content\/uploads\/2026\/01\/Rules-First-AI-Safe-Smart-Retry-768x429.jpg 768w, https:\/\/www.testleaf.com\/blog\/wp-content\/uploads\/2026\/01\/Rules-First-AI-Safe-Smart-Retry-1536x857.jpg 1536w, https:\/\/www.testleaf.com\/blog\/wp-content\/uploads\/2026\/01\/Rules-First-AI-Safe-Smart-Retry-150x84.jpg 150w\" sizes=\"(max-width: 2048px) 100vw, 2048px\" \/><\/p>\n<h3><strong>Step 1: Collect failure signals (URL + UI hint + console + network)<\/strong><\/h3>\n<pre>\/\/ helpers\/triageSignals.ts\r\n\r\nimport { Page } from \"@playwright\/test\";\r\n\r\nexport function wireCollectors(page: Page) {\r\n\r\n\u00a0 const consoleErrors: string[] = [];\r\n\r\n\u00a0 const suspiciousResponses: { url: string; status: number }[] = [];\r\n\r\n\u00a0 page.on(\"console\", (msg) =&gt; {\r\n\r\n\u00a0\u00a0\u00a0 if (msg.type() === \"error\") consoleErrors.push(msg.text());\r\n\r\n\u00a0 });\r\n\r\n\u00a0 page.on(\"response\", (res) =&gt; {\r\n\r\n\u00a0\u00a0\u00a0 const status = res.status();\r\n\r\n\u00a0\u00a0\u00a0 if ([401, 403, 500, 502, 503].includes(status)) {\r\n\r\n\u00a0\u00a0\u00a0\u00a0\u00a0 suspiciousResponses.push({ url: res.url(), status });\r\n\r\n\u00a0\u00a0\u00a0 }\r\n\r\n\u00a0 });\r\n\r\n\u00a0 return { consoleErrors, suspiciousResponses };\r\n\r\n}\r\n\r\nexport async function collectFailureSignals(page: Page, collectors: ReturnType&lt;typeof wireCollectors&gt;) {\r\n\r\n\u00a0 const finalUrl = page.url();\r\n\r\n\u00a0 const pageTitle = await page.title().catch(() =&gt; \"\");\r\n\r\n\u00a0 const uiAuthHint = await page\r\n\r\n\u00a0\u00a0\u00a0 .locator(\"text=\/session expired|sign in again|access denied|login\/i\")\r\n\r\n\u00a0\u00a0\u00a0 .first()\r\n\r\n\u00a0\u00a0\u00a0 .isVisible()\r\n\r\n\u00a0\u00a0\u00a0 .catch(() =&gt; false);\r\n\r\n\u00a0 return {\r\n\r\n\u00a0\u00a0\u00a0 finalUrl,\r\n\r\n\u00a0\u00a0\u00a0 pageTitle,\r\n\r\n\u00a0\u00a0\u00a0 uiAuthHint,\r\n\r\n\u00a0\u00a0\u00a0 consoleErrors: collectors.consoleErrors.slice(-10),\r\n\r\n\u00a0\u00a0\u00a0 suspiciousResponses: collectors.suspiciousResponses.slice(-15),\r\n\r\n\u00a0 };\r\n\r\n}<\/pre>\n<h3><strong>Step 2: Classify by rules (fast baseline)<\/strong><\/h3>\n<pre>\/\/ helpers\/rules.ts\r\n\r\nexport function classifyByRules(signals: { finalUrl: string; uiAuthHint: boolean; suspiciousResponses: {status:number}[] }) {\r\n\r\n\u00a0 const isLogin = \/\\\/login\\b\/i.test(signals.finalUrl) || \/returnUrl=\/i.test(signals.finalUrl);\r\n\r\n\u00a0 const has401 = signals.suspiciousResponses.some(r =&gt; r.status === 401);\r\n\r\n\u00a0 const has403 = signals.suspiciousResponses.some(r =&gt; r.status === 403);\r\n\r\n\r\n\u00a0 if (isLogin || has401 || signals.uiAuthHint) return { bucket: \"AUTH_ISSUE\", confidence: 0.85 };\r\n\r\n\u00a0 if (has403) return { bucket: \"PERMISSION_ISSUE\", confidence: 0.8 };\r\n\r\n\r\n\u00a0 return { bucket: \"UNKNOWN\", confidence: 0.3 };\r\n\r\n}<\/pre>\n<p><strong>Continue Reading:<\/strong> <a href=\"https:\/\/www.testleaf.com\/blog\/top-10-product-based-companies-in-chennai-for-tech-professionals\/\">Top 10 product based companies in chennai<\/a><\/p>\n<h3><strong>Step 3: Attach it to the report (so humans and AI can consume it)<\/strong><\/h3>\n<pre>\/\/ example.spec.ts\r\n\r\nimport { test } from \"@playwright\/test\";\r\n\r\nimport { wireCollectors, collectFailureSignals } from \".\/helpers\/triageSignals\";\r\n\r\nimport { classifyByRules } from \".\/helpers\/rules\";\r\n\r\ntest.beforeEach(async ({ page }, testInfo) =&gt; {\r\n\r\n\u00a0 (testInfo as any)._collectors = wireCollectors(page);\r\n\r\n});\r\n\r\ntest.afterEach(async ({ page }, testInfo) =&gt; {\r\n\r\n\u00a0 if (testInfo.status !== testInfo.expectedStatus) {\r\n\r\n\u00a0\u00a0\u00a0 const collectors = (testInfo as any)._collectors;\r\n\r\n\u00a0\u00a0\u00a0 const signals = await collectFailureSignals(page, collectors);\r\n\r\n\u00a0\u00a0\u00a0 const ruleVerdict = classifyByRules(signals);\r\n\r\n\r\n\u00a0\u00a0\u00a0 testInfo.attach(\"failure-signals.json\", {\r\n\r\n\u00a0\u00a0\u00a0\u00a0\u00a0 body: JSON.stringify({ signals, ruleVerdict }, null, 2),\r\n\r\n\u00a0\u00a0\u00a0\u00a0\u00a0 contentType: \"application\/json\",\r\n\r\n\u00a0\u00a0\u00a0 });\r\n\r\n\u00a0 }\r\n\r\n});<\/pre>\n<h2><span class=\"ez-toc-section\" id=\"5_Where_AI_fits_without_becoming_risky\"><\/span><strong>5) Where AI fits (without becoming risky)<\/strong><span class=\"ez-toc-section-end\"><\/span><\/h2>\n<p>Here\u2019s the simple rule:<\/p>\n<p>\u2705 Send <strong>signals<\/strong><br \/>\n\u274c Don\u2019t send secrets<\/p>\n<p><strong>What AI should ingest (safe inputs)<\/strong><\/p>\n<ul>\n<li>final URL + redirect indicators<\/li>\n<li>sanitized console errors<\/li>\n<li>key network status codes (401\/403\/500)<\/li>\n<li>page title + tiny UI hints<\/li>\n<\/ul>\n<p><strong>Do NOT send:<\/strong> raw tokens, passwords, full <a href=\"https:\/\/www.testleaf.com\/blog\/selenium-dom-properties-explained-fix-hidden-error-messages\/\">DOM<\/a> dumps with customer data.<\/p>\n<p><a href=\"https:\/\/playwright-webinar.testleaf.com\/?utm_source=Playwright_Webinar&amp;utm_medium=Organic&amp;utm_campaign=Playwright_Webinar\"><img loading=\"lazy\" decoding=\"async\" class=\"aligncenter wp-image-7702 size-full\" src=\"https:\/\/www.testleaf.com\/blog\/wp-content\/uploads\/2025\/11\/Playwright-Masterclass.png\" alt=\"Playwright Masterclass\" width=\"2048\" height=\"512\" srcset=\"https:\/\/www.testleaf.com\/blog\/wp-content\/uploads\/2025\/11\/Playwright-Masterclass.png 2048w, https:\/\/www.testleaf.com\/blog\/wp-content\/uploads\/2025\/11\/Playwright-Masterclass-300x75.png 300w, https:\/\/www.testleaf.com\/blog\/wp-content\/uploads\/2025\/11\/Playwright-Masterclass-1024x256.png 1024w, https:\/\/www.testleaf.com\/blog\/wp-content\/uploads\/2025\/11\/Playwright-Masterclass-768x192.png 768w, https:\/\/www.testleaf.com\/blog\/wp-content\/uploads\/2025\/11\/Playwright-Masterclass-1536x384.png 1536w, https:\/\/www.testleaf.com\/blog\/wp-content\/uploads\/2025\/11\/Playwright-Masterclass-150x38.png 150w\" sizes=\"(max-width: 2048px) 100vw, 2048px\" \/><\/a><\/p>\n<p><strong>What AI should output (useful summary)<\/strong><\/p>\n<p>A great AI response is structured:<\/p>\n<ul>\n<li><strong>Bucket:<\/strong> AUTH_ISSUE (0.86)<\/li>\n<li><strong>Evidence:<\/strong> \/orders \u2192 302 \u2192 \/login, 401 on \/api\/me<\/li>\n<li><strong>Likely cause:<\/strong> stale storageState<\/li>\n<li><strong>Recommendation:<\/strong> refresh auth state before suite; retry once<\/li>\n<\/ul>\n<p>Also: AI should be <strong>probabilistic<\/strong>. Treat it as triage help\u2014not truth.<\/p>\n<p><strong>You Might Also Like:<\/strong> <a href=\"https:\/\/www.testleaf.com\/blog\/2025-top-automation-testing-infosys-interview-questions-with-expert-answers-from-testleaf-for-2-to-5-years-experience\/\">Automation testing interview question<\/a><\/p>\n<h2><span class=\"ez-toc-section\" id=\"6_The_safe_%E2%80%9CSmart_Retry%E2%80%9D_loop_the_real_win\"><\/span><strong>6) The safe \u201cSmart Retry\u201d loop (the real win)<\/strong><span class=\"ez-toc-section-end\"><\/span><\/h2>\n<p>Most pipelines retry everything. That\u2019s lazy and dangerous\u2014it can hide real bugs.<\/p>\n<p>A smart retry has guardrails:<\/p>\n<ol>\n<li>Test fails<\/li>\n<li>Rules\/AI says: AUTH_ISSUE with confidence \u2265 0.8<\/li>\n<li>Refresh session (regenerate auth.json)<\/li>\n<li>Retry <strong>exactly once<\/strong><\/li>\n<li>Log the self-heal event so engineers can fix root cause later<\/li>\n<\/ol>\n<p><strong>Critical rule:<\/strong> Never auto-retry irreversible flows<br \/>\n(e.g., \u201cSubmit Payment\u201d, \u201cDelete User\u201d, \u201cPlace Order\u201d)<\/p>\n<h2><span class=\"ez-toc-section\" id=\"7_About_Playwright_traces_use_them_properly\"><\/span><strong>7) About <a href=\"https:\/\/www.testleaf.com\/blog\/playwright-trace-viewer\/\">Playwright traces<\/a> (use them properly)<\/strong><span class=\"ez-toc-section-end\"><\/span><\/h2>\n<p>Traces are excellent\u2014but they\u2019re heavy .zip artifacts.<\/p>\n<p>Best practice:<\/p>\n<ul>\n<li>Store trace .zip in CI as usual<\/li>\n<li>Prefer runtime-captured signals first (console\/response\/url)<\/li>\n<li>If needed, parse trace later to enrich reporting<\/li>\n<li>Post AI summaries to Slack \/ PR comments \/ dashboards<\/li>\n<\/ul>\n<p>This keeps costs down and security tight.<\/p>\n<h3><strong>Conclusion<\/strong><\/h3>\n<p>Authentication issues are the hidden killers of UI test reliability.<\/p>\n<p>Playwright gives you the mechanics (storageState, globalSetup, traces).<br \/>\nAI gives you the intelligence: <strong>triage, pattern recognition, and fast summaries.<\/strong><\/p>\n<p>With <strong>rules-first signals + AI summarization<\/strong>, you can:<\/p>\n<ul>\n<li>detect session failures quickly,<\/li>\n<li>stop treating auth flakes like regressions,<\/li>\n<li>safely self-heal common auth failures,<\/li>\n<li>and reduce time spent debugging false positives.<\/li>\n<\/ul>\n<p>You still own test design.<br \/>\nBut AI becomes the analyst that watches every run and tells you what actually happened.<\/p>\n<p>If you\u2019re considering a <a href=\"https:\/\/www.testleaf.com\/course\/playwright.html?utm_source=blog_post&amp;utm_medium=Organic&amp;utm_campaign=Blog_Post\"><strong data-start=\"24\" data-end=\"52\">playwright course online<\/strong><\/a>, this is the exact kind of CI-ready skill that pays off\u2014knowing how to diagnose auth flakes, apply safe guardrails, and keep pipelines green without hiding real bugs. Also, join our Playwright + AI webinar <strong data-start=\"259\" data-end=\"307\">\u201c<a href=\"https:\/\/playwright-webinar.testleaf.com\/?utm_source=Playwright_Webinar&amp;utm_medium=Organic&amp;utm_campaign=Playwright_Webinar\">Worried about your testing career in 2026?<\/a>\u201d<\/strong> to see these workflows (signal-based triage, AI summaries, and smart retries) demonstrated with real examples you can reuse in your own projects.<\/p>\n<p>&nbsp;<\/p>\n<h2 data-start=\"462\" data-end=\"513\"><span class=\"ez-toc-section\" id=\"FAQs\"><\/span><strong>FAQs<\/strong><span class=\"ez-toc-section-end\"><\/span><\/h2>\n<h3 data-start=\"515\" data-end=\"570\"><strong>1) What is \u201csession preservation\u201d in Playwright CI?<\/strong><\/h3>\n<p data-start=\"571\" data-end=\"743\">Session preservation means keeping tests consistently authenticated so they don\u2019t accidentally run as anonymous users during a CI run.<\/p>\n<h3 data-start=\"745\" data-end=\"802\"><strong>2) Does Playwright storageState keep a session alive?<\/strong><\/h3>\n<p data-start=\"803\" data-end=\"985\">No. <code data-start=\"807\" data-end=\"821\">storageState<\/code> is a snapshot of cookies and localStorage captured at a point in time\u2014it can become stale as cookies expire or tokens rotate.<\/p>\n<h3 data-start=\"987\" data-end=\"1049\"><strong>3) Why do Playwright tests still hit the login page in CI?<\/strong><\/h3>\n<p data-start=\"1050\" data-end=\"1254\">Common causes include early session expiry, storage state not applied correctly, redirect loops to <code data-start=\"1149\" data-end=\"1157\">\/login<\/code>, permission mismatches (403), and environment instability.<\/p>\n<h3 data-start=\"1256\" data-end=\"1308\"><strong>4) What is failure triage in automation testing?<\/strong><\/h3>\n<p data-start=\"1309\" data-end=\"1517\">Failure triage is classifying failures into actionable buckets (auth, permission, locator, perf\/timeout, app\/env) instead of staring at generic errors like \u201cTimeout 30s.\u201d<\/p>\n<h3 data-start=\"1519\" data-end=\"1584\"><strong>5) What are the best signals to detect auth\/session failures?<\/strong><\/h3>\n<p data-start=\"1585\" data-end=\"1768\">Use a signal map: final URL <code data-start=\"1613\" data-end=\"1621\">\/login<\/code> or <code data-start=\"1625\" data-end=\"1637\">returnUrl=<\/code>, redirect chain <code data-start=\"1654\" data-end=\"1668\">302 \u2192 \/login<\/code>, and 401 responses on key endpoints like <code data-start=\"1710\" data-end=\"1715\">\/me<\/code> or <code data-start=\"1719\" data-end=\"1729\">\/session<\/code>.<\/p>\n<h5><strong>We Also Provide Training In:<\/strong><\/h5>\n<ul>\n<li><a href=\"https:\/\/www.testleaf.com\/course\/selenium-automation-certification-training-course.html?utm_source=blog_post&amp;utm_medium=Organic&amp;utm_campaign=Blog_Post\"><strong>Advanced Selenium Training<\/strong><\/a><\/li>\n<li><a href=\"https:\/\/www.testleaf.com\/course\/playwright.html?utm_source=blog-post&amp;utm_medium=Organic&amp;utm_campaign=Blog_Post\"><strong>Playwright Training<\/strong><\/a><\/li>\n<li><a href=\"https:\/\/www.testleaf.com\/course\/genai-qa-engineers-training-course.html?utm_source=blog-post&amp;utm_medium=Organic&amp;utm_campaign=Blog_Post\"><strong>Gen AI Training<\/strong><\/a><\/li>\n<li><a href=\"https:\/\/www.testleaf.com\/course\/aws-cloud-architect-certification-training-course.html?utm_source=blog-post&amp;utm_medium=Organic&amp;utm_campaign=Blog_Post\"><strong>AWS Training<\/strong><\/a><\/li>\n<li><a href=\"https:\/\/www.testleaf.com\/course\/rest-api-testing-certification-training-course.html?utm_source=blog-post&amp;utm_medium=Organic&amp;utm_campaign=Blog_Post\"><strong>REST API Training<\/strong><\/a><\/li>\n<li><a href=\"https:\/\/www.testleaf.com\/course\/full-stack-developer-certification-training-course.html?utm_source=blog-post&amp;utm_medium=Organic&amp;utm_campaign=Blog_Post\"><strong>Full Stack Training<\/strong><\/a><\/li>\n<li><a href=\"https:\/\/www.testleaf.com\/course\/appium-mobile-automation-certification-training-course.html?utm_source=blog-post&amp;utm_medium=Organic&amp;utm_campaign=Blog_Post\"><strong>Appium Training<\/strong><\/a><\/li>\n<li><a href=\"https:\/\/www.testleaf.com\/course\/dev-ops-master-certification-training-course.html?utm_source=blog-post&amp;utm_medium=Organic&amp;utm_campaign=Blog_Post\"><strong>DevOps Training<\/strong><\/a><\/li>\n<li><a href=\"https:\/\/www.testleaf.com\/course\/apache-jmeter-testing-training-course.html?utm_source=blog-post&amp;utm_medium=Organic&amp;utm_campaign=Blog_Post\"><strong>JMeter Performance Training<\/strong><\/a><\/li>\n<\/ul>\n<h6><strong>Author\u2019s Bio<\/strong>:<\/h6>\n<p><img loading=\"lazy\" decoding=\"async\" class=\"wp-image-6744 size-full alignleft\" src=\"https:\/\/www.testleaf.com\/blog\/wp-content\/uploads\/2025\/09\/Kadhir.png\" sizes=\"(max-width: 200px) 100vw, 200px\" srcset=\"https:\/\/www.testleaf.com\/blog\/wp-content\/uploads\/2025\/09\/Kadhir.png 200w, https:\/\/www.testleaf.com\/blog\/wp-content\/uploads\/2025\/09\/Kadhir-150x150.png 150w, https:\/\/www.testleaf.com\/blog\/wp-content\/uploads\/2025\/09\/Kadhir-96x96.png 96w\" alt=\"Kadhir\" width=\"200\" height=\"200\" \/><\/p>\n<p>Content Writer at Testleaf, specializing in SEO-driven content for test automation, software development, and cybersecurity. I turn complex technical topics into clear, engaging stories that educate, inspire, and drive digital transformation.<\/p>\n<p><strong>Ezhirkadhir Raja<\/strong><\/p>\n<p>Content Writer \u2013 Testleaf<\/p>\n<p><a href=\"http:\/\/linkedin.com\/in\/ezhirkadhir\" target=\"_blank\" rel=\"noopener\"><img loading=\"lazy\" decoding=\"async\" src=\"https:\/\/www.testleaf.com\/blog\/wp-content\/uploads\/2025\/07\/linkedin.png\" alt=\"LinkedIn Logo\" width=\"28\" height=\"28\" \/><\/a><\/p>\n<p class=\"not-prose mt-0! mb-0! flex-auto truncate\">\n","protected":false},"excerpt":{"rendered":"<p>&nbsp; If you run Playwright in CI at scale, you\u2019ve seen this movie. \u2705 Test fails \u2705 You open the screenshot \u2705 Instead of the dashboard\u2026 you\u2019re staring at the login page And suddenly everyone wonders: \u201cDid we break prod?\u201d Most of the time, you didn\u2019t. What you hit is an auth\/session artifact\u2014a flaky failure &hellip;<\/p>\n<p class=\"read-more\"> <a class=\"\" href=\"https:\/\/www.testleaf.com\/blog\/playwright-ai-session-preservation-failure-triage-ci\/\"> <span class=\"screen-reader-text\">Playwright + AI: AI-Driven Session Preservation &#038; Failure Triage (Built for Real CI)<\/span> Read More &raquo;<\/a><\/p>\n","protected":false},"author":1,"featured_media":8751,"comment_status":"closed","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"_acf_changed":false,"om_disable_all_campaigns":false,"_monsterinsights_skip_tracking":false,"_monsterinsights_sitenote_active":false,"_monsterinsights_sitenote_note":"","_monsterinsights_sitenote_category":0,"site-sidebar-layout":"default","site-content-layout":"default","ast-main-header-display":"","ast-hfb-above-header-display":"","ast-hfb-below-header-display":"","ast-hfb-mobile-header-display":"","site-post-title":"","ast-breadcrumbs-content":"","ast-featured-img":"","footer-sml-layout":"","theme-transparent-header-meta":"default","adv-header-id-meta":"","stick-header-meta":"","header-above-stick-meta":"","header-main-stick-meta":"","header-below-stick-meta":"","footnotes":""},"categories":[345],"tags":[372,477,799,150,795],"class_list":["post-8748","post","type-post","status-publish","format-standard","has-post-thumbnail","hentry","category-playwright","tag-ai","tag-ai-testing","tag-ai-tools","tag-playwright","tag-playwright-automation-course"],"acf":[],"aioseo_notices":[],"amp_enabled":true,"_links":{"self":[{"href":"https:\/\/www.testleaf.com\/blog\/wp-json\/wp\/v2\/posts\/8748","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/www.testleaf.com\/blog\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/www.testleaf.com\/blog\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/www.testleaf.com\/blog\/wp-json\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/www.testleaf.com\/blog\/wp-json\/wp\/v2\/comments?post=8748"}],"version-history":[{"count":4,"href":"https:\/\/www.testleaf.com\/blog\/wp-json\/wp\/v2\/posts\/8748\/revisions"}],"predecessor-version":[{"id":8757,"href":"https:\/\/www.testleaf.com\/blog\/wp-json\/wp\/v2\/posts\/8748\/revisions\/8757"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/www.testleaf.com\/blog\/wp-json\/wp\/v2\/media\/8751"}],"wp:attachment":[{"href":"https:\/\/www.testleaf.com\/blog\/wp-json\/wp\/v2\/media?parent=8748"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/www.testleaf.com\/blog\/wp-json\/wp\/v2\/categories?post=8748"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/www.testleaf.com\/blog\/wp-json\/wp\/v2\/tags?post=8748"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}