starred/pi-skills
1
0
Fork 0
mirror of https://github.com/badlogic/pi-skills.git synced 2026-04-27 04:55:58 +03:00
Code Issues 13 Projects Releases Packages Wiki Activity

[PR #16] Add Playwright's ariaSnapshotForAI for improved LLM page understanding #18

New issue
Open
opened 2026-03-07 13:58:02 +03:00 by kerem · 0 comments
kerem commented 2026-03-07 13:58:02 +03:00
Owner
Copy link

Original Pull Request: https://github.com/badlogic/pi-skills/pull/16

State: open
Merged: No

This PR extracts Playwright's ariaSnapshotForAI functionality into a bundled dependency and integrates it into browser-page-structure.js.

Problem: LLMs were losing context on complex pages due to insufficient structural information from the basic page exploration script.

Solution:

  • Bundled Playwright's ariaSnapshotForAI source code into aria-snapshot-bundle.js (isolated from Playwright's full dependency tree)
  • Introduced browser-page-structure.js to use Playwright's ARIA snapshot generation
  • Added patch-from-playwright.ts script for semi-automated updates from the official Playwright repo

The new ARIA snapshot provides richer structural context including:

  • Full ARIA tree with roles and accessible names
  • Interactive element refs ([ref=e1], etc.)
  • Element states (checked, disabled, expanded, etc.)
  • Links with URLs for navigation

This significantly improves LLM comprehension of page structure and available actions on complex web applications.

I know this is a step up in complexity, but the gain in LLM accuracy is a major win for complex apps. I've added a patch script to keep maintenance low. If you decide this is more code than you want to take on right now, no worries—feel free to close the PR and we can look for a middle ground.

**Original Pull Request:** https://github.com/badlogic/pi-skills/pull/16 **State:** open **Merged:** No --- This PR extracts Playwright's `ariaSnapshotForAI` functionality into a bundled dependency and integrates it into `browser-page-structure.js`. **Problem:** LLMs were losing context on complex pages due to insufficient structural information from the basic page exploration script. Solution: - Bundled Playwright's `ariaSnapshotForAI` source code into `aria-snapshot-bundle.js` (isolated from Playwright's full dependency tree) - Introduced `browser-page-structure.js` to use Playwright's ARIA snapshot generation - Added `patch-from-playwright.ts` script for semi-automated updates from the official Playwright repo The new ARIA snapshot provides richer structural context including: - Full ARIA tree with roles and accessible names - Interactive element refs ([ref=e1], etc.) - Element states (checked, disabled, expanded, etc.) - Links with URLs for navigation This significantly improves LLM comprehension of page structure and available actions on complex web applications. I know this is a step up in complexity, but the gain in LLM accuracy is a major win for complex apps. I've added a patch script to keep maintenance low. If you decide this is more code than you want to take on right now, no worries—feel free to close the PR and we can look for a middle ground.
Sign in to join this conversation.
No Branch/Tag specified
Branches Tags
main
No results found.
Labels
Clear labels   
pull-request

Mirrored from GitHub Pull Request

No labels
pull-request
Milestone
Clear milestone
No items
No milestone
Projects
Clear projects
No items
No project
Assignees
Clear assignees
No assignees
1 participant
Notifications
Due date
The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set.

Reference
starred/pi-skills#18
No description provided.