testing tools · AI reliability · open to work

Nancy Singh

A woman with long curly hair, rendered in liquid chrome against a starry sky

I build tools that catch flaky browser tests and AI answers that make things up.

I work in TypeScript and Python on the bugs that are annoying to pin down, like a test that fails 1 run in 5 or a RAG answer that isn't actually in the PDF. I also send fixes upstream: seven are merged: six into Google's Puppeteer and Chrome DevTools MCP, and one into Hugging Face's sentence-transformers.

click the metal to reshape it · drop
this page's network, live: 0 requests in flight quiet for 0ms BUSY

Selected work

Six projects, each with something you can open and try. Zeitaku and Junban are the two I'm most proud of.

~/zeitaku

Zeitaku

TypeScript · testing

A CLI that finds waitForTimeout sleeps in Playwright tests and replaces them with a wait that checks when the network is actually done (using the Chrome DevTools Protocol). I parse the test files with the TypeScript compiler so it doesn't touch comments or strings.

100%
fixed sleeps failed on Slow 3G
0%
Zeitaku failures, same test
240
sleeps found in 7 OSS repos
44
intentional ones it leaves alone
TS Compiler APICDPPlaywrightVitest + CI
~/junban

Junban

TypeScript · testing

Finds the exact order of API responses that makes a flaky Playwright test fail. It holds each response in the browser and lets them through in different orders, then shrinks the failure down to one rule like "fails when /api/cart arrives before /api/user" and writes a test that fails every time.

6/6
rare races caught by run 5
2/6
caught by 3,000 plain reruns
20/20
repro tests fail every run
0
false alarms on the bug-free page
CDP FetchPlaywrightDelta debuggingVitest + CI
~/halluguard

HalluGuard

Python · LLM evaluation

Checks if a RAG answer is backed by the retrieved text. My first version used cosine similarity, but it scored wrong answers high if they used the same words, so I switched to an NLI cross-encoder (DeBERTa-v3).

NLI
DeBERTa-v3 cross-encoder
RAGTruth
benchmark dataset
PyTorchHugging FaceNLI
~/kensa

Kensa

Python · RAG testing

Tests a RAG app without knowing the right answers. It changes the input in ways where we know what should happen: reword the question, add an unrelated document, delete the document with the answer, change a number in it. Then it checks the answer reacts correctly and shrinks each failure to the documents that cause it.

5
input changes, no LLM in the loop
3/3
planted bugs found in the demo bot
Metamorphic testingDelta debuggingHalluGuardpytest
~/docmind

DocMind

Python · RAG

Upload a PDF and ask questions about it, scanned PDFs too. It only runs OCR on pages that have no text, which made normal PDFs way faster. If nothing in the doc matches, it just says it doesn't know instead of asking the LLM.

96%
right chunk in the top 3 (hybrid search; 88% with embeddings alone)
83%
scanned-page questions found (50% before hybrid)
FAISSsentence-transformersTesseractLlama 3.3
~/attention-guard

Attention-Guard

Python · ML · desktop

My own idea: it watches which app and site you're on and whether you're typing or just scrolling, asks you every 20 minutes how focused you really are, and learns your personal "drifting" pattern from those answers. It nudges you when you drift and draws a daily focus report. Everything stays on your computer.

Pythonscikit-learnWindow-title categoriesExperience sampling

Open source

Fixes I sent to tools I use: six merged into Google projects, one into Hugging Face, and a bug I found in Microsoft's Playwright that their team fixed.

Google · puppeteer #15587 Merged Fail fast when a browser connection hangs Added a handshake timeout to puppeteer.connect(), so a host that accepts the connection but never answers no longer hangs forever. Tested against a server that never replies. Google · puppeteer #15590 Merged page.setGeolocation(null) removes the override The maintainer asked for this while reviewing my Chrome DevTools MCP PR. Works over both CDP and WebDriver BiDi, with tests and API docs. Google · chrome-devtools-mcp #3148 Merged Unhandled promise errors keep their message Rejections showed as a bare "Uncaught (in promise)" and different ones merged into "[3 times]". Now each shows its reason. Google · chrome-devtools-mcp #3144 Merged Clear error when Lighthouse can't audit a page Navigation mode failed with a bare INVALID_URL on file:// pages. Now it names the URL and points to the mode that works. Google · chrome-devtools-mcp #3198 Merged Redirect chains list only earlier hops Puppeteer shares one redirect array across hops, so early hops listed themselves and later ones. Each hop now shows only what came before it. Google · chrome-devtools-mcp #3199 Merged Console messages show real Map, Set and DOM values Console arguments like Maps, Sets, functions, DOM nodes, symbols and cyclic objects printed as {} or failed. They're now described from the browser's own previews. Merged after two review rounds. Hugging Face · sentence-transformers #4204 Merged Custom rank discounts in PListMLE got the wrong input A custom rank_discount_fn was called with a padding mask instead of rank positions, so there was no position discount and shorter lists added zero loss. Found by reading the code; merged by the maintainer.
Microsoft · playwright Bug found · fixed Codegen wrote tests that crash on ${ in the page Multiline aria snapshots escaped ${ twice, so the generated test threw a ReferenceError. I reported it with the root cause, and the Playwright team fixed it. My bug report #43459 ↗ · Their fix #43472 ↗
Documenso #3445 issue Redundant sleeps in the E2E suite Found with Zeitaku's AST scan, including which ones are intentional and should stay.

Something I didn't expect

I assumed every waitForTimeout should go. Then I read through real test suites (Documenso, Supabase, n8n and a few more) and about 1 in 6 were there on purpose. They wait before checking that something didn't happen, so removing them would make the test pass when it shouldn't.

So now Zeitaku looks at the next few lines after each sleep before changing it. Out of 240 sleeps it found, 196 were safe to replace and 44 it leaves alone.

zsh — zeitaku
// documenso: we must NOT navigate away
await page.waitForTimeout(1000);
await expect(page).not
  .toHaveURL(/\/admin\/users\/\d+$/);

$ zeitaku scan ./tests
Total waitForTimeout:   240
  replaceable:          196
  intentional:          44  (left alone)

About

Education
B.E. Computer Science · SLIET Punjab · class of 2026 · available full-time now
Open source
Puppeteer, Chrome DevTools MCP, sentence-transformers, Playwright · 7 merged (6 Google, 1 Hugging Face), 1 bug fixed in Playwright
Languages
TypeScript, Python, Kotlin
Tools
Playwright, CDP, PyTorch, Hugging Face, FAISS, LangChain
Based in
Delhi, India · open to remote and relocation

Thank you for coming here.

Want to talk?

I'm looking for my first engineering job, ideally at a startup working on testing, browser tools, dev tools or AI. Remote or relocation both work for me. Email me or DM me on X, I usually reply the same day.