Selenium Tests: Passed Yesterday, Failed Today

Introduction: The QA Horror Story

Yesterday evening, I went home happy — my Selenium automation suite was green ✅. This morning, I ran the same tests, and boom 🚨 — half of them failed. The application hadn’t changed, the code hadn’t changed, yet the results were completely different.

If you’ve been in QA for a while, you know this nightmare. These are called flaky tests — tests that pass sometimes and fail other times without any real application bug. They waste time, confuse teams, and kill confidence in automation. Let me walk you through why this happens and how I debugged it step by step, so you don’t fall into the same trap.

What Are Flaky Tests? (In Simple Words)

Imagine a teacher giving the same question paper to students on two different days. One day they all pass, next day half of them fail — even though the paper didn’t change. That’s exactly what flaky tests are in Selenium. They’re unreliable, unpredictable, and make developers roll their eyes at automation reports.

Why Did My Tests Fail Randomly?

1. Timing Issues

   Sometimes the page loaded slowly. My test clicked a button before it was ready.

   Fix: Replace Thread.sleep with explicit waits.

2. Unstable Locators

   Yesterday, my XPath worked. Today, a tiny UI change broke it.

   Fix: Prefer stable IDs/CSS and use Page Object Model (POM).

3. Test Data Problems

   Script expected a fresh user. Today that user already existed.

   Fix: Create/clean test data in setup for every run.

4. Environment Differences

   Passed in local Chrome, failed on Jenkins Firefox.

   Fix: Standardize browsers and add cross-browser runs.

How I Debugged Step by Step

1. Re-run the failing test 2–3 times locally — if it sometimes passes, it’s flaky.
2. Add logs & screenshots — every failure tells a story.
3. Replace Thread.sleep with explicit waits (WebDriverWait).
4. Review and harden locators — avoid brittle absolute XPaths.
5. Isolate the test — run it alone to check hidden dependencies.
6. Compare environments — browser/OS/network can change behavior.

Sample Explicit Wait (Java)

WebDriverWait wait = new WebDriverWait(driver, Duration.ofSeconds(10));
WebElement username = wait.until(
    ExpectedConditions.visibilityOfElementLocated(By.id("username"))
);
username.sendKeys("qa_user");
    

Tools That Helped Me

• TestNG Retry Analyzer – auto-rerun failures to confirm flakiness.
• Allure / Extent Reports – visualize failure trends across builds.
• Jenkins / GitHub Actions – rerun plugins and build history.

How I Prevent Flaky Tests Now

• Use explicit waits everywhere; avoid sleeps.
• Prefer stable IDs/CSS and centralize in POM.
• Seed deterministic test data and/or mock external APIs.
• Run in parallel and across environments early.
• Treat test code like product code — reviews, lint, refactor.

Conclusion: Lesson Learned

A flaky test is worse than no test at all. After fixing waits, locators, and data setup, my flakiness dropped drastically — and the team regained confidence in automation. Next time your Selenium test passes today but fails tomorrow, don’t panic. Follow the steps above and you’ll catch the culprit.

FAQ