zeekayandHanzo Dev
e2503dae65
LUX_ENV=local|devnet|testnet|mainnet is now the only environment knob. e2e/env.ts
resolves every base URL, RPC endpoint, network ID and chain ID from NETWORKS.yaml
at runtime; playwright.config.ts and helpers.ts derive from it instead of keeping
their own copies. Those copies had already drifted — the config resolved the
testnet explorer to test-explore.lux.network, which is NXDOMAIN, while helpers.ts
used the real explore-test.lux.network. The retired two-axis ENV/LUX_NETWORK
model now fails loudly rather than being half-honoured.
Projects are viewports (mobile 390x844, tablet 834x1194, laptop 1440x900), so
"mobile and desktop" is the same three frames every run. Screenshots go to
~/Desktop/lux-e2e/<env>/<surface>-<viewport>.png on pass, and shot-reporter.ts
copies a failing test's dying state next to them — a failure you cannot look at
is a failure you cannot fix.
50-login.spec.ts proves sign-in for real: lux.cloud -> lux.id -> /dashboard, with
the password from LUX_E2E_PASSWORD only. It refuses the cheap answers, because
each one lies here — /dashboard is an SPA that returns 200 and keeps its URL when
anonymous, and /auth/callback renders "Sign-in failed" at 200. The proof is the
token: a JWT in sessionStorage whose iss is the IdP and whose email is the
account typed, which lux.id/v1/iam/oauth/userinfo accepts (401 bare, 401 for a
junk token). That is the leg that breaks under invalid_client. It also asserts
the handoff is a public PKCE client and carries a negative control so the
positive assertions cannot go vacuous. Verified with a wrong password: red.
Measured while building this, not assumed:
- workers are per-env. Mainnet is CDN-fronted and ran the gate at 4 with
342/342; testnet hits the raw LB and lost 66 cases to throttling at 4 that
all passed at 2.
- `locator('text=/block/i').first()` in two specs resolved to a hidden Radix
menu trigger, so both tracked the nav's collapse rather than the blocks
table. Replaced by one shared assertion on the section AND a block link.
- the token check runs inside the page: page.request.get attaches request
headers to a thrown error, which printed a live bearer token into the log.
- 40-lux-financial carried a plaintext password; it now comes from the
environment.
Co-authored-by: Hanzo Dev <[email protected]>
57 lines
2.4 KiB
TypeScript
57 lines
2.4 KiB
TypeScript
import type { Reporter, TestCase, TestResult, FullResult } from '@playwright/test/reporter'
|
|
import { copyFileSync, mkdirSync } from 'fs'
|
|
import { join } from 'path'
|
|
import { env, envBanner, SHOT_DIR, viewportName } from './env'
|
|
|
|
// ---------------------------------------------------------------------------
|
|
// Failure evidence, routed where a human will actually look at it.
|
|
// ---------------------------------------------------------------------------
|
|
// Playwright's own failure screenshots land in e2e/test-results/<hashed-dir>/,
|
|
// which nobody opens. This reporter copies each one to
|
|
//
|
|
// ~/Desktop/lux-e2e/<env>/fail-<spec>-<test>-<viewport>.png
|
|
//
|
|
// alongside the deliberate snap() shots, so pass OR fail, the run leaves behind
|
|
// something you can look at. Two orthogonal concerns, no overlap: snap() is the
|
|
// frame a spec MEANT to capture, this is the state a test DIED in.
|
|
//
|
|
// It also prints the env banner at both ends of the run, so a log excerpt can
|
|
// never be mistaken for a different environment's result.
|
|
function slug(s: string): string {
|
|
return s.toLowerCase().replace(/[^a-z0-9]+/g, '-').replace(/^-|-$/g, '').slice(0, 70)
|
|
}
|
|
|
|
export default class ShotReporter implements Reporter {
|
|
private failures = 0
|
|
|
|
onBegin() {
|
|
mkdirSync(SHOT_DIR, { recursive: true })
|
|
console.log(`\n▶ ${envBanner()}`)
|
|
console.log(` shots → ${SHOT_DIR}\n`)
|
|
}
|
|
|
|
onTestEnd(test: TestCase, result: TestResult) {
|
|
if (result.status === 'passed' || result.status === 'skipped') return
|
|
this.failures++
|
|
const shot = result.attachments.find((a) => a.name === 'screenshot' && a.path)
|
|
if (!shot?.path) return
|
|
// Project name IS the viewport name (see playwright.config.ts projects).
|
|
const vp = test.parent.project()?.name
|
|
?? viewportName(null)
|
|
const spec = test.location.file.split('/').pop()?.replace(/\.spec\.ts$/, '') ?? 'spec'
|
|
const name = `fail-${slug(spec)}-${slug(test.title)}-${vp}.png`
|
|
try {
|
|
mkdirSync(SHOT_DIR, { recursive: true })
|
|
copyFileSync(shot.path, join(SHOT_DIR, name))
|
|
console.log(` ✗ evidence → ${join(SHOT_DIR, name)}`)
|
|
} catch (e) {
|
|
console.log(` ! could not copy failure screenshot: ${(e as Error).message}`)
|
|
}
|
|
}
|
|
|
|
onEnd(result: FullResult) {
|
|
console.log(`\n◀ ${result.status} — LUX_ENV=${env.name}, ${this.failures} failing`)
|
|
console.log(` shots → ${SHOT_DIR}\n`)
|
|
}
|
|
}
|