Expectation Engine — Implementation Plan¶
Design Document: docs/designs/teltest/README.md §2 (Expectation Engine), §API Design (MUDClient API)
Priority: P0 — Core framework capability; all E2E assertions depend on this
Estimated Duration: 4 weeks across 6 sections (2.1–2.6)
Summary¶
This plan implements the TelTest Expectation Engine — the pattern-matching assertion layer that powers every expect() call in the E2E testing framework. The expectation engine is the primary mechanism by which tests verify server output, and every other TelTest component (fixtures, script DSL, session recording) depends on it.
The implementation adds:
Matchdataclass — frozen result object returned by successfulexpect()calls with matched text, full line, pattern, elapsed time, and buffer contextExpectTimeoutexception — richAssertionErrorsubclass with full buffer dump, transcript excerpt, and diagnostic hintsConnectionClosedexception — raised on server disconnect duringexpect()with buffer contextUnexpectedMatchexception — raised byexpect_not()when a forbidden pattern appears before the positive boundaryTranscriptEntrydataclass — timestamped record of send/recv/gmcp events for debugging- Expectation methods —
expect(),expect_prompt(),expect_line(),expect_sequence(),expect_any(),expect_not()with cursor-based matching, configurable timeouts, and timeout multiplier support
Key design decisions from the design doc:
- Cursor-based matching — Each
expect()andsend()advances a read cursor so assertions only match text received since the last interaction. This eliminates an entire class of flaky tests where stale output matches new patterns. (Design doc §2, post-review) from_start=Trueopt-in — Full-buffer scan is available but never the defaultexpect_not()requiresuntil— A positive boundary parameter avoids unconditional sleeps. All three adversarial reviewers flagged the original sleep-based design. (Design doc Appendix C, finding #4)- Multi-signal prompt detection —
expect_prompt()uses GA signal, regex matching, and quiescence detection to avoid TCP fragmentation race conditions. (Design doc §2, finding #6) - Timeout multiplier — All timeouts scale via
TELTEST_TIMEOUT_MULTIPLIERfor CI environments (Design doc §Configuration)
Existing infrastructure leveraged:
OutputBufferatpackages/teltest/src/teltest/buffer.py(Phase 1) — line splitting, prompt tracking, ring bufferTelnetProtocolatpackages/teltest/src/teltest/protocol.py(Phase 1) — IAC negotiation, GA signal detectionMUDClientatpackages/teltest/src/teltest/client.py(Phase 1) — connection lifecycle,send(), background reader task,_cursorposition,_new_dataevent
Prerequisites (must be complete before this phase):
- Phase 1 (MUDClient core) —
connect(),disconnect(),send(),send_raw(), background reader task,OutputBuffer,TelnetProtocol,ANSIStripper OutputBuffermust expose:get_text(from_pos) -> str,get_line_at(pos) -> str,current_length -> int,get_lines(start, count) -> list[str],has_unterminated_line -> bool,get_unterminated_line -> strMUDClientmust expose:_cursor: int,_new_data: asyncio.Event,_buffer: OutputBuffer,_connected: bool,_reader_error: Exception | None,_timeout: float
2.1 Data Models¶
Package:
teltest| Priority: P0 | Dependencies: Phase 1 (OutputBuffer)
2.1.1 Match Dataclass¶
- [ ] Create
Matchinpackages/teltest/src/teltest/expect.py:@dataclass(frozen=True) class Match: """Result of a successful expect() call.""" text: str # The matched text (substring or regex group 0) line: str # The full line containing the match pattern: str # String representation of the pattern that matched elapsed: float # Seconds waited before match was found buffer_before: str # Buffer context before the match (last N lines) text— For substring matches, the literal substring found. For regex matches, the full match (re.Match.group(0)).line— The complete line (delimited by\r\nor\n) containing the match. For multi-line regex matches, this is the first line of the match.pattern— String representation: forstrpatterns, the literal string; forre.Pattern, the.patternattribute.elapsed—time.monotonic()delta from when the expect call started to when the match was found.buffer_before— The last_CONTEXT_LINES(default 20) lines of buffer content preceding the match position, for debugging context.- [ ] Add module-level constant
_CONTEXT_LINES: int = 20controlling how many lines of contextbuffer_beforecaptures. - [ ] Implement
__str__onMatchreturning a human-readable summary:"Matched '{self.text}' on line '{self.line}' (pattern='{self.pattern}', elapsed={self.elapsed:.3f}s)"
2.1.2 TranscriptEntry Dataclass¶
- [ ] Create
TranscriptEntryinpackages/teltest/src/teltest/expect.py: timestampis absolutetime.monotonic()recorded at event time; display code computes relative offsets from the first entrytextforrecventries is post-ANSI-stripping (if stripping is enabled) and post-IAC-filteringtextforgmcp_send/gmcp_recvis the JSON-serialized GMCP payload prefixed with the package name:"package.name {json}"
2.1.3 Transcript Recorder¶
- [ ] Create
Transcriptclass inpackages/teltest/src/teltest/expect.py:class Transcript: """Ring-buffer transcript of session events.""" def __init__(self, max_entries: int = 10_000) -> None: ... def record_send(self, text: str) -> None: ... def record_recv(self, text: str) -> None: ... def record_gmcp_send(self, package: str, data: dict[str, Any]) -> None: ... def record_gmcp_recv(self, package: str, data: dict[str, Any]) -> None: ... @property def entries(self) -> list[TranscriptEntry]: ... def recent(self, n: int = 5) -> list[TranscriptEntry]: ... def format_recent(self, n: int = 5, relative_to: float | None = None) -> str: ... - Uses
collections.deque(maxlen=max_entries)for O(1) append with automatic eviction record_send()creates aTranscriptEntry(timestamp=time.monotonic(), direction="send", text=text)record_recv()creates aTranscriptEntry(timestamp=time.monotonic(), direction="recv", text=text)record_gmcp_send()/record_gmcp_recv()serialize the data dict to JSON and prefix with package nameentriesproperty returns a list copy of the deque (newest last)recent(n)returns the lastnentriesformat_recent(n, relative_to)returns a formatted string for error messages:If[0.00s] RECV: "Character Summary" [0.01s] RECV: "Create this character?" [0.50s] SEND: "Y" [0.52s] RECV: "Character created successfully."relative_tois None, uses the timestamp of the first entry in the returned set as the base.
2.1.4 Unit Tests — Data Models¶
- [ ] Write tests in
packages/teltest/tests/test_expect.py: - [ ]
test_match_frozen— VerifyMatchis immutable (assigning tomatch.textraisesFrozenInstanceError) - [ ]
test_match_str_representation— Verify__str__output format - [ ]
test_match_fields— Verify all fields are accessible and correctly typed - [ ]
test_match_buffer_before_content— Verifybuffer_beforecontains context lines - [ ]
test_transcript_entry_frozen— VerifyTranscriptEntryis immutable - [ ]
test_transcript_entry_directions— Verify all four direction literals are valid - [ ]
test_transcript_record_send— Record a send, verify entry fields - [ ]
test_transcript_record_recv— Record a recv, verify entry fields - [ ]
test_transcript_record_gmcp_send— Record GMCP send, verify package+JSON text format - [ ]
test_transcript_record_gmcp_recv— Record GMCP recv, verify package+JSON text format - [ ]
test_transcript_ring_buffer_eviction— Create transcript withmax_entries=3, record 5 entries, verify only last 3 remain - [ ]
test_transcript_recent_returns_n_entries— Record 10 entries,recent(3)returns last 3 - [ ]
test_transcript_recent_returns_all_if_fewer— Record 2 entries,recent(5)returns both - [ ]
test_transcript_format_recent_output— Verify formatted string matches expected pattern with timestamps and direction labels - [ ]
test_transcript_format_recent_relative_timestamps— Verify timestamps are relative to base
2.2 Exception Classes¶
Package:
teltest| Priority: P0 | Dependencies: 2.1
2.2.1 ExpectTimeout¶
- [ ] Create
ExpectTimeoutinpackages/teltest/src/teltest/expect.py:class ExpectTimeout(AssertionError): """Raised when expect() times out. Provides rich error formatting with full buffer dump, transcript excerpt, and diagnostic hints. """ def __init__( self, pattern: str, timeout: float, buffer: str, transcript: Transcript | None = None, hint: str | None = None, ) -> None: ... @property def pattern(self) -> str: ... @property def timeout(self) -> float: ... @property def buffer(self) -> str: ... - Extends
AssertionErrorso pytest treats it as a test assertion failure (not an unexpected exception) __init__storespattern,timeout,bufferas instance attributes__str__produces the rich error format from the design doc §Error Handling:teltest.ExpectTimeout: Pattern not found within {timeout}s Expected: "{pattern}" Received (last 20 lines): ───────────────────────── | {line 1} | {line 2} | ... ───────────────────────── Transcript (last 5 interactions): [0.00s] RECV: "..." [0.50s] SEND: "..." [5.00s] TIMEOUT waiting for "{pattern}" Hint: {hint or default hint}- Default hint:
"The server may be waiting for additional input. Check if a prompt was expected before this step." - If
transcriptis None, the "Transcript" section is omitted - Buffer display shows the last 20 lines of the
bufferparameter, each prefixed with| - The buffer section uses Unicode box-drawing characters (
─) for visual separation
2.2.2 ConnectionClosed¶
- [ ] Create
ConnectionClosedinpackages/teltest/src/teltest/expect.py: __init__storesbuffer; sets defaultmessageto"Server closed the connection"__str__produces:- Does NOT extend
AssertionError— connection closure is an infrastructure failure, not a test assertion
2.2.3 UnexpectedMatch¶
- [ ] Create
UnexpectedMatchinpackages/teltest/src/teltest/expect.py:class UnexpectedMatch(AssertionError): """Raised by expect_not() when the forbidden pattern appears. Attributes: pattern: The pattern that should not have appeared. match: The Match object showing where it appeared. until: The boundary pattern that was being waited for. """ def __init__( self, pattern: str, match: Match, until: str, ) -> None: ... @property def pattern(self) -> str: ... @property def match(self) -> Match: ... @property def until(self) -> str: ... - Extends
AssertionError— this is a test assertion failure (the negative expectation was violated) __str__produces:
2.2.4 Unit Tests — Exceptions¶
- [ ] Write tests in
packages/teltest/tests/test_expect.py: - [ ]
test_expect_timeout_is_assertion_error—assert issubclass(ExpectTimeout, AssertionError) - [ ]
test_expect_timeout_attributes— Verify.pattern,.timeout,.bufferstored correctly - [ ]
test_expect_timeout_str_format— Verifystr(exc)contains pattern, timeout, buffer lines, hint - [ ]
test_expect_timeout_str_with_transcript— Verify transcript section appears when transcript is provided - [ ]
test_expect_timeout_str_without_transcript— Verify transcript section is omitted when transcript is None - [ ]
test_expect_timeout_str_with_custom_hint— Verify custom hint replaces default - [ ]
test_expect_timeout_buffer_truncated_to_20_lines— Provide 50-line buffer, verify only last 20 displayed - [ ]
test_connection_closed_is_not_assertion_error—assert not issubclass(ConnectionClosed, AssertionError) - [ ]
test_connection_closed_attributes— Verify.bufferstored correctly - [ ]
test_connection_closed_str_format— Verifystr(exc)contains buffer lines and hint - [ ]
test_connection_closed_custom_message— Verify custom message overrides default - [ ]
test_unexpected_match_is_assertion_error—assert issubclass(UnexpectedMatch, AssertionError) - [ ]
test_unexpected_match_attributes— Verify.pattern,.match,.untilstored correctly - [ ]
test_unexpected_match_str_format— Verifystr(exc)contains forbidden pattern, boundary, matched text, context
2.3 Timeout Scaling¶
Package:
teltest| Priority: P0 | Dependencies: none
2.3.1 Timeout Multiplier Helper¶
- [ ] Create
_resolve_timeouthelper function inpackages/teltest/src/teltest/expect.py:_TIMEOUT_MULTIPLIER: float | None = None def _get_timeout_multiplier() -> float: """Read TELTEST_TIMEOUT_MULTIPLIER from environment (cached).""" global _TIMEOUT_MULTIPLIER if _TIMEOUT_MULTIPLIER is None: _TIMEOUT_MULTIPLIER = float(os.environ.get("TELTEST_TIMEOUT_MULTIPLIER", "1.0")) return _TIMEOUT_MULTIPLIER def _resolve_timeout(explicit: float | None, default: float) -> float: """Resolve an explicit or default timeout, applying the CI multiplier. Args: explicit: Timeout passed by the caller (None means use default). default: The MUDClient's default timeout. Returns: Final timeout in seconds, scaled by TELTEST_TIMEOUT_MULTIPLIER. """ base = explicit if explicit is not None else default return base * _get_timeout_multiplier() - The multiplier is read once from
TELTEST_TIMEOUT_MULTIPLIERand cached in a module global - Default multiplier is
1.0(no scaling) - CI environments set
TELTEST_TIMEOUT_MULTIPLIER=2.0per the design doc §CI/CD
2.3.2 Unit Tests — Timeout Scaling¶
- [ ] Write tests in
packages/teltest/tests/test_expect.py: - [ ]
test_resolve_timeout_explicit_value—_resolve_timeout(3.0, 5.0)returns3.0(no multiplier set) - [ ]
test_resolve_timeout_uses_default—_resolve_timeout(None, 5.0)returns5.0 - [ ]
test_resolve_timeout_with_multiplier— SetTELTEST_TIMEOUT_MULTIPLIER=2.0in env, verify_resolve_timeout(3.0, 5.0)returns6.0 - [ ]
test_resolve_timeout_default_with_multiplier— Set multiplier to2.0, verify_resolve_timeout(None, 5.0)returns10.0 - [ ]
test_resolve_timeout_multiplier_default_is_1— Unset env var, verify multiplier is1.0 - Use
monkeypatch.setenv/monkeypatch.delenvfor environment variable isolation - Reset
_TIMEOUT_MULTIPLIER = Nonein fixture teardown to clear the cache between tests
2.4 Core Expect Engine¶
Package:
teltest| Priority: P0 | Dependencies: 2.1, 2.2, 2.3, Phase 1 (MUDClient, OutputBuffer)
The expectation engine is implemented as methods on MUDClient. The internal matching logic lives in packages/teltest/src/teltest/expect.py as free functions, and MUDClient delegates to them.
2.4.1 Internal Matching Helpers¶
- [ ] Create
_match_patternfunction inpackages/teltest/src/teltest/expect.py: - For
strpatterns: usetext.find(pattern, start_pos). Return(idx, idx + len(pattern), pattern). - For
re.Patternpatterns: usepattern.search(text, pos=start_pos). Return(m.start(), m.end(), m.group(0)). -
Returns
Noneif no match found. -
[ ] Create
_extract_linefunction inpackages/teltest/src/teltest/expect.py: - Scans backward from
posto find\n(or start of string) - Scans forward from
posto find\n(or end of string) -
Returns the text between those boundaries, stripped of
\r\n -
[ ] Create
_extract_contextfunction inpackages/teltest/src/teltest/expect.py: - Splits
text[:pos]by\n, takes the lastcontext_lineslines, joins with\n
2.4.2 expect() Method¶
- [ ] Implement
expect()onMUDClientinpackages/teltest/src/teltest/client.py: -
Algorithm:
- Compute effective timeout:
effective = _resolve_timeout(timeout, self._timeout) - Record start time:
start = time.monotonic() - Compute deadline:
deadline = start + effective - Determine search start position:
- If
from_start=True:search_from = 0 - Else:
search_from = self._cursor - Scan loop:
a. Check for reader error: if
self._reader_erroris not None, raiseConnectionClosed(buffer=self._buffer.get_text(self._cursor))b. Check for disconnection: if notself._connectedand no reader error, raiseConnectionClosed(buffer=self._buffer.get_text(self._cursor))c. Get current buffer text:text = self._buffer.get_text(search_from)d. Attempt match:result = _match_pattern(text, pattern)e. If match found:- Compute absolute positions:
abs_start = search_from + result[0],abs_end = search_from + result[1] - If not
from_start: advance cursor past match:self._cursor = abs_end - Extract line:
line = _extract_line(self._buffer.get_all_text(), abs_start) - Extract context:
context = _extract_context(self._buffer.get_all_text(), abs_start) - Compute elapsed:
elapsed = time.monotonic() - start - Return
Match(text=result[2], line=line, pattern=_pattern_str(pattern), elapsed=elapsed, buffer_before=context)f. If no match and deadline exceeded: - Raise
ExpectTimeout(pattern=_pattern_str(pattern), timeout=effective, buffer=self._buffer.get_text(self._cursor), transcript=self._transcript)g. Wait for new data with remaining timeout:remaining = deadline - time.monotonic() self._new_data.clear()await asyncio.wait_for(self._new_data.wait(), timeout=max(0.01, remaining))- On
asyncio.TimeoutError: continue loop (will hit deadline check on next iteration)
- Compute absolute positions:
- The
_new_dataevent is set by the background reader whenever new data arrives in the buffer
- Compute effective timeout:
-
[ ] Add
_pattern_strhelper:
2.4.3 expect_prompt() Method¶
- [ ] Implement
expect_prompt()onMUDClientinpackages/teltest/src/teltest/client.py: - Multi-signal detection algorithm (per design doc §2):
- Compute effective timeout and deadline
- Scan loop:
a. Signal 1 — Telnet GA: Check if
self._protocol.ga_receivedflag is set since the last cursor advance. If GA received and buffer has unterminated line matchingprompt, match immediately. Clear the GA flag. b. Signal 2 — Prompt regex: If buffer has an unterminated line (no trailing\r\n), match it againstprompt(substring forstr, search forre.Pattern). If match found, this is a candidate — proceed to quiescence check. c. Signal 3 — Quiescence: If candidate found, waitself._prompt_settle_time(default 0.3s). If no new data arrives during that window, accept the match. d. If no candidate: wait for new data (same asexpect()) e. On timeout: raiseExpectTimeout - On match: advance cursor past the prompt, record in transcript, return the matched prompt text
- Add
_prompt_settle_time: float = 0.3attribute toMUDClient.__init__(configurable) - Add
prompt_settle_timeparameter toMUDClient.__init__constructor
2.4.4 expect_line() Method¶
- [ ] Implement
expect_line()onMUDClientinpackages/teltest/src/teltest/client.py: - Algorithm:
- Compute effective timeout and deadline
- Scan loop:
a. Get buffer text from cursor:
buf = self._buffer.get_text(self._cursor)b. Split into complete lines (lines terminated by\r\nor\n) c. For each complete line, strip trailing whitespace and compare withtext.strip()d. If exact match found:- Advance cursor past the line (including the newline delimiter)
- Return the matched line text
e. On timeout: raise
ExpectTimeout
- Wait for new data between scan attempts
2.4.5 expect_sequence() Method¶
- [ ] Implement
expect_sequence()onMUDClientinpackages/teltest/src/teltest/client.py: - Timeout semantics (per design doc §2, finding #11):
- If
per_step_timeoutis provided: each pattern getsper_step_timeoutseconds (total time is unbounded, up tolen(patterns) * per_step_timeout) - If only
timeoutis provided (default):timeoutis the total time for all patterns combined. Each individual step gets at most the remaining time. - If neither is provided: use
self._timeoutas the total timeout.
- If
- Algorithm:
- Resolve total timeout:
total = _resolve_timeout(timeout, self._timeout) - Record start time and compute global deadline:
deadline = start + total matches: list[Match] = []- For each
patterninpatterns: a. Ifper_step_timeoutis not None:step_timeout = _resolve_timeout(per_step_timeout, per_step_timeout)(apply multiplier) b. Else:remaining = deadline - time.monotonic()- If
remaining <= 0: raiseExpectTimeoutwith info about which step failed (index, pattern) step_timeout = remainingc. Try:match = await self.expect(pattern, timeout=step_timeout)— Note: pass the rawstep_timeoutvalue; do NOT letexpect()apply the multiplier again. Use an internal_expect_raw()or pass a sentinel to skip double-multiplication. d. Appendmatchtomatches
- Return
matches
- Resolve total timeout:
- Important: To avoid double-applying the timeout multiplier, implement an internal
_expect_impl()that takes pre-resolved timeouts, and have bothexpect()andexpect_sequence()call it.
2.4.6 expect_any() Method¶
- [ ] Implement
expect_any()onMUDClientinpackages/teltest/src/teltest/client.py: - Algorithm:
- Compute effective timeout and deadline
- Scan loop:
a. Get buffer text from cursor:
text = self._buffer.get_text(self._cursor)b. For each(index, pattern)inenumerate(patterns):result = _match_pattern(text, pattern)- If match found: record match position as
(index, abs_start, result)c. If one or more patterns matched: - Select the match with the earliest position in the buffer (lowest
abs_start). If two patterns match at the same position, prefer the one with the lower index. - Advance cursor past the winning match
- Build and return
(winning_index, Match(...))d. On timeout: raiseExpectTimeout— list all patterns in the error message e. Wait for new data
- Design decision: "First match in buffer" (positional) rather than "first match detected" (temporal) ensures deterministic behavior regardless of data arrival timing.
2.4.7 expect_not() Method¶
- [ ] Implement
expect_not()onMUDClientinpackages/teltest/src/teltest/client.py: - Algorithm (per design doc §2, finding #4):
- Compute effective timeout and deadline
- Record cursor position at entry:
entry_cursor = self._cursor - Scan loop:
a. Get buffer text from cursor:
text = self._buffer.get_text(self._cursor)b. Check for forbidden pattern:forbidden = _match_pattern(text, pattern)c. Check for boundary pattern:boundary = _match_pattern(text, until)d. Case 1: Both found — compare positions:- If
forbiddenstarts beforeboundary: raiseUnexpectedMatch(pattern=_pattern_str(pattern), match=<Match from forbidden>, until=_pattern_str(until)) - If
boundarystarts before or atforbidden: boundary wins — advance cursor past boundary, returnNone(success) e. Case 2: Only forbidden found (no boundary yet): - Raise
UnexpectedMatchf. Case 3: Only boundary found (no forbidden): - Advance cursor past boundary, return
None(success — forbidden pattern never appeared) g. Case 4: Neither found: - On timeout: raise
ExpectTimeoutwith pattern set to theuntilboundary pattern (we timed out waiting for the boundary, not the forbidden pattern) - Wait for new data
- If
- Important:
expect_not()does NOT advance the cursor incrementally during scanning — it preserves the entry cursor until resolution. This ensures the forbidden pattern check covers ALL text between the entry point and the boundary, not just new increments. - Returns
Noneon success (the forbidden pattern was not found before the boundary)
2.4.8 Cursor Management Integration¶
- [ ] Ensure
MUDClient.send()advances the cursor: - Per design doc §2: "send() advances the cursor to the current buffer end"
-
This ensures
expect()aftersend()only matches NEW output -
[ ] Ensure
MUDClient.clear_buffer()resets cursor: -
[ ] Add
outputproperty toMUDClient: -
[ ] Add
recentproperty toMUDClient: -
[ ] Add
transcriptproperty toMUDClient:
2.4.9 Transcript Integration with Reader¶
-
[ ] Modify the background reader task in
MUDClient(Phase 1 code inclient.py) to record received text: -
[ ] Initialize
self._transcript = Transcript(max_entries=10_000)inMUDClient.__init__
2.5 Public API Exports¶
Package:
teltest| Priority: P0 | Dependencies: 2.1–2.4
-
[ ] Update
packages/teltest/src/teltest/__init__.pyto export: -
[ ] Verify all public types have Google-style docstrings
- [ ] Verify all public method signatures have full type annotations (MyPy strict)
2.6 Unit Tests — Expectation Methods¶
Package:
teltest| Priority: P0 | Dependencies: 2.4
All tests use a mock OutputBuffer and mock transport to avoid real network I/O. The test helper FakeMUDClient wraps a MUDClient with an in-process buffer that can be fed data programmatically.
2.6.1 Test Infrastructure¶
- [ ] Create test helper in
packages/teltest/tests/conftest.py:class FakeTransport: """Mock asyncio transport for testing without network I/O.""" def __init__(self) -> None: self.sent: list[bytes] = [] self.closed: bool = False def write(self, data: bytes) -> None: self.sent.append(data) def close(self) -> None: self.closed = True def is_closing(self) -> bool: return self.closed class BufferFeeder: """Feeds data into a MUDClient's buffer as if received from the network. Usage: feeder = BufferFeeder(client) feeder.feed("Welcome to the MUD\r\n") feeder.feed("Your choice: ") match = await client.expect("Welcome") """ def __init__(self, client: MUDClient) -> None: ... def feed(self, text: str) -> None: """Append text to the client's output buffer and signal new data.""" def feed_after(self, text: str, delay: float) -> asyncio.Task: """Schedule text to be fed after a delay (seconds).""" def close_connection(self) -> None: """Simulate server closing the connection.""" feed()directly appends toclient._buffer, increments buffer length, and setsclient._new_dataeventfeed_after()creates anasyncio.Taskthat sleeps then callsfeed()-
close_connection()setsclient._connected = Falseand setsclient._new_dataevent to unblock waiting expects -
[ ] Create
make_clientfixture inpackages/teltest/tests/conftest.py:
2.6.2 expect() Tests¶
- [ ] Write tests in
packages/teltest/tests/test_expect.py:
Success cases:
- [ ] test_expect_substring_match — Feed "Welcome to the MUD\r\n", expect("Welcome") returns Match with text="Welcome", line="Welcome to the MUD"
- [ ] test_expect_regex_match — Feed "Level 42 Warrior\r\n", expect(re.compile(r"Level \d+")) returns Match with text="Level 42"
- [ ] test_expect_regex_match_group — Feed "HP: 100/100\r\n", expect(re.compile(r"HP: (\d+)/(\d+)")) returns Match with text="HP: 100/100"
- [ ] test_expect_match_elapsed_time — Feed data after 0.1s delay, verify match.elapsed >= 0.1
- [ ] test_expect_match_buffer_before — Feed 5 lines then target, verify match.buffer_before contains preceding lines
- [ ] test_expect_match_pattern_string — Verify match.pattern is the string representation of the pattern used
- [ ] test_expect_immediate_match — Feed data before calling expect, verify immediate return with elapsed ≈ 0
Cursor behavior:
- [ ] test_expect_advances_cursor — Feed "AAA\r\nBBB\r\n", expect("AAA"), then expect("BBB") succeeds (cursor moved past AAA)
- [ ] test_expect_does_not_match_before_cursor — Feed "AAA\r\nBBB\r\n", expect("AAA"), then expect("AAA", timeout=0.1) raises ExpectTimeout (cursor is past first AAA)
- [ ] test_send_advances_cursor — Feed "AAA\r\n", call send("cmd"), feed "BBB\r\n", expect("AAA", timeout=0.1) raises ExpectTimeout (send moved cursor past AAA)
- [ ] test_expect_from_start_ignores_cursor — Feed "AAA\r\nBBB\r\n", expect("AAA") (moves cursor), then expect("AAA", from_start=True) succeeds
- [ ] test_expect_from_start_does_not_advance_cursor — Feed "AAA\r\nBBB\r\n", call expect("AAA", from_start=True), then expect("AAA") still works (cursor unchanged by from_start)
- [ ] test_clear_buffer_resets_cursor — Feed "AAA\r\n", call clear_buffer(), feed "BBB\r\n", expect("AAA", timeout=0.1) raises ExpectTimeout
Timeout cases:
- [ ] test_expect_timeout_raises — Call expect("NEVER", timeout=0.1) with no data, verify ExpectTimeout raised
- [ ] test_expect_timeout_exception_attributes — Verify raised exception has correct .pattern, .timeout, .buffer
- [ ] test_expect_timeout_exception_has_transcript — Verify raised exception includes transcript entries
- [ ] test_expect_timeout_default_timeout — Create client with timeout=0.2, call expect("NEVER") without explicit timeout, verify times out at ~0.2s
- [ ] test_expect_timeout_multiplier — Set TELTEST_TIMEOUT_MULTIPLIER=2.0, create client with timeout=0.1, call expect("NEVER"), verify times out at ~0.2s
Partial match / data arrival:
- [ ] test_expect_waits_for_complete_data — Feed "Wel" immediately, feed "come\r\n" after 0.1s, expect("Welcome") succeeds after ~0.1s
- [ ] test_expect_matches_across_chunks — Feed "Hello " then "World\r\n" in separate chunks, expect("Hello World") succeeds
Connection closed:
- [ ] test_expect_connection_closed_raises — Close connection during expect, verify ConnectionClosed raised
- [ ] test_expect_connection_closed_has_buffer — Verify ConnectionClosed.buffer contains text received before close
- [ ] test_expect_reader_error_raises_connection_closed — Set _reader_error to an exception, call expect(), verify ConnectionClosed raised
2.6.3 expect_prompt() Tests¶
- [ ] Write tests in
packages/teltest/tests/test_expect.py: - [ ]
test_expect_prompt_default_pattern— Feed"Room description\r\n> ",expect_prompt()returns"> " - [ ]
test_expect_prompt_custom_pattern— Feed"Room\r\nwarrior> ",expect_prompt("warrior> ")returns"warrior> " - [ ]
test_expect_prompt_regex_pattern— Feed"Room\r\nmage> ",expect_prompt(re.compile(r"\w+> "))returns"mage> " - [ ]
test_expect_prompt_ga_signal— Set GA received flag, feed"Enter name: ",expect_prompt("Enter name: ")matches immediately without quiescence wait - [ ]
test_expect_prompt_quiescence_detection— Feed"Some prompt: "(no\r\n), no GA signal, verify prompt detected afterprompt_settle_timeelapses - [ ]
test_expect_prompt_not_triggered_by_partial_line— Feed"Partial", then feed" line\r\n"before settle time, verify no false prompt detection - [ ]
test_expect_prompt_advances_cursor— After prompt match, verify cursor is past the prompt - [ ]
test_expect_prompt_timeout— No prompt arrives, verifyExpectTimeoutraised - [ ]
test_expect_prompt_custom_settle_time— Create client withprompt_settle_time=0.1, verify quiescence detection uses 0.1s
2.6.4 expect_line() Tests¶
- [ ] Write tests in
packages/teltest/tests/test_expect.py: - [ ]
test_expect_line_exact_match— Feed"Hello World\r\n",expect_line("Hello World")returns"Hello World" - [ ]
test_expect_line_strips_whitespace— Feed"Hello World \r\n",expect_line("Hello World")matches (trailing whitespace stripped) - [ ]
test_expect_line_no_partial_match— Feed"Hello World Extra\r\n",expect_line("Hello World", timeout=0.1)raisesExpectTimeout(must be exact) - [ ]
test_expect_line_no_substring_match— Feed"Hello World\r\n",expect_line("Hello", timeout=0.1)raisesExpectTimeout(must match full line) - [ ]
test_expect_line_skips_non_matching_lines— Feed"Line 1\r\nTarget Line\r\nLine 3\r\n",expect_line("Target Line")matches second line - [ ]
test_expect_line_advances_cursor— After match, subsequentexpect_line("Target Line", timeout=0.1)raisesExpectTimeout - [ ]
test_expect_line_timeout— No matching line arrives, verifyExpectTimeout - [ ]
test_expect_line_waits_for_newline— Feed"Hello World"(no\r\n), then feed"\r\n"after delay, verify matches only after newline arrives - [ ]
test_expect_line_cursor_behavior— Feed"A\r\nB\r\nC\r\n",expect_line("A"), thenexpect_line("B")succeeds
2.6.5 expect_sequence() Tests¶
- [ ] Write tests in
packages/teltest/tests/test_expect.py: - [ ]
test_expect_sequence_all_match— Feed"Name: \r\nRace: \r\nClass: \r\n",expect_sequence(["Name:", "Race:", "Class:"])returns list of 3 Match objects - [ ]
test_expect_sequence_returns_correct_matches— Verify each Match in the returned list has the correct.textand.pattern - [ ]
test_expect_sequence_order_matters— Feed"Class: \r\nName: \r\n",expect_sequence(["Name:", "Class:"])— only"Name:"is found (after cursor),"Class:"was already consumed or before cursor - [ ]
test_expect_sequence_total_timeout— Feed first pattern, delay beyond total timeout for second, verifyExpectTimeoutraised - [ ]
test_expect_sequence_per_step_timeout— Setper_step_timeout=0.1, feed first pattern immediately, delay second pattern 0.2s, verifyExpectTimeouton second step - [ ]
test_expect_sequence_total_timeout_shared— Settimeout=0.5, feed 3 patterns each taking 0.1s, verify all match within 0.5s budget - [ ]
test_expect_sequence_cursor_advances_through— After sequence completes, cursor is past all matched patterns - [ ]
test_expect_sequence_empty_patterns_returns_empty—expect_sequence([])returns[]immediately - [ ]
test_expect_sequence_partial_failure_reports_step— Second of three patterns times out; verify error message indicates which step failed - [ ]
test_expect_sequence_with_regex_patterns— Mix of string and regex patterns all match correctly - [ ]
test_expect_sequence_no_double_timeout_multiplier— SetTELTEST_TIMEOUT_MULTIPLIER=2.0andtimeout=1.0, verify total timeout is 2.0s (not 4.0s from double multiplication)
2.6.6 expect_any() Tests¶
- [ ] Write tests in
packages/teltest/tests/test_expect.py: - [ ]
test_expect_any_first_pattern_matches— Feed"Yes\r\n",expect_any(["Yes", "No"])returns(0, Match(text="Yes")) - [ ]
test_expect_any_second_pattern_matches— Feed"No\r\n",expect_any(["Yes", "No"])returns(1, Match(text="No")) - [ ]
test_expect_any_returns_earliest_position— Feed"No then Yes\r\n",expect_any(["Yes", "No"])returns(1, Match(text="No"))because "No" appears first positionally - [ ]
test_expect_any_same_position_prefers_lower_index— Feed"AB\r\n"where patterns["AB", "A"]both match at pos 0, returns(0, Match(text="AB")) - [ ]
test_expect_any_with_regex— Feed"HP: 42\r\n",expect_any([re.compile(r"HP: \d+"), "MP:"])returns(0, Match(text="HP: 42")) - [ ]
test_expect_any_advances_cursor— After match, verify cursor is past the matched text - [ ]
test_expect_any_timeout— None of the patterns appear, verifyExpectTimeoutwith all patterns listed - [ ]
test_expect_any_waits_for_data— Feed matching data after 0.1s delay, verify it waits and returns - [ ]
test_expect_any_single_pattern—expect_any(["Only"])with matching data works likeexpect("Only")
2.6.7 expect_not() Tests¶
- [ ] Write tests in
packages/teltest/tests/test_expect.py:
Success cases (forbidden pattern NOT found):
- [ ] test_expect_not_success — Feed "Room description\r\n> ", expect_not("Error", until="> ") returns None
- [ ] test_expect_not_boundary_found_first — Feed "Good stuff\r\n> ", verify returns None
- [ ] test_expect_not_advances_cursor_past_boundary — After success, cursor is past the until match
Failure cases (forbidden pattern found):
- [ ] test_expect_not_forbidden_found_before_boundary — Feed "Error: bad\r\n> ", expect_not("Error", until="> ") raises UnexpectedMatch
- [ ] test_expect_not_unexpected_match_attributes — Verify raised exception has correct .pattern, .match, .until
- [ ] test_expect_not_forbidden_found_no_boundary — Feed "Error occurred\r\n" only, expect_not("Error", until="> ") raises UnexpectedMatch
Timeout cases:
- [ ] test_expect_not_timeout_waiting_for_boundary — Feed "Stuff\r\n" (no boundary, no forbidden), expect_not("Error", until="> ", timeout=0.1) raises ExpectTimeout for the boundary pattern
- [ ] test_expect_not_timeout_message_references_boundary — Verify the ExpectTimeout.pattern is the until pattern, not the forbidden pattern
Edge cases:
- [ ] test_expect_not_with_regex_patterns — Both pattern and until are re.Pattern, verify correct behavior
- [ ] test_expect_not_forbidden_and_boundary_overlap — Feed "> Error >", where forbidden="Error" and until=">", verify boundary at position 0 wins (boundary found first)
- [ ] test_expect_not_cursor_preserved_during_scan — Verify the cursor does not advance incrementally while scanning; it only advances to the boundary match position on success
2.6.8 output / recent / clear_buffer() / transcript Tests¶
- [ ] Write tests in
packages/teltest/tests/test_expect.py: - [ ]
test_output_returns_all_text— Feed multiple chunks, verifyclient.outputcontains all text - [ ]
test_recent_returns_text_since_cursor— Feed text, callsend(), feed more, verifyclient.recentonly shows post-send text - [ ]
test_recent_after_expect— Feed"A\r\nB\r\n",expect("A"), verifyclient.recentis"B\r\n"(text after match) - [ ]
test_clear_buffer_empties_output— Feed text,clear_buffer(), verifyclient.output == "" - [ ]
test_clear_buffer_resets_recent— Feed text,clear_buffer(), verifyclient.recent == "" - [ ]
test_transcript_records_send_and_recv— Send a command, feed response, verify transcript has both entries in order - [ ]
test_transcript_records_correct_timestamps— Verify transcript timestamps are monotonically increasing - [ ]
test_transcript_gmcp_entries— Record GMCP send/recv, verify entries appear with correct direction
2.7 Integration Tests¶
Package:
teltest| Priority: P0 | Dependencies: 2.1–2.6
2.7.1 End-to-End Expect Flow Tests¶
These tests verify the expect methods work together in realistic multi-step scenarios using the FakeMUDClient infrastructure.
- [ ] Write tests in
packages/teltest/tests/test_expect_integration.py: -
[ ]
test_login_flow_simulation— Simulate a login flow:- Feed
"Welcome to TestMUD\r\nPlease select: [L]ogin [R]egister\r\n> " expect("Welcome")expect_prompt("> ")send("L")- Feed
"Username: " expect_prompt("Username: ")send("testuser")- Feed
"Password: " expect_prompt("Password: ")send("testpass")- Feed
"Login successful!\r\nEntering world...\r\n> " expect("Login successful")expect_prompt("> ")Verify all matches succeed and transcript has all entries in order.
- Feed
-
[ ]
test_expect_sequence_then_any— Feed a character creation flow:- Feed
"Name: \r\nRace: \r\nClass: \r\n" expect_sequence(["Name:", "Race:", "Class:"])- Feed
"Confirm? [Y/N]\r\n" send("Y")- Feed
"Character created!\r\n"or"Creation failed!\r\n" expect_any(["Character created", "Creation failed"])→ verify returns(0, ...)
- Feed
-
[ ]
test_expect_not_in_gameplay— Verify no error during command execution:send("look")- Feed
"A sunny meadow\r\nExits: north south\r\n> " expect_not("Error", until="> ")succeedsexpect("sunny meadow", from_start=True)succeeds (verifyfrom_startworks after expect_not)
-
[ ]
test_cursor_isolation_across_commands— Multi-command sequence:send("command1")- Feed
"Response 1\r\n> " expect("Response 1")send("command2")- Feed
"Response 2\r\n> " expect("Response 2")succeedsexpect("Response 1", timeout=0.1)raisesExpectTimeout(cursor past it)
-
[ ]
test_timeout_error_diagnostic_quality— Trigger a timeout and verify the error message contains:- The pattern searched for
- The timeout value
- The last 20 lines of received output
- The transcript of recent interactions
- A diagnostic hint
-
[ ]
test_connection_closed_mid_expect— Start an expect, close connection from feeder, verifyConnectionClosedraised with buffer context -
[ ]
test_transcript_completeness— Execute a multi-step flow, verifyclient.transcriptcontains every send and recv in chronological order with monotonically increasing timestamps
File Summary¶
| File | Purpose | New/Modified |
|---|---|---|
packages/teltest/src/teltest/expect.py |
Match, TranscriptEntry, Transcript, ExpectTimeout, ConnectionClosed, UnexpectedMatch, _match_pattern, _extract_line, _extract_context, _resolve_timeout, _pattern_str |
New |
packages/teltest/src/teltest/client.py |
expect(), expect_prompt(), expect_line(), expect_sequence(), expect_any(), expect_not(), output, recent, clear_buffer(), transcript, cursor management |
Modified |
packages/teltest/src/teltest/__init__.py |
Public API exports | Modified |
packages/teltest/tests/conftest.py |
FakeTransport, BufferFeeder, make_client fixture |
New/Modified |
packages/teltest/tests/test_expect.py |
All unit tests for data models, exceptions, timeout scaling, and expect methods | New |
packages/teltest/tests/test_expect_integration.py |
Integration tests for multi-step expect flows | New |
Test Summary¶
| Section | Test Count | File |
|---|---|---|
| 2.1 Data Models | 15 | test_expect.py |
| 2.2 Exceptions | 14 | test_expect.py |
| 2.3 Timeout Scaling | 5 | test_expect.py |
2.6.2 expect() |
23 | test_expect.py |
2.6.3 expect_prompt() |
9 | test_expect.py |
2.6.4 expect_line() |
9 | test_expect.py |
2.6.5 expect_sequence() |
11 | test_expect.py |
2.6.6 expect_any() |
9 | test_expect.py |
2.6.7 expect_not() |
11 | test_expect.py |
| 2.6.8 Properties/Buffer | 8 | test_expect.py |
| 2.7 Integration | 7 | test_expect_integration.py |
| Total | 121 |
Verification Checklist¶
- [ ]
uv run pytest packages/teltest/tests/test_expect.py -v— all unit tests pass - [ ]
uv run pytest packages/teltest/tests/test_expect_integration.py -v— all integration tests pass - [ ]
uv run mypy packages/teltest/src/teltest/expect.py --strict— no type errors - [ ]
uv run mypy packages/teltest/src/teltest/client.py --strict— no type errors - [ ]
uv run ruff check packages/teltest/— no lint errors - [ ] All public classes/methods have Google-style docstrings
- [ ]
ExpectTimeouterror message matches the format in design doc §Error Handling - [ ]
TELTEST_TIMEOUT_MULTIPLIER=2.0correctly doubles all timeouts - [ ] No test uses
asyncio.sleepfor synchronization (usefeed_afteror events) - [ ] All expect methods respect cursor semantics documented in design doc §2
Estimated Timeline¶
| Week | Tasks | Output |
|---|---|---|
| 1 | 2.1 (Data Models), 2.2 (Exceptions), 2.3 (Timeout Scaling) | Match, TranscriptEntry, Transcript, all 3 exceptions, timeout helpers, 33 unit tests |
| 2 | 2.4.1–2.4.4 (expect, expect_prompt, expect_line, helpers) |
Core expect loop, prompt detection, exact-line matching, 36 unit tests |
| 3 | 2.4.5–2.4.8 (expect_sequence, expect_any, expect_not, cursor/transcript) |
All remaining expect methods, cursor integration, 39 unit tests |
| 4 | 2.5 (Exports), 2.6.8 (Property tests), 2.7 (Integration) | Public API, 15 integration + property tests, verification checklist |