Skip to content

Beta Test Plan

Overview

This document provides manual test scenarios for Inquisitor v0.9.0-beta.1. The goal is to verify that all core features work correctly in a live IRC environment before promoting the beta release to a stable v1.0.0 release.

Version under test: v0.9.0-beta.1

Purpose: Verify gameplay correctness, permission enforcement, configuration management, question management, import workflows, and edge case handling across real IRC sessions with real MariaDB data.

Reporting issues: Submit bug reports to the Forgejo issue tracker at https://git.ptp-trivia.me/Inquisitor/Inquisitor/issues. Include the version (v0.9.0-beta.1), the channel and network, the exact commands run, the expected behavior, and the actual behavior.

Prerequisites

Before beginning beta testing, ensure the following:

  • Working Inquisitor installation (plugin loaded in Limnoria, @load Inquisitor)
  • MariaDB 10.5+ running and accessible
  • Database credentials configured via @config plugins.Inquisitor.database.*
  • At least one question set imported (standard questions and at least one KAOS question set)
  • At least 2 IRC clients available for multiplayer testing scenarios
  • Global setup completed (@setup) and at least one network activated (@activate)
  • Test user accounts with Gazelle-style hostmasks (e.g., [email protected])

Test Scenarios

Game Loop (Standard Questions)

  • [ ] Start a game with @trivia — bot announces game start with version banner and question set name
  • [ ] Question is delivered with: question ID (#N), category, question text, and point value, followed immediately by the first hint (fully obfuscated)
  • [ ] Type a correct answer — bot awards points, shows canonical answer, answer speed, standings, and period totals in one message
  • [ ] Observe hint progression: second hint appears after game.hintDelay seconds with partial letter reveal, showing current points and remaining time
  • [ ] Observe second hint: more letters revealed, point value halved
  • [ ] Observe third hint: maximum reveal, point value halved again
  • [ ] Let the question time out — bot reveals the correct answer
  • [ ] Observe point decay: verify 3-hint points are approximately half of 2-hint, etc.
  • [ ] Stop the game with @trivia — bot announces game stopped and no more questions delivered
  • [ ] Verify pool reshuffle: after exhausting all questions, bot announces pool reset and continues
  • [ ] Verify @recall during active question: shows question, current hint level, category, remaining points, approximate time remaining
  • [ ] Verify @recall when no game is active: bot reports no active game

KAOS Questions

  • [ ] Trigger a KAOS question (set kaos.frequency=1 to force frequent KAOS)
  • [ ] Verify KAOS display: shows topic, answer count, and point value; 1st hint shows answer count and per-answer points; 2nd+ hints show bracketed slots
  • [ ] Two players each claim different answers — both receive full base points for their claimed answers
  • [ ] Observe progressive KAOS hints: slots already claimed are shown filled, remaining show progressive letter reveal
  • [ ] One player claims all answers alone — verify NO completion bonus (bonus requires multi-player)
  • [ ] Multi-player: all answers found collectively — verify completion bonus is awarded and displayed
  • [ ] KAOS answer does NOT increment player streak
  • [ ] Non-KAOS correct answer after KAOS question properly increments streak
  • [ ] Duplicate answer claim (same answer twice by different players) — bot silently ignores, no double award
  • [ ] Set kaos.frequency=-1 (KAOS-only mode) — verify only KAOS questions are delivered
  • [ ] Set kaos.frequency=0 (disabled) — verify no KAOS questions appear in game session
  • [ ] @recall during KAOS: shows claimed answers and remaining unclaimed slots

Scramble Questions

  • [ ] Trigger a scramble question (requires scramble type questions in the question set)
  • [ ] Verify scramble display: #N Unscramble: <question text> [Category] [pts] followed by HINT: DRSECBMAL (scrambled answer in uppercase)
  • [ ] Type the exact unscrambled word — bot awards points
  • [ ] Attempt fuzzy answer (close but not exact) — bot does NOT accept it
  • [ ] Observe progressive hints: first hint is the scrambled word shown immediately, then timed hints with partial letter reveal, points, and remaining time
  • [ ] Verify scramble.hintDelay setting works independently from game.hintDelay
  • [ ] Verify scramble.questionTimeout setting works independently from game.timeoutDelay
  • [ ] Multi-word scramble: each word scrambled independently, spaces preserved

Scoring and Streaks

  • [ ] Answer multiple consecutive questions correctly — streak counter increments and is displayed
  • [ ] Another player answers a question — your streak resets to zero, no streak displayed on your next correct answer
  • [ ] Streak is NOT shown when it equals 1 (only shown when > 1)
  • [ ] Verify period totals in @stats: daily, weekly, monthly, all-time totals all update after each correct answer
  • [ ] Verify leaderboard @topday shows correct ranking with comma-formatted point values
  • [ ] Verify @topweek, @topmonth, @topever/@topall all function
  • [ ] Verify paging: @topday20 shows ranks 11-20 (second page)
  • [ ] Verify @stats <nick> shows stats for another player
  • [ ] Verify @mystats shows your own stats without typing your nick
  • [ ] Period totals reset correctly at the configured timezone boundary (daily reset at midnight per scoring.timezone)
  • [ ] Runner-up detection: second correct answer within 2 seconds of the first receives runner-up recognition (no points awarded)

Configuration

  • [ ] @config list — displays all settings organized by namespace with current values and scope indicators
  • [ ] @config get game.hintDelay — returns current value and scope (channel/network/global)
  • [ ] @config set game.hintDelay 20 — updates the setting, verify shorter/longer hint delay in next game
  • [ ] @config reset game.hintDelay — resets to next-level default
  • [ ] @config set-default scoring.minPoints 5 — changes global default, verify inherited by channels with no override
  • [ ] Constraint validation: @config set game.hintDelay 999 where 999 >= game.timeoutDelay — bot rejects with constraint violation message
  • [ ] Type validation: @config set game.hintDelay abc — bot rejects with type error
  • [ ] Unknown setting: @config get nonexistent.setting — bot reports unknown setting and suggests @config list
  • [ ] @config export — returns JSON string of current settings
  • [ ] @config import with exported JSON — re-applies same settings, no changes reported

Permissions

  • [ ] @grant <nick> PLAY — user can answer questions and view stats
  • [ ] @grant <nick> START — user can @trivia to start/stop games
  • [ ] @grant <nick> MODERATE — user can @poolreset, @setquestionset, and manage config settings
  • [ ] @grant <nick> MANAGE — user can @question add/edit/delete
  • [ ] @grant <nick> ADMIN — user has full control including @grant/@revoke
  • [ ] @revoke <nick> START — user can no longer start games; verify denial message on @trivia
  • [ ] @deny <nick> PLAY — user is blocked even from default PLAY permission; verify denial on answer
  • [ ] @undeny <nick> PLAY — denial removed; user can answer again
  • [ ] @mypermissions — shows your effective permissions with source attribution (explicit vs default)
  • [ ] @permissions <nick> — shows another user's effective permissions (requires ADMIN)
  • [ ] @listperms — lists all explicit permission grants on current network (ADMIN-only, via private notice)
  • [ ] Hierarchy: user with ADMIN can use MANAGE/MODERATE/START/PLAY commands without needing individual grants
  • [ ] Default permission: set permissions.defaultStart=true, verify users without explicit grants can @trivia

Question Management

  • [ ] @question add "Question text" "answer" --set SetName --category Cat — adds question, verify with @question show <id>
  • [ ] @question add with KAOS type: @question add "Topic" "answer1" --set SetName --type kaos --alternatives "answer2,answer3"
  • [ ] @question edit <id> --answer "new_answer" — changes answer, shows diff output answer: 'old' -> 'new'
  • [ ] @question delete <id> — shows preview requiring @confirm within 5 minutes
  • [ ] @confirm within time window — question deleted
  • [ ] @question list --set SetName — lists questions with truncated display, paginated
  • [ ] @question search "term" — returns results sorted by relevance score percentage
  • [ ] @question move <id> <new_set> — moves question to different set
  • [ ] @questionset add <name> — creates a new empty question set
  • [ ] @questionset rename <old> <new> — renames question set
  • [ ] @questionset list — lists all sets with question counts and active/inactive status
  • [ ] @questionset stats <name> — shows detailed stats on one line (total, types, categories, asked, answered)
  • [ ] @questionset delete <name> — preview and confirm required; verify cascade delete of all questions
  • [ ] @category list --set SetName — lists categories with question counts
  • [ ] Duplicate detection: add a question very similar to an existing one — bot warns about match percentage
  • [ ] @question add --force — bypasses duplicate check for intentional near-duplicate

Import

  • [ ] IRC import: @importpreview /path/to/file.ques General — shows preview with question count, duplicates found
  • [ ] @importconfirm /path/to/file.ques General — commits the import; verify questions appear in @questionset stats
  • [ ] Import with duplicate questions: preview shows duplicate count, confirmed import skips duplicates
  • [ ] @importpointspreview /path/to/points.txt — previews player score import
  • [ ] @importpointsconfirm /path/to/points.txt — commits points import
  • [ ] CLI tool: python tools/import_questions.py --config supybot.conf path/to/file.ques — imports without IRC
  • [ ] CLI --dry-run flag: reports what would be imported without writing to database
  • [ ] CLI wildcard: python tools/import_questions.py *.ques — imports multiple files
  • [ ] Resume after interruption: interrupt a large import, restart, verify checkpoint resumes from last saved position

Multi-Network

  • [ ] Activate on a second network with a different question set — verify scores are isolated (no cross-contamination)
  • [ ] Game on network A does not affect scores or state on network B
  • [ ] @deactivate on one network — verify game stops and network is deactivated; other network unaffected
  • [ ] @reactivate — verify network is reactivated and game can start again
  • [ ] Master log channel receives events from all networks (game start/stop, answer events, admin actions)
  • [ ] Per-network log channel receives only events from its own network
  • [ ] @listnetworks — shows all configured networks with their status and question sets (ADMIN)
  • [ ] @crossstats — shows combined statistics across all networks (ADMIN)
  • [ ] Per-network timezone: set different scoring.timezone on each network; verify daily reset at correct local midnight

Edge Cases

  • [ ] Multi-channel: verify two channels on the same network run independent games (separate state, separate pools)
  • [ ] Special characters in answers: answers with apostrophes, hyphens, accented characters all matched correctly
  • [ ] Long question answer: answer longer than 400 chars handled without IRC truncation errors
  • [ ] Empty question set: @trivia with a set containing no questions — bot reports no questions available
  • [ ] Rate-limited denial: trigger a permission denial, trigger again within 60 seconds — only one denial message is sent to IRC (both are logged)
  • [ ] Game active guard: attempt to change question set with @setquestionset while game is running — bot refuses with clear message
  • [ ] @question bulkdelete --set "SetName" — requires ADMIN, shows preview with count, @confirm required
  • [ ] @question bulkmove --from-set "Set1" --to-set "Set2" — bulk move requires ADMIN, preview and confirm required
  • [ ] Invalid hostmask: attempt to @grant a user whose nick is not in IRC state — bot falls back to nick!*@* pattern

Known Limitations

  • See SECURITY.md at the repository root for a full list of known security limitations, including:
    • SQL injection defenses and audit scope
    • Input validation rules and what is not validated
    • Rate limiting scope and bypass scenarios
    • File path validation rules for import commands

Additional known limitations for beta:

  • Multi-channel support on a single network is not guaranteed to be stable; single-channel per network is the tested configuration
  • Very large question sets (>100,000 questions) have not been load-tested; checkpoint imports should be used
  • Questions with extremely long text may be truncated by IRC protocol limits on some clients
  • debug.verbose generates high log volume; disable on production systems

Reporting Issues

Submit issues to the Forgejo tracker:

URL: https://git.ptp-trivia.me/Inquisitor/Inquisitor/issues

When reporting, include:

  1. Version: v0.9.0-beta.1
  2. Network and channel: e.g., EFNet #trivia
  3. Steps to reproduce: The exact sequence of IRC commands
  4. Expected behavior: What you expected to happen
  5. Actual behavior: What actually happened (include bot output if possible)
  6. Log excerpt: Relevant lines from the Limnoria bot log (if available)

Label issues with the appropriate label (see CLAUDE.md label reference) to help route them to the right area.