brAInby Spark Inventory

Changelog

What's new in brAIn. Updates, improvements, and fixes.

v0.9.0

Structured AI Responses & Confidence Accuracy

  • -All AI responses now use structured tool use — no more JSON parsing failures or missing fields
  • -Confidence scores reflect the AI's genuine assessment with no artificial penalties
  • -Gap scanner now finds files even when the AI uses shortened paths
AddedStructured tool use for all AI calls — brain doc generation, file classification, and gap selection now use schema-validated responses instead of free-form JSON
AddedForced gap resolution — when the AI reports below 90% with no questions, it's re-prompted to identify which unread files would resolve its uncertainty
AddedDebounced doc regeneration — rapid answer submissions (e.g. 5 questions via MCP) are batched into a single update
ImprovedRemoved all artificial confidence penalties — scores now reflect the AI's genuine self-assessment without double-counting
ImprovedAnswering questions triggers targeted doc refinement to incorporate answers immediately
Improvedconfidence_reason is now required — the AI must explain what was verified and what gaps remain
FixedGap scanner now uses fuzzy path matching — finds files even when the AI uses shortened paths like 'lib/storage.ts' instead of 'src/lib/storage.ts'
FixedMalformed AI question objects no longer crash the indexer
FixedAnswering review questions no longer drops confidence scores
v0.8.0

Indexing Confidence Overhaul

  • -Brain document confidence scores now reach 85%+ on first index
  • -AI agents can answer review questions via MCP to push scores higher
  • -Gap scanning now runs on all repos, not just large ones
AddedPublic changelog page at /changelog — linked from the main site navigation
AddedMCP tool `brain_answer_review_item` — agents can resolve open review questions directly to improve confidence scores
AddedCross-doc-type file sharing — important files classified to one doc type are now also available to related doc types during generation
AddedGap scanning enabled for all repo sizes — the AI can now request additional files to resolve uncertainties regardless of repo size
ImprovedSmarter file classification — explicit guidance prevents UI components from being misclassified as API endpoints and vice versa
ImprovedRe-index penalty reduced — open review questions now have less impact on confidence during fresh indexes (capped at 10% instead of 24%)
ImprovedConfidence scoring rewritten — the AI now rates whether an agent can rely on the document as a primary reference, not just whether it saw every file
ImprovedMCP capabilities list on dashboard now shows all 12 tools and 6 resources
ImprovedAI must now articulate specific gaps when confidence is below 90% — enables the gap scanner to find and read the missing files automatically
ImprovedGap scanner now uses the AI's open questions to target which additional files to read, instead of guessing
FixedDashboard no longer crashes when viewing repos with older brain documents
FixedDashboard no longer crashes when the AI returns unexpected data shapes in brain doc fields
FixedSync confidence no longer inflated — uses the AI's actual assessment instead of a hardcoded value
v0.7.0

Eval Suites & Remediation

  • -Create eval suites to test brain document accuracy with pass/fail cases
  • -Failing evals automatically create remediation work items
  • -Review queue groups open items by eval suite for efficient triage
AddedEval suite dashboard — create, edit, run, and review eval suites per repo
AddedMCP tools for eval management: `brain_run_eval_suite`, `brain_list_eval_results`, `brain_list_eval_queue`, `brain_manage_eval_suite_reviews`
AddedEval remediation queue with deep links to affected brain doc sections
AddedSuite-grouped review queue on the Evals tab
ImprovedFailing eval cases automatically create drift alerts and review items
ImprovedReindexing prioritizes files referenced by eval gaps
v0.6.0

Knowledge Entries & Proactive Context

  • -Store domain knowledge, gotchas, and business rules via MCP
  • -Pre-modification checks warn agents about gotchas before they edit files
  • -Knowledge entries are fed into brain doc generation for richer context
AddedMCP tools: `brain_store_insight`, `brain_store_insights`, `brain_list_insights`, `brain_resolve_insight`
Added`brain_check_before_modify` — returns gotchas, recent changes, and warnings before file edits
AddedKnowledge entries tab in dashboard with search, filtering, and status management
AddedAutomatic staleness detection — knowledge entries are flagged when referenced files change
ImprovedBrain doc generation incorporates active knowledge entries as additional context